T - CC

Lecture 22. Dynamic Programming (2)Feng ChenDepartment of Industrial Engineering and LogisticsManagementShanghai Jiao Tong University20 May 2011

An Inventory Example

An Inventory ExampleA company know the demand for its product of the next four months:43.532.521.510.50month 1month 2month 3month 4demandcost for productioninventorysetup cost cost/per limit cost/per u limit3 1 max 5 0.5 max 4May 20, 2011

An Inventory Instance (cont’d)Objective: minimize the cost: production + inventoryConstraints:– Meet the demand– Inventory limitation.– Production limitation.Decision Variables:– At the beginning of each month, how many should be produced ?Assumption: At the beginning of the first month, there is noinventory.May 20, 2011

An Inventory Instance (cont’d)Dynamic Programming?????1,?2,?3,?4,?0????‣Stages?‣States?‣Recursion formulation?May 20, 2011

An Inventory Instance (cont’d)The simplest situation (From backward)01,?2,?3,?4,????I 3最 优 解 :最 优 值 :最 优 解 :最 优 值 :x( I 4)f( I 4)x( I 3)fRelation?( I 3)May 20, 2011

An Inventory InstanceLetft(i) : the minimum cost of meeting demands for months t,t+1, …, 4, if i units are on hand at the beginning of month t.☼ Here you can understand i units on hand as an input inventoryfrom previous stage).1,?2,?3,?4,?0????Let x t (i): a production level during month t thatminimizes the total cost during months t, t + 1, …,4 if i units are on hand at the beginning of month t.May 20, 2011

An Inventory Instancecost for productionsetup costcost/per3 1max 50.51,?2,?3,?4,?0????limitinventorycost/per ulimitx 4 (1)=d 4 -1 =3imax 443.532.521.510.50month 1month 2month 3month 4demand► If the input inventory i=1, we need not tomanufacture 4 to satisfy the demand of period 4,we only need to makeWe need not to make more productsin stock► with f 4 (1)=3 (setup cost) +3 (number to produce)*1 (cost)May 20, 2011

t=4During month 4, the firm will produce just enough units to ensure that0the month 4 demand of 4 units is met.f 4(0) = c(4) = 3 + 4 = $7 and x 4(0) = 4f 4(1) = c(3) = 3 + 3 = $6 and x 4(1) = 3f 4(2) = c(2) = 3 + 2 = $5 and x 4(2) = 2f 4(3) = c(1) = 3 + 1 = $4 and x 4(3) = 1f 4(4) = c(0) = 0 = $0 and x 4(4) = 0.cost for productioninventorysetup costcost/perlimitcost/per ulimit3 1max 50.5max 4May 20, 201143.532.521.510.50month 1month 2month 3month 41,?2,?3,?4,?????demand

t=3Determine f 3(i) for i = 0, 1, 2, 3, 4f 3(i) is the minimum cost incurred during months 3 and 4 given that at the1,?beginning of month 3, the on-hand inventory is i units.May 20, 2011⎧⎛1 ⎞f3( i)= min⎨⎜⎟2x ⎩⎝2 ⎠4x

An Inventory InstanceRecursion formula:f t (i)=3+x t (i)+0.5(i+x t (i) -d t )+f t+1 (i+x t (i)-d t )1,?2,?0??Production costInventory cost3,??cost for productioninventorysetup costcost/perlimitcost/per ulimit3 1max 50.5max 4May 20, 201143.532.521.510.50month 1month 2month 3month 44,??demand

x t (i) is the solution to the following programming.May 20, 2011min 3 + x + 0.5( i + x − dt) + ft+1(i + x − ds.t.x ≤ 5i + x − dti + x − dt≤ 4≥ 0i ≥ 0 and integerscost for productioninventorysetup costcost/perlimitcost/per ulimit3 1max 50.5max 443.532.521.510.50month 1month 2month 3month 4t)1,?2,?3,?4,?0????demand

t=30000i x (1/2)(i+x-2)+c(x) f 4(i+x-2)23450 + 5 = 5(1/2) + 6 = 13/21 + 7 = 8(3/2) + 8 = 19/27654Total cost:Months 3-45 + 7 = 12*(13/2)+6=25/28 + 5 = 13(19/2)+4=(27/2)f 3(i) x 3(i)f 3(0)=12x 3(0)=211111123450 + 4 = 4(1/2) + 5 = 11/21 + 6 = 7(3/2) + 7 = 17/22 + 8 = 10765404 + 7 = 11(11/2)+6=23/27 + 5 = 12(17/2)+4=(25/2)10 + 0 = 10*f 3(1)=10x 3(1)=522222012340 + 0 = 0(1/2) + 4 = 9/21 + 5 = 6(3/2) + 6 = 15/22 + 7 = 9765400 + 7 = 7*(9/2)+6=21/26 + 5 = 11(15/2)+4=(23/2)9 + 0 = 9f 3(2)=7x 3(2)=033330123(1/2) + 0 = 1/21 + 4 = 5(3/2) + 5 = 13/22 + 6 = 86540(1/2)+6=13/2*5 + 5 = 10(13/2)+4=(21/2)8 + 0 = 8f 3(3)=13/2x 3(3)=04440121 + 0 = 1(3/2) + 4 = 11/22 + 5 = 75401 + 5 = 6*(11/2)+4=(19/2)7 + 0 = 8f 3(4)=6x 3(4)=0May 20, 2011

t=2f 2(i) is the minimum cost incurred during months 2, 3 and 4 given that atthe beginning of month 2, the on-hand inventory is i units.f2( i)⎧⎛⎨⎜⎩⎝1= min33x 2⎟⎠⎞( ) ( ) ⎬ ⎫i + x − 3 + c(x)+ f i + x −⎭x

t=2i x (1/2)(i+x-3)+c(x) f 3(i+x-3)0003450 + 6 = 6(1/2) + 7 = 15/21 + 8 = 912107Total cost:Months 2-46 + 12 = 18(15/2)+10=35/29 + 7 = 16*f 2(i)x 2(i)f 2(0)=16x 2(0)=5111123450 + 5 = 5(1/2) + 6 = 13/21 + 7 = 8(3/2) + 8 = 19/21210713/25 + 12 = 17(13/2)+10=33/28 + 5 = 15*(19/2)+(13/2)=16f 2(1)=15x 2(1)=4222223333123450123434 04 14 24May 20, 320110 + 4 = 4(1/2) + 5 = 11/21 + 6 = 7(3/2) + 7 = 17/22 + 8 = 100 + 0 = 0(1/2) + 4 = 9/21 + 5 = 6(3/2) + 6 = 15/22 + 7 = 9(1/2) + 0 = 1/21 + 4 = 53/2 + 5 = 13/22 + 6 = 81210713/261210713/2610713/264 + 12 = 16(11/2)+10=31/27 + 7 = 14*(17/2)+(13/2)=1510 + 6 = 160 + 12 = 12*(9/2)+10=29/26 + 7 = 13(15/2)+(13/2)=149 + 6 = 15½ + 10 = 21/2*5 + 7 = 12(13/2)+(13/2)=138 + 6 = 14f 3(2)=7x 3(2)=0f 3(3)=12x 3(3)=0f 2(4)=21/2x 2(4)=0

t=1f 1(i) is the minimum cost incurred during months 1, 2, 3 and 4 given that at thebeginning of month 1, the on-hand inventory is i units.f1( i)⎧⎛⎨⎜⎩⎝1= min12x 2⎟⎠⎞( ) ( ) ⎬ ⎫i + x − 1 + c(x)+ f i + x −⎭x must be less than or equal to 5 (production capacity)0 ≤ (i + x - 1) ≤ 4 (ending inventory capacity)May 20, 2011

t=1i x (1/2)(i+x-1)+c(x) f 3(i+x-1)Total cost:Months 1-4f 1(i)x 1(i)00000123450 + 4 = 4(1/2) + 5 = 11/21 + 6 = 73/2 + 7 = 17/22 + 8 = 101615141221/24 + 16 = 20*(11/2)+15=41/27 + 14 = 2117/2 + 12 = 41/210 + 21/2 = 41/2f 1(0)=20x 1(0)=1Optimal solution:f 1(0) = 20, x 1(0) = 1, month 1 ending inv. = 0x 2(0) = 5, month 2 ending inv. = 2.x 3(2) = 0, month 3 ending inv. = 0.x 4(0) = 4, month 4 ending inv. = 0.May 20, 2011

An Inventory ProblemAn inventory problem with the following characteristics can beshown to be solved by using a dynamic programmingapproach.Time is divided into periods, At the beginning of period 1, thedemand during each period is known.At the beginning of each period, the firm must determine howmany units should be produced. Production capacity duringeach period is limited.May 20, 2011

An Inventory Problem (Cont’d)3. Each period's demand must be met on time from inventory orcurrent production. During any period in which productiontakes place, a fixed cost of production as well as a variableper-unit cost is incurred.4. The firm has limited storage capacity. This is reflected by alimit on end-of-period inventory. A per-unit holding cost isincurred on each period's ending inventory.5. The firm's goal is to minimize the total cost of meeting ontime the demands for periods 1, 2, …, T.This is a periodic review model (the firm's inventory position is reviewedat the end of each period, and then the production decision is made).May 20, 2011

Inventory problems –network problems2, 03, 04, 02,13,14,11, 0 2, 23, 24, 25, 02, 33, 34, 3Month 1Stage 12, 4Month 2Stage 23, 4Month 3Stage 34, 4Month 4Stage 4Month 5Stage 5May 20, 2011

General Resource Allocation Problem

Problem DescriptionSuppose we have w units of resource, and T activities (e.g.,stocks) to which the resource can be allocated.Activity t is implemented at a level x t , then g t (x t ) units of theresource are used by activity t, and a benefit r t (x t ) is obtained.Maxs.t.T∑t=1T∑t=1gtr ( xt( xtt)) ≤wMay 20, 2011

Dynamic Programming ProcedureDefine f t (d) to be the maximum benefit that can be obtainedfrom activity t, t+1,…, T if d units of the resource can beallocated to activities t, t+1,…, T.fftt( d)( d)==0xfor all dmaxt{ r ( x ) + f ( d − g ( x ))}ttt+1 t tWhere x t must be a nonnegative integer satisfying g t (x t )≤d.May 20, 2011

Lot Size Inventory Model

Lot-size ModelAn inventory model:– t=1,2,…,T period– dt is known at the beginning of period 1.– c(x)=K+cx: the cost to produce x units, c is the variable perunitcost of production, K-setup cost.– h-per unit cost of inventory every period.– Objective: Minimum cost– ct: limitation for the inventory of period t– r t : limit for the production of period t .May 20, 2011

Instance of lot-size inventory PK=250, c=2, h=1,d 1 =220, d 2 =280,d 3 =360,d 4 =140,d 5 =270i 0 =0Computational difficulty for large scale.May 20, 2011

Wagner-Whitin lemmaLemma1 : Given any optimal solution x 1 , x 2 ,…,x t to dynamic lot-size model.For x k , exists j=0,1,…,T-k, such thatxk= dk+ dk+ 1+ +dk+jcost=cx k ’j=0 1 2 … T-k1 2 … k k+1 … k+j … k+(T-k-1) Td k d k +1 d k +jx kcost=cx k ’+h(j+1)x k ’May 20, 2011

Wagner-Whitin lemmaLemma2, Given any optimal solution x 1 , x 2 ,…,x k to dynamiclot-size model. If x k >0,theni k-1

Wagner-Whitin AlgorithmRecursive formulaft{ c + f }= min tj t+j+1j=0,1,2,...,T −tctjK + c( dt+ dt+ 1+ +dt+j) + h(jdt+j+ ( j −1)dt+( j−1)+ +d+ 1)=tj=0 1 2 … T-k1 2 … K K+1 … K+j … k+(T-k-1) Td k d k +1 d k +jX kMay 20, 2011

The silver-meal Heuristic1 2 … t K+1 … K+j … k+(T-k-1) Td 1 d 2 d t d t +1 d Tx 1Heuristic: minimize average cost per period.TC(1)=K+HC(1)TC(2)=K+HC(2)TC(3)=K+HC(3)TC(t)=K+HC(t)AC( t)=HC(t)tAC( 1) ≥ AC(2)≥ AC(3)≥ AC(t −1)≥ AC(t)< AC(t + 1)May 20, 2011

The silver-meal HeuristicIn extensive testing, the S-M heuristic usually yieldeda production schedule costing less than 1% above theoptimal policy obtained by the Wagner-WhitinalgorithmMay 20, 2011

HomeworkP 516, Group A, 4p538, Group A, 2.May 20, 2011

The End