A Stochastic Two-Echelon Supply Chain Model for the Petrol Station Replenishment Problem

In this paper, a new mathematical model is developed to optimize replenishment policies and inventory costs of a two-echelon supply chain system of kerosene product under demand uncertainty. The system consists of a fuel depot at the upper echelon and four petrol stations at the lower echelon. The petrol stations face stochastic stationary demand where inventory replenishment periods are uniformly fixed over the echelons. Adopting a Markov decision process approach, the states of a Markov chain represent possible states of demand for the inventory item. The replenishment cost, holding cost and shortage costs are combined with demand and inventory positions in order to generate the inventory cost matrix over the echelons. The matrix represents the long run measure of performance for the decision problem. The objective is to determine in each echelon of the planning horizon an optimal replenishment policy so that the long run inventory costs are minimized for a given state of demand. Using weekly equal intervals, the decisions of when to replenish additional units are made using dynamic programming over a finite period planning horizon. A numerical example demonstrates the existence of an optimal state-dependent replenishment policy and inventory costs over the echelons.


INTRODUCTION
The goal of a supply chain network is to procure raw materials, transform them into intermediate goods and then final products.Finally, delivery of products to customers is required through a distribution system that includes an echelon inventory system.The system spans procurement, manufacturing and distribution with inventory management as one key element.To cope with current turbulent market demands, there is still need to adopt coordinated inventory control across supply chain facilities by establishing optimal replenishment policies in a stochastic demand environment.In practice, large industries continually strive to optimize replenishment policies of products in multi-echelon inventory systems.This is a considerable challenge when the demand for manufactured items follows a stochastic trend.One major challenge is usually encountered: determining the most desirable period during which to replenish additional units of the item in question given a periodic review production-inventory system when demand is uncertain.
In this paper, a two-echelon inventory system is considered whose goal is to optimize replenishment policies and the inventory costs associated with kerosene product.At the beginning of each period, a major decision has to be made, namely whether to replenish additional units of fuel or not to replenish and keep fuel at prevailing inventory position in order to sustain demand at a given echelon.The paper is organized as follows.After reviewing the relevant literature §2, a mathematical model is described in §3 where consideration is given to the process of estimating the model parameters.The model is solved in §4 and applied to a special case study in §5.Some final remarks lastly follow in §6.Rodney and Roman (2004) examined the optimal policies study in the context of a capacitated twoechelon inventory system.This model includes installations with production capacity limits, and demonstrates that a modified base stock policy is optimal in a two-stage system when there is a smaller capacity at the downstream facility.This is shown by decomposing the dynamic programming value function into value functions dependent upon individual echelon stock variables.The optimal structure holds for both stationary and non stationary customer demand.

LITERATURE REVIEWS
Axsater S (2005) formulated a simple decision rule for decentralized two-echelon inventory control.A two-echelon distribution inventory system with a central warehouse and a number of retailers is considered.The retailers face stochastic demand and the system is controlled by continuous review installation stock policies with given batch quantities.A back order cost is provided to the warehouse and the warehouse chooses the reorder point so that the sum of the expected holding and backorder costs are minimized.Given the resulting warehouse policy, the retailers similarly optimize their costs with respect to the reorder points.The study provides a simple technique for determining the backorder cost to be used by the warehouse.Cornillier F,Boctor F,Laporte G and Renand J(2008) developed an exact algorithm for the petrol station replenishment problem.The algorithm decomposes the problem into a truck loading and routing problem.The authors determine quantities to deliver within a given interval of allocating products to tank truck compartments and of designing delivery routes to stations.In related work by Cornillier F,Boctor F,Laporte G and Renand J(2009) , a heuristic for the multi-period petrol station replenishment problem was developed.In this article, the objective is to maximize the total profit equal to the revenue minus the sum of routing costs and of regular and overtime costs.Procedures are provided for the route construction, truck loading and route packing enabling anticipation or the postponement of deliveries.The solution procedure to the problem was extended by Cornillier F,Boctor F,Laporte G and and Renand J(2009).The authors analyzed the petrol station replenishment problem with time windows.In this article, the aim is to optimize the delivery of several petroleum products to a set of petrol stations using limited heterogeneous fleet of trucks by assigning products to truck compartments, delivery routes and schedules.
In related work by Haji R (2011), a two-echelon inventory system is considered consisting of one central warehouse and a number of non-identical retailers.The warehouse uses a one-for-one policy to replenish its inventory, but the retailers apply a new policy that is each retailer orders one unit to central warehouse in a predetermined time interval; thus retailer orders are deterministic not random.
Abhijeet S and Saroj K (2011) considered a vendor managed Two-Echelon inventory system for an integrated production procurement case.Joint econom-ic lot size models are presented for the two supply situations, namely staggered supply and uniform supply.Cases are employed that describe the inventory situation of a single vendor supplying an item to a manufacturer that is further processed before it is supplied to the end user.Using illustrative examples, the comparative advantages of a uniform sub batch supply over a staggered alternative are investigated and uniform supply models are found to be comparatively more beneficial and robust than the staggered sub batch supply.

The Stochastic Two-Echelon Supply chain Model versus Petrol Station Replenishment Models
The literature cited provides profound insights by authors that are crucial in analyzing two-echelon inventory systems.Existing models that address the petrol station replenishment problem are similarly presented.Based on the existing models by scholars, a new stochastic dynamic programming approach is sought in order to relate state-transitions with customers, demand and inventory positions of the item over the echelons.This is done with a view of optimizing replenishment policies and inventory costs of the supply chain in a multi-stage decision setting.
As noted by Cornillier F,Boctor F,Larporte G and Renand J(2008,2009,2009),the three models address the petrol station replenishment problem from the transportation and logistics perspective.The source (depot) is not vividly known and the overall goal is to minimize transportation costs of petroleum products.Randomness of demand is not a salient issue or not discussed at all.However, demand uncertainty has a direct bearing in answering the inventory question of "when to deliver or replenish" at minimum inventory costs.
On a comparative note, the stochastic Two-Echelon supply chain Model incorporates demand uncertainty in determining optimal replenishment decisions where "shortage" or "no shortage" conditions are catered for when calculating total inventory costs over the echelons.The Model can assist inventory managers of petroleum products in answering the question of "when to replenish" at minimum costs under demand uncertainty.Petrol stations within a supply chain framework that share a common fuel depot can consider adopting the stochastic Two-Echelon supply chain model.As cost minimization strategy, the model provides a practical solution to replenishment decisions of petroleum products under demand uncertainty.We consider a two-echelon inventory system consisting of a fuel depot storing kerosene fuel for a designated number of petrol stations at echelon 1.At echelon 2; customers demand kerosene at petrol stations.The demand during each time period over a fixed planning horizon for a given echelon (h) is classified as either favorable (denoted by state F) or unfavorable (denoted by state U) and the demand of any such period is assumed to depend on the demand of the preceding period.The transition probabilities over the planning horizon from one demand state to another may be described by means of a Markov chain.Suppose one is interested in determining an optimal course of action, namely to replenish additional units of kerosene (a decision denoted by Z=1) or not to replenish additional units of kerosene (a decision denoted by Z=0) during each time period over the planning horizon, where Z is a binary decision variable.Optimality is defined such that the minimum inventory costs are accumulated at the end of N consecutive time periods spanning the planning horizon under consideration.In this paper, a two-echelon (h =2) and two-period (N=2) planning horizon is considered.

Finite -period dynamic programming problem formulation
Recalling that the demand can either be in state F or in state U, the problem of finding an optimal replenishment policy may be expressed as a finite period dynamic programming model.
Let g n (i , h) denote the optimal expected inventory costs accumulated during the periods n,n+1,…...,N given that the state of the system at the beginning of period n is iє{ F, U }.The recursive equation relating g n and g n+1 is (1) iє{F , U } , h={1,2} , n= 1,2,…………………….N together with the final conditions This recursive relationship may be justified by noting that the cumulative inventory costs C Z ij (h)+ g N+1 (j) resulting from reaching state j є{F, U} at the start of period n+1 from state i є{ F, U } at the start of period n occurs with probability Q where 'T' denoted matrix transposition, and hence the dynamic programming recursive equations result where (4) represents the Markov chain stable state.

Computing Q Z (h) and C Z (h)
The demand transition probability from state iє{ F, U } to state j є{ F, U },given replenishment policy Z є{ 0,1 } may be taken as the number of customers observed over echelon h with demand initially in state i and later with demand changing to state j, divided by the sum of customers over all states.That is, When demand outweighs on-hand inventory, the inventory cost matrix C Z (h) may be computed by means of the relation Therefore, for all i,jє{ F, U }, h є{1,2} and Zє{0,1}.
The justification for expression ( 6) is that D Z ij (h) -I Z ij (h) units must be replenished to meet excess demand.Otherwise replenishment is cancelled when demand is less than or equal to on-hand inventory.
The following conditions must, however hold: Z=1 when c r > 0 and Z=0 when c r = 0 c s > 0 when shortages are allowed and c s = 0 when shortages are not allowed.

OPTIMIZATION
The optimal replenishment policy and profits are found in this section for each period over echelon h separately.

Optimization during period 1
When demand is favorable (ie. in state F), the optimal replenishment policy during period 1 is The associated inventory costs are then Similarly, when demand is unfavorable (ie. in state U ), the optimal replenishment policy during period 1 is In this case, the associated inventory costs are

Optimization during period 2
Using (2),( 3) and recalling that a Z i (h)denotes the already accumulated inventory costs at the end of period 1 as a result of decisions made during that period, it follows that Therefore when demand is favorable (ie.in state F),the optimal replenishment policy during period 2 is while the associated inventory costs are Similarly, when the demand is unfavorable (ie. in state U), the optimal replenishment policy during period 2 is In this case the associated inventory costs are

CASE STUDY
In order to demonstrate use of the model in §2-3, real case applications from Total(U)Ltd, a fuel company for kerosene product and four Total petrol stations in Uganda are presented in this section.The fuel depot supplies kerosene at petrol stations (echelon 1), while end come to petrol stations for kerosene (echelon 2).The demand for kerosene fluctuates every week at both echelons.The fuel depot and petrol stations want to avoid excess inventory when demand is Unfavorable (state U) or running out of stock when demand is Favorable (state F) and hence seek decision support in terms of an optimal replenishment policy and the associated inventory cost of kerosene in a two-week planning period.The network topology of a two-echelon inventory system for kerosene is illustrated in Figure 1 below: Figure 1: A two-echelon supply chain for kerosene product hence seek decision support in terms of an optimal replenishment policy and the associated inventory cost of kerosene in a two-week planning period.The network topology of a twoechelon inventory system for kerosene is illustrated in Figure 1 Figure 1: A two-echelon supply chain for kerosene product

Data collection
Samples of customers demand and inventory levels were taken for kerosene product (in thousand litres) at echelons 1 and 2 over the state-transitions and the respective replenishment policies for twelve weeks as shown in Table 1.

Data collection
Samples of customers demand and inventory levels were taken for kerosene product (in thousand litres) at echelons 1 and 2 over the state-transitions and the respective replenishment policies for twelve weeks as shown in Table 1.In either case, the unit replenishment cost (c r ) is $1.50, the unit holding cost per week (c h ) is $0.50 and the unit shortage cost per week (c s ) is $0.75

Computation of Model Parameters
Using ( 5) and ( 6), the state transition matrices and inventory costs (in million UGX) at each respective echelon for week1 are for the case when additional units were replenished (Z=1) during week 1, while these matrices are given by for the case when additional units were not replenished (Z=0) during week 1.

The Optimal Replenishment Policy
Week1: Echelon 1 Since 111.32 < 184.25, it follows that Z=1 is an optimal replenishment policy for week 1 with associated inventory costs of $111.32 for the case of favorable demand.Since 39.01 < 64.32, it follows that Z=1 is an optimal replenishment policy for week 1 with associated inventory costs of $39.01 for the case when demand is unfavorable.
Week1: Echelon 2 Since 2.90 < 42.45, it follows that Z=0 is an optimal replenishment policy for week 1 with associated inventory costs of $2.90 when demand is favorable.Since 8.44 < 59.14, it follows that Z=0 is an optimal replenishment policy for week 1 with associated inventory costs of $8.44 when demand is unfavorable.
Week 2: Echelon 1 Since 190.95 < 329.0, it follows that Z=1 is an optimal replenishment policy for week 2 with associated accumulated inventory costs of $190.95 when demand is favorable.Since 138.12 < 203.18, it follows that Z=1 is an optimal replenishment policy for week 2 with associated accumulated inventory costs of $138.12 when demand is unfavorable.
Week 2: Echelon 2 Since 112.94 < 125.35, it follows that Z=1 is an optimal replenishment policy for week 2 with associated accumulated inventory costs of $112.94 for the case of favorable demand.Since 157.40 < 164.16, it follows that Z=1 is an optimal replenishment policy for week 2 with associated accumulated inventory costs of $157.40 for the case of unfavorable demand.

CONCLUSION
A two-echelon supply chain model with stochastic demand was presented in this paper.The model determines an optimal replenishment policy and inventory costs of kerosene product under demand uncertainty.The decision of whether or not to replenish additional units is modeled as a multi-period decision problem using dynamic programming over a finite planning horizon.Results from the model indicate optimal replenishment policies and inventory costs over the echelons for the given problem.As a cost minimization strategy in echelon-based inventory systems, computational efforts of using Markov decision process approach provide promising results for the petrol station replenishment problem.However, further extensions of research are sought in order to analyze replenishment policies that minimize inventory costs under non stationary demand conditions over the echelons.In the same spirit, the model developed raises a number of salient issues to consider: Lead time of kerosene during the replenishment cycle and customer response to abrupt changes in price of the product.Finally, special interest is thought in further extending our model by considering replenishment policies for minimum inventory costs in the context of Continuous Time Markov Chains (CTMC).