
The beer game is a widely used inclass game that is played in supply chain
management classes to demonstrate the bullwhip effect. The game is a
decentralized, multiagent, cooperative problem that can be modeled as a serial
supply chain network in which agents cooperatively attempt to minimize the
total cost of the network even though each agent can only observe its own local
information. Each agent chooses order quantities to replenish its stock. Under
some conditions, a basestock replenishment policy is known to be optimal.
However, in a decentralized supply chain in which some agents (stages) may act
irrationally (as they do in the beer game), there is no known optimal policy
for an agent wishing to act optimally.
We propose a machine learning algorithm, based on deep Qnetworks, to
optimize the replenishment decisions at a given stage. When playing alongside
agents who follow a basestock policy, our algorithm obtains nearoptimal order
quantities. It performs much better than a basestock policy when the other
agents use a more realistic model of human ordering behavior. Unlike most other
algorithms in the literature, our algorithm does not have any limits on the
beer game parameter values. Like any deep learning algorithm, training the
algorithm can be computationally intensive, but this can be performed ahead of
time; the algorithm executes in real time when the game is played. Moreover, we
propose a transfer learning approach so that the training performed for one
agent and one set of cost coefficients can be adapted quickly for other agents
and costs. Our algorithm can be extended to other decentralized multiagent
cooperative games with partially observed information, which is a common type
of situation in realworld supply chain problems.

In multiechelon inventory systems the performance of a given node is
affected by events that occur at many other nodes and in many other time
periods. For example, a supply disruption upstream will have an effect on
downstream, customerfacing nodes several periods later as the disruption
"cascades" through the system. There is very little research on stockout
prediction in singleechelon systems and (to the best of our knowledge) none on
multiechelon systems. However, in real the world, it is clear that there is
significant interest in techniques for this sort of stockout prediction.
Therefore, our research aims to fill this gap by using deep neural networks
(DNN) to predict stockouts in multiechelon supply chains.

The newsvendor problem is one of the most basic and widely applied inventory
models. There are numerous extensions of this problem. If the probability
distribution of the demand is known, the problem can be solved analytically.
However, approximating the probability distribution is not easy and is prone
to error; therefore, the resulting solution to the newsvendor problem may be
not optimal. To address this issue, we propose an algorithm based on deep
learning that optimizes the order quantities for all products based on features
of the demand data. Our algorithm integrates the forecasting and
inventoryoptimization steps, rather than solving them separately, as is
typically done, and does not require knowledge of the probability distributions
of the demand. Numerical experiments on realworld data suggest that our
algorithm outperforms other approaches, including datadriven and machine
learning approaches, especially for demands with high volatility. Finally, in
order to show how this approach can be used for other inventory optimization
problems, we provide an extension for (r,Q) policies.