Proceedings of the 1997 Winter Simulation Conference

ed. S. Andradóttir, K. J. Healy, D. H. Withers, and B. L. Nelson

INTRODUCTION TO MODELING AND SIMULATION

Anu Maria

State University of New York at Binghamton

Department of Systems Science and Industrial Engineering
Binghamton, NY 13902-6000, U.S.A.

ABSTRACT output variables is probabilistic); static (time is not taken

into account) or dynamic (time-varying interactions
This introductory tutorial is an overview of simulation among variables are taken into account). Typically,
modeling and analysis. Many critical questions are simulation models are stochastic and dynamic.
answered in the paper. What is modeling? What is
simulation? What is simulation modeling and analysis? 2 WHAT IS SIMULATION?
What types of problems are suitable for simulation? How
to select simulation software? What are the benefits and A simulation of a system is the operation of a model of
pitfalls in modeling and simulation? The intended the system. The model can be reconfigured and
audience is those unfamiliar with the area of discrete experimented with; usually, this is impossible, too
event simulation as well as beginners looking for an expensive or impractical to do in the system it represents.
overview of the area. This includes anyone who is The operation of the model can be studied, and hence,
involved in system design and modification - system properties concerning the behavior of the actual system
analysts, management personnel, engineers, military or its subsystem can be inferred. In its broadest sense,
planners, economists, banking analysts, and computer simulation is a tool to evaluate the performance of a
scientists. Familiarity with probability and statistics is system, existing or proposed, under different
assumed. configurations of interest and over long periods of real
time.
1 WHAT IS MODELING? Simulation is used before an existing system is
altered or a new system built, to reduce the chances of
Modeling is the process of producing a model; a model failure to meet specifications, to eliminate unforeseen
is a representation of the construction and working of bottlenecks, to prevent under or over-utilization of
some system of interest. A model is similar to but resources, and to optimize system performance. For
simpler than the system it represents. One purpose of a instance, simulation can be used to answer questions
model is to enable the analyst to predict the effect of like: What is the best design for a new
changes to the system. On the one hand, a model should telecommunications network? What are the associated
be a close approximation to the real system and resource requirements? How will a telecommunication
incorporate most of its salient features. On the other network perform when the traffic load increases by 50%?
hand, it should not be so complex that it is impossible to How will a new routing algorithm affect its
understand and experiment with it. A good model is a performance? Which network protocol optimizes
judicious tradeoff between realism and simplicity. network performance? What will be the impact of a link
Simulation practitioners recommend increasing the failure?
complexity of a model iteratively. An important issue in The subject of this tutorial is discrete event
modeling is model validity. Model validation techniques simulation in which the central assumption is that the
include simulating the model under known input system changes instantaneously in response to certain
conditions and comparing model output with system discrete events. For instance, in an M/M/1 queue - a
output. single server queuing process in which time between
Generally, a model intended for a simulation study arrivals and service time are exponential - an arrival
is a mathematical model developed with the help of causes the system to change instantaneously. On the
simulation software. Mathematical model classifications other hand, continuous simulators, like flight simulators
include deterministic (input and output variables are and weather simulators, attempt to quantify the changes
fixed values) or stochastic (at least one of the input or in a system continuously over time in response to

7
8 Maria

controls. Discrete event simulation is less detailed Step 5. Validate the model.
(coarser in its smallest time unit) than continuous Step 6. Document model for future use.
simulation but it is much simpler to implement, and Step 7. Select appropriate experimental design.
hence, is used in a wide variety of situations. Step 8. Establish experimental conditions for runs.
Figure 1 is a schematic of a simulation study. The Step 9. Perform simulation runs.
iterative nature of the process is indicated by the system Step 10. Interpret and present results.
under study becoming the altered system which then Step 11. Recommend further course of action.
becomes the system under study and the cycle repeats. In Although this is a logical ordering of steps in a
a simulation study, human decision making is required at simulation study, many iterations at various sub-stages
all stages, namely, model development, experiment may be required before the objectives of a simulation
design, output analysis, conclusion formulation, and study are achieved. Not all the steps may be possible
making decisions to alter the system under study. The and/or required. On the other hand, additional steps may
only stage where human intervention is not required is have to be performed. The next three sections describe
the running of the simulations, which most simulation these steps in detail.
software packages perform efficiently. The important
point is that powerful simulation software is merely a 3 HOW TO DEVELOP A SIMULATION
hygiene factor - its absence can hurt a simulation study MODEL?
but its presence will not ensure success. Experienced
problem formulators and simulation modelers and Simulation models consist of the following components:
analysts are indispensable for a successful simulation system entities, input variables, performance measures,
study. and functional relationships. For instance in a simulation
model of an M/M/1 queue, the server and the queue are
system entities, arrival rate and service rate are input
Real Simulation variables, mean wait time and maximum queue length
World Study are performance measures, and 'time in system = wait
time + service time' is an example of a functional
relationship. Almost all simulation software packages
System Simulation provide constructs to model each of the above
Under Model components. Modeling is arguably the most important
Study part of a simulation study. Indeed, a simulation study is
as good as the simulation model. Simulation modeling
comprises the following steps:
Simulation
Experiment
Step 1. Identify the problem. Enumerate problems
with an existing system. Produce requirements for a
proposed system.
Simulation
Analysis Step 2. Formulate the problem. Select the bounds
of the system, the problem or a part thereof, to be
studied. Define overall objective of the study and a few
Conclusions specific issues to be addressed. Define performance
measures - quantitative criteria on the basis of which
Altered different system configurations will be compared and
System ranked. Identify, briefly at this stage, the configurations
of interest and formulate hypotheses about system
performance. Decide the time frame of the study, i.e.,
Figure 1: Simulation Study Schematic
will the model be used for a one-time decision (e.g.,
capital expenditure) or over a period of time on a regular
The steps involved in developing a simulation
basis (e.g., air traffic scheduling). Identify the end user
model, designing a simulation experiment, and
of the simulation model, e.g., corporate management
performing simulation analysis are:
versus a production supervisor. Problems must be
Step 1. Identify the problem.
formulated as precisely as possible.
Step 2. Formulate the problem.
Step 3. Collect and process real system data.
Step 3. Collect and process real system data.
Collect data on system specifications (e.g., bandwidth for
Step 4. Formulate and develop a model.
a communication network), input variables, as well as
 Introduction to Modeling and Simulation 9

performance of the existing system. Identify sources of variables of a simulation model so that we may observe
randomness in the system, i.e., the stochastic input and identify the reasons for changes in the performance
variables. Select an appropriate input probability measures. The number of experiments in a simulation
distribution for each stochastic input variable and study is greater than or equal to the number of questions
estimate corresponding parameter(s). being asked about the model (e.g., Is there a significant
Software packages for distribution fitting and difference between the mean delay in communication
selection include ExpertFit, BestFit, and add-ons in some networks A and B?, Which network has the least delay:
standard statistical packages. These aids combine A, B, or C? How will a new routing algorithm affect the
goodness-of-fit tests, e.g., χ2 test, Kolmogorov-Smirnov performance of network B?). Design of a simulation
test, and Anderson-Darling test, and parameter experiment involves answering the question: what data
estimation in a user friendly format. need to be obtained, in what form, and how much? The
Standard distributions, e.g., exponential, Poisson, following steps illustrate the process of designing a
normal, hyperexponential, etc., are easy to model and simulation experiment.
simulate. Although most simulation software packages
include many distributions as a standard feature, issues Step 7. Select appropriate experimental design.
relating to random number generators and generating Select a performance measure, a few input variables that
random variates from various distributions are pertinent are likely to influence it, and the levels of each input
and should be looked into. Empirical distributions are variable. When the number of possible configurations
used when standard distributions are not appropriate or (product of the number of input variables and the levels
do not fit the available system data. Triangular, uniform of each input variable) is large and the simulation model
or normal distribution is used as a first guess when no is complex, common second-order design classes
data are available. For a detailed treatment of probability including central composite, Box-Behnken, and full-
distributions see Maria and Zhang (1997). factorial should be considered. Document the
experimental design.
Step 4. Formulate and develop a model. Develop
schematics and network diagrams of the system (How do Step 8. Establish experimental conditions for runs.
entities flow through the system?). Translate these Address the question of obtaining accurate information
conceptual models to simulation software acceptable and the most information from each run. Determine if the
form. Verify that the simulation model executes as system is stationary (performance measure does not
intended. Verification techniques include traces, varying change over time) or non-stationary (performance
input parameters over their acceptable range and measure changes over time). Generally, in stationary
checking the output, substituting constants for random systems, steady-state behavior of the response variable is
variables and manually checking results, and animation. of interest. Ascertain whether a terminating or a non-
terminating simulation run is appropriate. Select the run
Step 5. Validate the model. Compare the model's length. Select appropriate starting conditions (e.g., empty
performance under known conditions with the and idle, five customers in queue at time 0). Select the
performance of the real system. Perform statistical length of the warm-up period, if required. Decide the
inference tests and get the model examined by system number of independent runs - each run uses a different
experts. Assess the confidence that the end user places random number stream and the same starting conditions -
on the model and address problems if any. For major by considering output data sample size. Sample size must
simulation studies, experienced consultants advocate a be large enough (at least 3-5 runs for each configuration)
structured presentation of the model by the simulation to provide the required confidence in the performance
analyst(s) before an audience of management and system measure estimates. Alternately, use common random
experts. This not only ensures that the model numbers to compare alternative configurations by using a
assumptions are correct, complete and consistent, but separate random number stream for each sampling
also enhances confidence in the model. process in a configuration. Identify output data most
likely to be correlated.
Step 6. Document model for future use. Document
objectives, assumptions and input variables in detail. Step 9. Perform simulation runs. Perform runs
according to steps 7-8 above.
4 HOW TO DESIGN A SIMULATION
EXPERIMENT? 5 HOW TO PERFORM SIMULATION
ANALYSIS?
A simulation experiment is a test or a series of tests in
which meaningful changes are made to the input
10 Maria

Most simulation packages provide run statistics (mean, 6 AN EXAMPLE

standard deviation, minimum value, maximum value) on
the performance measures, e.g., wait time (non-time A machine shop contains two drills, one straightener, and
persistent statistic), inventory on hand (time persistent one finishing operator. Figure 2 shows a schematic of the
statistic). Let the mean wait time in an M/M/1 queue machine shop. Two types of parts enter the machine shop.
observed from n runs be W1 ,W2 ,...,Wn . It is important to
understand that the mean wait time W is a random Straightener
variable and the objective of output analysis is to
estimate the true mean of W and to quantify its
variability.
Notwithstanding the facts that there are no data
collection errors in simulation, the underlying model is Drill #1
fully known, and replications and configurations are user
controlled, simulation results are difficult to interpret. An Finishing
observation may be due to system characteristics or just Drill #2
Operator
a random occurrence. Normally, statistical inference can
assess the significance of an observed phenomenon, but
Legend:
most statistical inference techniques assume Type 1 parts
independent, identically distributed (iid) data. Most types Type 2 parts
of simulation data are autocorrelated, and hence, do not
satisfy this assumption. Analysis of simulation output Figure 2: Schematic of the Machine Shop
data consists of the following steps.
Type 1 parts require drilling, straightening, and finishing
in sequence. Type 2 parts require only drilling and
Step 10. Interpret and present results. Compute
finishing. The frequency of arrival and the time to be
numerical estimates (e.g., mean, confidence intervals) of
routed to the drilling area are deterministic for both types
the desired performance measure for each configuration
of parts.
of interest. To obtain confidence intervals for the mean
of autocorrelated data, the technique of batch means can
be used. In batch means, original contiguous data set
Step 1. Identify the problem. The utilization of
from a run is replaced with a smaller data set containing
drills, straightener, and finishing operator needs to be
the means of contiguous batches of original observations.
assessed. In addition, the following modification to the
The assumption that batch means are independent may
original system is of interest: the frequency of arrival of
not always be true; increasing total sample size and
both parts is exponential with the same respective means
increasing the batch length may help.
as in the original system.
Test hypotheses about system performance.
Construct graphical displays (e.g., pie charts, histograms)
Step 2. Formulate the problem. The objective is to
of the output data. Document results and conclusions.
obtain the utilization of drills, straightener, and finishing
operator for the original system and the modification. The
Step 11. Recommend further course of action. This
assumptions include:
may include further experiments to increase the precision
♦ The two drills are identical
and reduce the bias of estimators, to perform sensitivity
analyses, etc. ♦ There is no material handling time between the three
operations.
♦ Machine availability implies operator availability.
♦ Parts are processed on a FIFO basis.
♦ All times are in minutes.

Step 3. Collect and process real system data. At

the job shop, a Type 1 part arrives every 30 minutes, and a
Type 2 part arrives every 20 minutes. It takes 2 minutes to
route a Type 1 part and 10 minutes to route a Type 2 part
to the drilling area. Parts wait in a queue till one of the two
drilling machines becomes available. After drilling, Type 1
parts are routed to the straightener and Type 2 parts are
 Introduction to Modeling and Simulation 11

routed to the finishing operator. After straightening, Type underutilized. Consequently, the average utilization did not
1 parts are routed to the finishing operator. change substantially between the original system and the
The operation times for either part were determined to modification; the standard deviation of the drilling
be as follows. Drilling time is normally distributed with operation seems to have increased because of the increased
mean 10.0 and standard deviation 1.0. Straightening time randomness in the modification. The statistical
is exponentially distributed with a mean of 15.0. Finishing significance of these observations can be determined by
requires 5 minutes per part. computing confidence intervals on the mean utilization of
the original and modified systems.
Step 4. Formulate and develop a model. A model
of the system and the modification was developed using Step 11. Recommend further course of action. Other
a simulation package. A trace verified that the parts performance measures of interest may be: throughput of
flowed through the job shop as expected. parts for the system, mean time in system for both types of
parts, average and maximum queue lengths for each
Step 5. Validate the model. The utilization for a operation. Other modifications of interest may be: the
sufficiently long run of the original system was judged to flow of parts to the machine shop doubles, the finishing
be reasonable by the machine shop operators. operation will be repeated for 10% of the products on a
probabilistic basis.
Step 6. Document model for future use. The
models of the original system and the modification were 7 WHAT MAKES A PROBLEM SUITABLE FOR
documented as thoroughly as possible. SIMULATION MODELING AND ANALYSIS?

Step 7. Select appropriate experimental design. In general, whenever there is a need to model and
The original system and the modification described analyze randomness in a system, simulation is the tool of
above were studied. choice. More specifically, situations in which simulation
modeling and analysis is used include the following:
Step 8. Establish experimental conditions for runs. ♦ It is impossible or extremely expensive to observe
Each model was run three times for 4000 minutes and certain processes in the real world, e.g., next year's
statistical registers were cleared at time 1000, so the cancer statistics, performance of the next space
statistics below were collected on the time interval [1000, shuttle, and the effect of Internet advertising on a
4000]. At the beginning of a simulation run, there were no company's sales.
parts in the machine shop. ♦ Problems in which mathematical model can be
formulated but analytic solutions are either
Step 9. Perform simulation runs. Runs were impossible (e.g., job shop scheduling problem, high-
performed as specified in Step 8 above. order difference equations) or too complicated (e.g.,
complex systems like the stock market, and large
Step 10. Interpret and present results. Table 1 scale queuing models).
contains the utilization statistics of the three operations for ♦ It is impossible or extremely expensive to validate
the original system and the modification (in parentheses). the mathematical model describing the system, e.g.,
due to insufficient data.
Table 1: Utilization Statistics Applications of simulation abound in the areas of
Drilling Straightening Finishing government, defense, computer and communication
Mean Run #1 0.83 (0.78) 0.51 (0.58) 0.42 (0.39)
systems, manufacturing, transportation (air traffic
Mean Run #2 0.82 (0.90) 0.52 (0.49) 0.41 (0.45)
Mean Run #3 0.84 (0.81) 0.42 (0.56) 0.42 (0.40)
control), health care, ecology and environment,
Std. Dev. Run #1 0.69 (0.75) 0.50 (0.49) 0.49 (0.49) sociological and behavioral studies, biosciences,
Std. Dev. Run #2 0.68 (0.78) 0.50 (0.50) 0.49 (0.50) epidemiology, services (bank teller scheduling),
Std. Dev. Run #3 0.69 (0.76) 0.49 (0.50) 0.49 (0.49) economics and business analysis.

Mean utilization represents the fraction of time a server is 8 HOW TO SELECT SIMULATION
busy, i.e., busy time/total time. Furthermore, the average SOFTWARE?
utilization output for drilling must be divided by the
number of drills in order to get the utilization per drill. Although a simulation model can be built using general
Each drill is busy about 40% of the time and straightening purpose programming languages which are familiar to
and finishing operations are busy about half the time. This the analyst, available over a wide variety of platforms,
implies that for the given work load, the system is and less expensive, most simulation studies today are
implemented using a simulation package. The
12 Maria

advantages are reduced programming requirements; interest, and observing the system's operation in
natural framework for simulation modeling; conceptual detail over long periods of time.
guidance; automated gathering of statistics; graphic ♦ Test hypotheses about the system for feasibility.
symbolism for communication; animation; and ♦ Compress time to observe certain phenomena over
increasingly, flexibility to change the model. There are long periods or expand time to observe a complex
hundreds of simulation products on the market, many phenomenon in detail.
with price tags of $15,000 or more. Naturally, the ♦ Study the effects of certain informational,
question of how to select the best simulation software for organizational, environmental and policy changes on
an application arises. Metrics for evaluation include the operation of a system by altering the system's
modeling flexibility, ease of use, modeling structure model; this can be done without disrupting the real
(hierarchical v/s flat; object-oriented v/s nested), code system and significantly reduces the risk of
reusability, graphic user interface, animation, dynamic experimenting with the real system.
business graphics, hardware and software requirements, ♦ Experiment with new or unknown situations about
statistical capabilities, output reports and graphical plots, which only weak information is available.
customer support, and documentation. ♦ Identify the "driving" variables - ones that
The two types of simulation packages are simulation performance measures are most sensitive to - and the
languages and application-oriented simulators (Table 2). inter-relationships among them.
Simulation languages offer more flexibility than the ♦ Identify bottlenecks in the flow of entities (material,
application-oriented simulators. On the other hand, people, etc.) or information.
languages require varying amounts of programming ♦ Use multiple performance metrics for analyzing
expertise. Application-oriented simulators are easier to system configurations.
learn and have modeling constructs closely related to the ♦ Employ a systems approach to problem solving.
application. Most simulation packages incorporate
♦ Develop well designed and robust systems and
animation which is excellent for communication and can
reduce system development time.
be used to debug the simulation program; a "correct
looking" animation, however, is not a guarantee of a
10 WHAT ARE SOME PITFALLS TO GUARD
valid model. More importantly, animation is not a
AGAINST IN SIMULATION?
substitute for output analysis.
Simulation can be a time consuming and complex
Table 2: Simulation Packages
exercise, from modeling through output analysis, that
Type Of Examples
Simulation necessitates the involvement of resident experts and
Package decision makers in the entire process. Following is a
Simulation Arena (previously SIMAN), AweSim! (previously checklist of pitfalls to guard against.
languages SLAM II), Extend, GPSS, Micro Saint, ♦ Unclear objective.
SIMSCRIPT, SLX
Object-oriented software: MODSIM III, SIMPLE++ ♦ Using simulation when an analytic solution is
Animation software: Proof Animation appropriate.
Application Manufacturing: AutoMod, Extend+MFG, ♦ Invalid model.
-Oriented FACTOR/AIM, ManSim/X, MP$IM,
♦ Simulation model too complex or too simple.
Simulators ProModel, QUEST, Taylor II, WITNESS
Communications/computer: COMNET III, ♦ Erroneous assumptions.
NETWORK II.5, OPNET Modeler, OPNET ♦ Undocumented assumptions. This is extremely
Planner, SES/Strategizer, SES/workbench important and it is strongly suggested that
Business: BP$IM, Extend+BPR, ProcessModel,
ServiceModel, SIMPROCESS, Time machine
assumptions made at each stage of the simulation
Health Care: MedModel modeling and analysis exercise be documented
thoroughly.
9 BENEFITS OF SIMULATION MODELING ♦ Using the wrong input probability distribution.
AND ANALYSIS ♦ Replacing a distribution (stochastic) by its mean
(deterministic).
According to practitioners, simulation modeling and ♦ Using the wrong performance measure.
analysis is one of the most frequently used operations ♦ Bugs in the simulation program.
research techniques. When used judiciously, simulation ♦ Using standard statistical formulas that assume
modeling and analysis makes it possible to: independence in simulation output analysis.
♦ Obtain a better understanding of the system by ♦ Initial bias in output data.
developing a mathematical model of a system of ♦ Making one simulation run for a configuration.
 Introduction to Modeling and Simulation 13

♦ Poor schedule and budget planning.

♦ Poor communication among the personnel involved
in the simulation study.

REFERENCES

Banks, J., J. S. Carson, II, and B. L. Nelson. 1996.

Discrete-Event System Simulation, Second Edition,
Prentice Hall.
Bratley, P., B. L. Fox, and L. E. Schrage. 1987. A Guide
to Simulation, Second Edition, Springer-Verlag.
Fishwick, P. A. 1995. Simulation Model Design and
Execution: Building Digital Worlds, Prentice-Hall.
Freund, J. E. 1992. Mathematical Statistics, Fifth Edition,
Prentice-Hall.
Hogg, R. V., and A. T. Craig. 1995. Introduction to
Mathematical Statistics, Fifth Edition, Prentice-Hall.
Kleijnen, J. P. C. 1987. Statistical Tools for Simulation
Practitioners, Marcel Dekker, New York.
Law, A. M., and W. D. Kelton. 1991. Simulation
Modeling and Analysis, Second Edition,
McGraw-Hill.
Law, A. M., and M. G. McComas. 1991. Secrets of
Successful Simulation Studies, Proceedings of the
1991 Winter Simulation Conference, ed. J. M.
Charnes, D. M. Morrice, D. T. Brunner, and J. J.
Swain, 21-27. Institute of Electrical and Electronics
Engineers, Piscataway, New Jersey.
Maria, A., and L. Zhang. 1997. Probability Distributions,
Version 1.0, July 1997, Monograph, Department of
Systems Science and Industrial Engineering, SUNY
at Binghamton, Binghamton, NY 13902.
Montgomery, D. C. 1997. Design and Analysis of
Experiments, Third Edition, John Wiley.
Naylor, T. H., J. L. Balintfy, D. S. Burdick, and K. Chu.
1966. Computer Simulation Techniques, John Wiley.
Nelson, B. L. 1995. Stochastic Modeling: Analysis and
Simulation, McGraw-Hill.

AUTHOR BIOGRAPHY

ANU MARIA is an assistant professor in the department

of Systems Science & Industrial Engineering at the State
University of New York at Binghamton. She received
her PhD in Industrial Engineering from the University of
Oklahoma. Her research interests include optimizing the
performance of materials used in electronic packaging
(including solder paste, conductive adhesives, and
underfills), simulation optimization techniques, genetics
based algorithms for optimization of problems with a
large number of continuous variables, multi criteria
optimization, simulation, and interior-point methods.