Introduction To Modeling and Simulation Techniques
Introduction To Modeling and Simulation Techniques
Proceedings Paper:
Yin, C and McKay, A orcid.org/0000-0002-8187-4759 (2018) Introduction to Modeling and
Simulation Techniques. In: Proceedings of ISCIIA 2018 and ITCA 2018. The 8th
International Symposium on Computational Intelligence and Industrial Applications and
The 12th China-Japan International Workshop on Information Technology and Control
Applications, 02-06 Nov 2018, Tengzhou, China. .
This is an author produced version of a paper presented at ISCIIA 2018 and ITCA 2018.
Reuse
Items deposited in White Rose Research Online are protected by copyright, with all rights reserved unless
indicated otherwise. They may be downloaded and/or printed for private study, or other acts as permitted by
national copyright laws. The publisher or other rights holders may allow further reproduction and re-use of
the full text version. This is indicated by the licence information on the White Rose Research Online record
for the item.
Takedown
If you consider content in White Rose Research Online to be in breach of UK law, please notify us by
emailing eprints@whiterose.ac.uk including the URL of the record and the reason for the withdrawal request.
eprints@whiterose.ac.uk
https://eprints.whiterose.ac.uk/
Introduction to Modeling and Simulation Techniques
Abstract: Modeling and simulation techniques are simulation (DES) method [5], and Macal and North
becoming an important research method for proposed an agent-based simulation (ABS) tutorial [4].
investigating operational and organizational systems. Siebers et. al. presented advantages and disadvantages
Many literatures report different aspects and views of between DES and ABS, Sargent considered different
modeling and simulation but there is little literature that approaches for simulation model verification and validation
covers a full cycle of modeling and simulation, including [7], Hughes et. al. reported modeling and simulation
both model design & development and model applications to organizational systems [8], and Abar et. al.
verification & validation, for use in industrial product provided a review of agent-based simulation methods and
development systems. This paper introduces modeling development [9].
and simulation concepts, methods and tools, and Many researchers work on modeling and simulation
discusses approaches that can be used for model methods, procedures, strategies and applications in different
verification and validation. A modeling and simulation scientific research areas. However, there is little literature
procedure, designed for use in understanding industrial that covers a full cycle of modeling and simulation,
product development systems, is introduced that including both model design & development and model
accommodates both model creation and verification & verification & validation, for use in industrial product
validation. The overall goal of the research is to bridge development systems. As a result, it can be difficult for
the gap between model design & development and model practitioners to determine the validity of given simulation
verification & validation in a modeling and simulation models and so the reliability of results from simulation
procedure which, as a whole, is essential for the experiments.
application of modeling and simulation techniques to This paper introduces a procedure (see Section 5) that
understand any real-world system. covers a full cycle of modeling and simulation, including
both model design & development and model verification &
Keywords: Modeling and Simulation, Modeling and validation, for use in industrial product development
Simulation Procedure, Model Verification and systems. The procedure was evaluated through application
Validation, Agent-Based Simulation (ABS), Discrete- to a real-world new product development process case study
Event Simulation (DES) as part of a PhD research project [48]. The procedure is
based on modeling and simulation concepts discussed in
Section 2; and modeling and simulation domains and
methods that are introduced in Sections 3 and 4 respectively.
1. BACKGROUND Section 6 considers model verification and validation
methods in more details and Section 7 concludes the paper.
Modeling and simulation techniques are being widely
applied in organizational and operational systems, in
2. MODELING AND SIMULATION CONCEPTS
addition to their success in physical system design,
manufacture, analysis and improvement. Modeling and Two definitions of modeling and simulation were used as
simulation involves a process of designing a model of a real- the basis of this work. Modeling and simulation is defined
world or anticipated system such as a design concept, then by Bratley et. al. as a process of driving a model of a system
conducting experiments with the model for the purposes of with suitable inputs and observing the correspondingly
understanding the performance of the system under outputs [10] and by Shannon as the process of designing a
different operating conditions and evaluating alternative model of a conceptual system and using it to conduct
management strategies and decision-making processes [1, experiments for the purpose of understanding the
2]. Modeling and simulation technology is increasingly performance of the system and/or evaluating alternative
considered to be a third scientific research methodology, in management strategies and decision-making processes
addition to the traditional deductive and inductive using simulation results [1, 2].
approaches [3, 4]. The purpose of modeling and simulation includes
Many researchers have contributed to modeling and performance assessment, proof, prediction, discovery,
simulation technologies. For example, Shannon gave a training, entertainment and education [3]. Simulation
definition of simulation and predictive modeling [1], techniques are applied in various research fields including
Klingstam and Gullander introduced the discrete-event computer systems, manufacturing processes, societal
The 8th International Symposium on Computational Intelligence and Industrial Applications (ISCIIA2018)
The 12th China-Japan International Workshop on Information Technology and Control Applications (ITCA2018) 1
Binjiang International Hotel, Tengzhou, Shandong, China, Nov. 2-6, 2018
Introduction to Modeling and Simulation Techniques
systems, business organizations, government systems, additional time resources for design iteration across
ecology environment systems, and other complex processes different stages in the process with a view to identifying
and systems [1, 2]. Modeling and simulation methods have improved management strategies, with an overall goal to
also been applied to interdisciplinary research fields such as shorten product development duration and so improve
design system decision-making mechanisms [11, 12], the time-to-market performance. The focus of the rest of this
management of integrated product teams [13], new product paper is on process and system modeling and simulation.
development processes [14, 15, 16], and organizational
management [8]. Application of modeling and simulation
methods to understand the performance of complex socio-
technical systems is becoming a promising research area [3,
8].
The 8th International Symposium on Computational Intelligence and Industrial Applications (ISCIIA2018)
The 12th China-Japan International Workshop on Information Technology and Control Applications (ITCA2018) 3
Binjiang International Hotel, Tengzhou, Shandong, China, Nov. 2-6, 2018
Introduction to Modeling and Simulation Techniques
3. Collect data & information: the data and information enhances confidence for users applying the model to
needed for the definition of both conceptual and solve the real-world problems. In the case study used to
simulation models is specified and collected as input evaluate this process, a user manual for operating the
data for the simulation experiment. It should be noted simulation model was developed.
that this is typically an iterative negotiation process
between the research team and case study owners 6. MODEL VERIFICATION AND VALIDATION
because the necessary data needs to exist in an
The purpose of model verification and validation is to make
appropriate form and be accessible. If it is not then case
the simulation model meaningful in a real-world context.
study owners might work with the research team to
For this reason, the modeling and simulation procedure
develop a synthetic data set.
includes model verification and validation, in addition to
4. Build up conceptual model: a conceptual model is
model design and development. Fig. 3 gives an example of
defined and built up with respect to specified research
full vision of a model verification and validation
purpose using data and information collected, in order
architecture. Model verification and validation activities
to represent relationships that are relevant to the
include validation of the simulation model with respect to
research problem.
the real-world situation and the conceptual model.
5. Select simulation method: a modeling and simulation
method is selected to represent the defined research 6.1. Model Verification and Validation Concepts
problem. Selection of the simulation method includes Model verification concerns the identification and removal
consideration of both suitability and feasibility. of errors in the simulation model by comparing simulation
6. Choose simulation tool: the software tool in which the results from the model to analytical solutions from the real-
simulation model will be implemented is selected. This world situation [34]. In this way, the model verification
selection process includes consideration of both tool process deals with the mathematical relationships and
availability and adaptability. simulation specifications associated with the model [35, 36].
7. Develop simulation model: a computer-based Model verification ensures that the model is as complete and
simulation model of the conceptual model is developed correct as is necessary to give a sufficiently accurate
using the selected simulation method and tool. representation of the real-world situation [37, 38]. In this
8. Conduct verification experiments: verification way, model validation ensures that the simulation model is
experiments are conducted with the simulation model, useful for real-world problem-solving [35]. Model
with a focus on checking whether the simulation model validation processes are concerned with quantifying the
gives reliable and anticipated outputs for given inputs accuracy of the model by comparing simulation results to
in scenarios with which the case study owners are experimental or operational outcomes in the real world [36].
familiar. Contributors to model verification and validation
9. Verify the simulation model: simulation results from knowledge domains include [7, 39, 40, 41, 42, 43, 44, 45].
verification experiments are reviewed; the simulation
model and results are verified against specified 6.2. Verification and Validation Architecture
verification methods and indicators. If necessary, steps Fig. 4 illustrates a model verification and validation
3, 4, 5, 6, 7 and 8 may be revisited. By addressing architecture for the implementation of relevant activities in
comments, feedbacks, and suggestions from different Fig. 3.
perspectives, the simulation model is improved and
upgraded for the next stage.
10. Conduct validation experiments: validation
experiments are conducted using revised simulation
model. Validation experiments are to check whether the
simulation model possesses sufficient accuracy to
represent and then address the research problem, with
respect to the specified research purpose.
11. Validate the simulation model: results from the
validation experiments are validated against specified
validation methods and indicators. If necessary, earlier
steps may be revisited.
12. Simulation experiments: simulation experiments are
conducted to simulate real-world operational scenarios.
Simulation results are analyzed and discussed. Potential
management solutions are considered to address the
specified research problems. Fig. 4 Model verification and validation architecture
13. Documentation: instructions and documents
supporting the simulation model and simulation The oblongs in Fig. 4 represent outcomes from three
experiments are developed, e.g. how to operate the primary steps in Fig. 3: the definition of the research
simulation model, how to set input data values, and how problem from the research problem definition (Step 1), the
to analyze model results. This step is necessary for other conceptual model (from Step 4) and the simulation model
users or clients to understand, modify, or further (from Step 7).
improve the simulation model if necessary. It also
The 8th International Symposium on Computational Intelligence and Industrial Applications (ISCIIA2018)
The 12th China-Japan International Workshop on Information Technology and Control Applications (ITCA2018) 4
Binjiang International Hotel, Tengzhou, Shandong, China, Nov. 2-6, 2018
Introduction to Modeling and Simulation Techniques
As shown in Fig. 4, three verification and validation Different model verification and validation methods have
activities are needed. Firstly, conceptual model verification different advantages and limitations, which means, again,
ensures that the conceptual model is an accurate that different methods are suitable for different real-world
representation of the research problem in real-world problem situations and research purposes. Regarding model
situation. Secondly, simulation model verification ensures verification and validation method selection, more
that the computer-based simulation model is a sufficiently information can be found in publications cited above.
accurate implementation of the concept model. And finally,
simulation model validation conducts a series of simulation 7. CONCLUSIONS
experiments which focus on checking the model’s
Modeling and simulation techniques are becoming an
efficiency and accuracy with respect to specific research
purpose. In addition, at the center of the diagram in Fig. 4, important research method for investigating operational and
all data used in all aspects of both model design & organizational systems. Existing literatures report different
development and model verification & validation needs to views and aspects of modeling and simulation techniques
be validated. with specified interests, but there is limited literature that
presents a full vision of modeling and simulation in a
6.3. Model Verification and Validation Strategies
procedure suitable for engineering design applications. The
Researchers have developed different model verification contribution of this paper is to bridge the gap between model
and validation strategies with their experience in either design and development, and model verification and
academic research or industry. Contributions to this
validation by providing such a modeling and simulation
knowledge domain include [7, 36, 39, 41, 46, 47]. There are
four primary strategies used to verify and validate procedure.
simulation models [7]: A modeling and simulation procedure was demonstrated
with focus on a real-world problem-solving case study. The
• Self-Validation: The simulation model development procedure accommodates model verification & validation
team itself makes the decision as to whether a activities into model design & development process which,
simulation model is valid or not;
as a whole, forms a full cycle of modeling and simulation.
• Co-Validation: The simulation team involves model
This paper discussed modeling and simulation concepts,
users within model development process; the model
validation process is integrated within the model domains, methods and processes. Two common modeling
development process; and simulation methods were discussed, i.e. agent-based
• Independent Validation: An independent third party simulation (ABS) and discrete-event simulation (DES). In
is employed to decide whether a simulation model is addition, model verification and validation concepts,
valid or not; and strategies and methods were evaluated. The research is
• Scoring Validation: A scoring model is used to expected to be helpful for researchers and practitioners
determine whether a simulation model is valid or not. working in modeling and simulation fields.
Each strategy has distinct features which means that
different strategies are suitable for different real-world
situations and simulation purposes. Detailed explanation of ACKNOWLEDGEMENTS
model verification and validation strategies is available in The research is supported by China 13th Five-Year National
references listed above. Key Research and Development Plan (Project No.:
6.4. Model Verification and Validation Methods 2017YFD0701102-1). The work reported in this paper was
part of a PhD project that was co-supervised by Professor
There are many model verification and validation methods
Chris W. Clegg (deceased).
developed for specific simulation situations. Contributions
to the model verification and validation methods
community include [7, 39, 40, 41, 42, 43, 44, 45]. A series
of model verification and validation methods are listed as REFERENCES:
follows. [1] R. E. Shannon, Simulation: An overview, Winter Simulation
Conference, Arlington, Virginia, USA, 1983.
• Animation Validation [2] R. E. Shannon, Introduction to simulation, Winter Simulation
• Model to Model Validation Conference, Arlington, Virginia, USA, 1992.
• Event Validation [3] R. Axelrod, Advancing the Art of Simulation in the Social Sciences,
Japanese Journal for Management Information System, Special Issue
• Extreme Condition Validation on Agent-Based Modeling, 2003, 12(3).
• Face Validation [4] C. M. Macal and M. J. North, Agent-based modeling and simulation:
• Historical Data Validation Desktop ABMS, Winter Simulation Conference, 2007.
• Operational Graphics Validation [5] P. Klingstam and P. Gullander, Overview of simulation tools for
computer-aided production engineering, Computers in Industry, 1999,
• Sensitivity Analysis Validation 38(2), pp. 173-186.
• Predictive Validation [6] P. O. Siebers, C. M. Macal, J. Garnett, D. Buxton, and M. Pidd,
• Traces Validation Discrete-event simulation is dead, long live agent-based simulation!
Journal of Simulation, 2010, 4(3), pp. 204-210.
• Turing Test Validation
[7] R. G. Sargent, Verification and validation of simulation models,
• Game Validation Journal of Simulation, 2013, 7(1), pp. 12-24.
The 8th International Symposium on Computational Intelligence and Industrial Applications (ISCIIA2018)
The 12th China-Japan International Workshop on Information Technology and Control Applications (ITCA2018) 5
Binjiang International Hotel, Tengzhou, Shandong, China, Nov. 2-6, 2018
Introduction to Modeling and Simulation Techniques
[8] H. P. N. Hughes, C. W. Clegg, M. A. Robinson, and R. M. Crowder, [32] C. D. Pegden, R.E. Shannon, and R.P. Sadowski, Introduction to
Agent-based modeling and simulation: The potential contribution to Simulation using SIMAN (Second Edition), New York: McGraw-
organizational psychology, Journal of Occupational and Hill, 1995.
Organizational Psychology, 2012, 85(3), pp. 487-502. [33] A. F. Seila, Introduction to simulation, Winter Simulation Conference,
[9] S. Abar, K. G. Theodoropoulos, P. Lemarinier, and M. P. G. O’Hare, 1995.
Agent Based Modeling and Simulation tools: A review of the state- [34] A. M. Law, Simulation modeling and analysis (4th Edition), New
of-art software, Computer Science Review, 2017. York: McGraw-Hill, 2007.
[10] P. Bratley, B. L. Fox, and L. E. Schrage, A guide to simulation (2nd [35] C. M. Macal, Model verification and validation, Workshop on Threat
Ed.), New York, Springer, 1987. anticipation: Social science methods and models, University of
[11] H. M. E. Abdelsalam and H. P. Bao, A simulation-based optimization Chicago and Argonne National Laboraory, 2005.
framework for product development cycle time reduction, [36] B. H. Thacker, S. W. Doebling, F. M. Hemez, M. C. Anderson, J. E.
Engineering Management, 2006, 53(1), pp. 69-85. Pepin, and E. A. Rodriguez, Concepts of Model Verification and
[12] L. T. T. Pesonen, S. J. Salminen, J.-P. Ylen, and P. Riihimaki, Validation, 2004.
Dynamic simulation of product process, Simulation Modeling [37] O. Balci, Verification validation and accreditation of simulation
Practice and Theory, 2008, 16(8), pp. 1091-1102. models, Winter simulation, Atlanta, Georgia, USA, 1997.
[13] Y. W. Sim, R. Crowder, M. Robinson, and H. Hughes, An Agent- [38] D. S. Riha, B. H. Thacker, J. B. Pleming, J. D. Walker, S. A. Mullin,
based approach to modeling integrated product teams undertaking a et al., Verification and validation for a penetration model using a
design activity, ASME 2009 International Design Engineering deterministic and probabilistic design tool. International Journal of
Technical Conferences and Computers and Information in Impact Engineering, 2006, 33(1–12), pp. 681-690.
Engineering Conference, San Diago, CA, USA, 2009.
[39] J. S., II. Carson, Model verification and validation, Winter Simulation
[14] N. Bhuiyan, D. Gerwin, and V. Thomson, Simulation of the New Conference, 2002.
Product Development Process for Performance Improvement,
Management Science, 2004, 50(12), pp. 1690-1703. [40] C. M. Macal and M. J. North, Validation of an agent-based model of
deregulated electric power markets, North American Computational
[15] R. Garcia, Uses of Agent-Based Modeling in Innovation/New Social and Organization Science Conference, 2005.
Product Development Research*, Journal of Product Innovation
Management, 2005, 22(5), pp. 380-398. [41] S. Robinson, Simulation model verification and validation:
increasing the users' confidence, the 29th Winter Simulation
[16] D. C. Wynn, C. M. Eckert, and P. J. Clarkson, Modeling interation in Conference, Atlanta, Georgia, USA, 1997.
engineering design, International conference on engineering design,
[42] X. Xiang, R. Kennedy, G. Madey, and S. Cabaniss, Verification and
Paris, 2007.
Validation of Agent-based Scientific Simulation Models, ADS, 2005.
[17] J. Barbosa and P. Leitao, Simulation of Multi-agent Manufacturing
[43] N. Gilbert, Model building and Validation, Siumulation Innovation:
Systems using Agent-based Modeling Platforms, the 9th IEEE
a Node, University of Surry, England, 2010.
International Conference on Industrial Informatics, 2011.
[18] M. Pereda and J. M. Zamarreno, Agent-based modeling of an [44] O. Balci, Golden rules of verification, validation, testing, and
activated sludge process in a batch reactor, the 19th Mediterranean certification of modeling and simulation applications, SCS Modeling
Conference on Control & Automation, 2011. & Simulation Magazine, 2010.
[19] O. Nicolae and G. Wagner, Modeling and simulating organisations, [45] O. Balci, A life cycle for modeling and simulation, Journal of
the 7th International Workshop on Enterprise & Organizational Simulation, 2012, 88(7), pp. 870-883.
Modeling and Simulation, London, 2011. [46] O. Balci, Validation, verification, and testing techniques throughout
[20] E. Bonabeau, Agent-based modeling: methods and techniques for the life cycle of a simulation study, Annals of Operations Research,
simulation human systems, the National Academy of Sciences, CA, 1994, 53(1), pp. 121-173.
2002. [47] J. Banks, J. S. Carson, B. L. Nelson, and D. M. Nicol, Discrete-event
[21] D. Kibira and C. R. McLean, Generic simulation of automotive system simulation (5th Ed.), NJ: Upper Saddle River, 2010.
assembly for interoperability testing, Winter Simulation Conference, [48] C.-G. Yin, Simulating vicious circles in new product introduction
2007. systems, PhD thesis, University of Leeds, Leeds, UK, 2014
[22] S.-H. Cho and S. Eppinger, Product development process modeling
using advanced simulation, 2001.
[23] C. M. Macal and M. J. North, Tutorial on agent-based modeling and
simulation, Journal of Simulation, 2010, 4(3), pp. 151-162. AUTHOR BIOGRAPHIES
[24] A. Garro and W. Russo, easyABMS: A domain-expert oriented Chenggang Yin is a Lecturer in Department of Mechancial Design and
methodology for agent-based modeling and simulation, Simulation Manufacture, China Agricultural University. He holes a PhD in Mechanical
Modeling Practice and Theory, 2010, 18(10), pp. 1453-1467. Engineering at University of Leeds, UK. He obtained M.Eng at Beijing
[25] C. Macal and M. North, Introductory tutorial: agent-based modeling Institute of Technology. He worked as a research fellow (internal team
and simulation, Winter Simulation Conference, Savannah, Georgia, leader) for H2020 RiserSure project at London South Bank Innovation
2014. Centre (LSBIC) in TWI Ltd between 2016 and 2017. He worked as a
[26] W. N. Robinson and Y. Ding, A survey of customization support in project engineer at InnoTecUK Ltd from 2014 to 2016. He has been a
agent-based business process simulation tools, ACM Transactions on visiting scholar at University of Alberta, Canada, between 2009 and 2010.
Modeling and Computer Simulation, 2010, 20(3), pp. 1-29. His research interests include product design and development, modeling
and simualtion, agent-based simulaiton, complex design system, decision-
[27] N. Seekhao, C. Shung, J. JaJa, L. Mongeau, and N. Y. K. Li-Jessen,
making mechanism, orgnizational behavior, and others.
Real-Time Agent-Based Modeling Simulation with in-situ
Visualization of Complex Biological Systems: A Case Study on Alison McKay is Professor of Design Systems at the University of Leeds
Vocal Fold Inflammation and Healing, IEEE International and director of the Leeds Socio-Technical Centre. She holds a PhD
Symposium on Parallel & Distributed Processing, Workshops and in Mechanical Engineering. Her research centres on socio-technical
PhD Forum, 2016, pp. 463-472. aspects of engineering design systems and the networks of organisations
that both develop and deliver products to market, and support them through
[28] J. Banks, Introduction to simulation, Winter Simulation Conference,
life to disposal or reuse. The focus of her personal research lies in the
Phoenix, Arizona, United States, 1999.
establishment of systematic and, where possible, well-founded
[29] E. Sklar, Software review: NetLogo, a multi-agent simulation underpinnings for such systems, in particular, for the definition of product
environment, Artificial Life, 2007, 13(3), pp. 303-311. data.
[30] C.-G. Yin and Y.-S. Ma, Parametric feature constraint modeling and
mapping in product development, Advanced Engineering Informatics,
2012, 26(3), pp. 539-552.
[31] W. D. Kelton, R. P. Sadowski, and D. A. Sadowski, Simulation with
Arena, McGraw-Hill, Inc., 2002.
The 8th International Symposium on Computational Intelligence and Industrial Applications (ISCIIA2018)
The 12th China-Japan International Workshop on Information Technology and Control Applications (ITCA2018) 6
Binjiang International Hotel, Tengzhou, Shandong, China, Nov. 2-6, 2018