Engineering Simulation for Military Technology October 24-26 2011, Washington DC
Simulation Based Approaches for Systems Engineering
Mikel D. Petty, Ph.D. Center for Modeling, Simulation, and Analysis University of Alabama in Huntsville
Center for Modeling, Simulation, and Analysis
Simulation Based Approaches for Systems Engineering
Presentation outline Motivation and definitions Methodologies and tools Challenges from modeling and simulation Summary
A Simulationists Perspective on the Use of Models in Model-Based Systems Engineering
Simulation Based Approaches for Systems Engineering
Motivation and definitions
Simulation Based Approaches for Systems Engineering
Models in engineering
hs = T c P dt = c P (T2 T1 )
1
T2
Mathematical model ~1895 Enthalpy or total heat of superheated steam [1]
Physical model ~1942 Wind tunnel model for P-51 Mustang
Whats new? Complexity, scope, pervasiveness of models Dependence of process on models
[1] R. T. Kent, Kents Mechanical Engineers Handbook: Power, Eleventh Edition, John Wiley & Sons, New York NY, 1937.
Simulation Based Approaches for Systems Engineering
MBSE defined?
Systems Engineering [2] Model-Based Engineering [3]
Model-Based Systems Engineering [4]
Model-Oriented Systems Engineering Science [5]
Model-based systems engineering (MBSE) is the formalized application of modeling to support system requirements, design, analysis, verification and validation activities beginning in the conceptual design phase and continuing throughout development and later life cycle phases. [6]
[2] [3] [4] [5] [6] B. S. Blanchard and W. J. Fabrycky, Systems Engineering and Analysis, Third Edition, Prentice Hall, Upper Saddle River NJ, 1998. NDIA, Final Report of the Model Based Engineering (MBE) Subcommittee, NDIA Systems Engineering Division, M&S Committee, 2011. A. W. Wymore, Model-Based Systems Engineering, CRC Press, Boca Raton FL, 1993. D. W. Hybertson, Model-Oriented Systems Engineering Science, CRC Press, Boca Raton FL, 2009. INCOSE, Systems Engineering Vision 2020, INCOSE-TP-2004-004-02, Version 2.03, 2007.
Simulation Based Approaches for Systems Engineering
MBSE Vee
Prediction Requirements Engineering System Design Subsystem Design Component Design Conceptual Models High Level System Models Detailed Technical Models Component Models Validation Conceptual Models High Level System Models Detailed Technical Models Component Models Acceptance Test System Test Integration Test Unit Test
Component Development
Adapted from [3]
Simulation Based Approaches for Systems Engineering
Methodologies and tools [7]
[7] J. A. Estefan, Survey of Model-Based Systems Engineering (MBSE) Methodologies, INCOSE-TD-2007-003-02, Revision B, 2008.
Simulation Based Approaches for Systems Engineering
IBM Telelogic Harmony-SE
Description
Systems and software engineering process Process similar to vee Repositories for requirements, models, test data OMG SysML models Telelogic Rhapsody development environment Telelogic Tau UML and SysML modeling tool
Models and tools
Simulation Based Approaches for Systems Engineering
INCOSE Object-Oriented Systems Engineering Method
Description
Top-down, model-driven process Combines object-oriented concepts and classic SE process activities OMG SysML models COTS SysML editors
Models and tools
Simulation Based Approaches for Systems Engineering
10
IBM Rational Unified Process for Systems Engineering
Description
Process framework RUP SE (systems) adapted for RUP (software) Model viewpoints and levels define views OMG UML and SysML models SE process framework tool as RUP SE plug in to Rational Method Composer
Models and tools
Simulation Based Approaches for Systems Engineering
11
Vitech Model-Based System Engineering Methodology
Description
Based on four conventional SE activities Textual System Design Language expresses artifacts Incremental SE process onion model Graphical system model diagrams Integrated Vitech CORE tool set
Models and tools
Simulation Based Approaches for Systems Engineering
12
JPL State Analysis
Description
System models describe system states over time State: all system aspects of interest Iterative process of state discovery and modeling State database, relational with SQL
Models and tools
Simulation Based Approaches for Systems Engineering
13
Dori Object-Process Methodology
Description
System function expressed with simple visual models Basic concepts: object, process, state Constrained natural language descriptions Object-Process Diagrams Object-Process Language OPCAT tool set
Models and tools
Simulation Based Approaches for Systems Engineering
14
Challenges from modeling and simulation: Complex systems
Simulation Based Approaches for Systems Engineering
15
Complex systems
A system comprised of a (usually large) number of (usually strongly) interacting entities, processes, or agents, the understanding of which requires the development, or the use of, new scientific tools, nonlinear models, out-of equilibrium descriptions and computer simulations. [8] A complex system is one whose evolution is very sensitive to initial conditions or to small perturbations, one in which the number of independent interacting components is large, or one in which there are multiple pathways by which the system can evolve. [9]
Stock market
Air traffic
[8] Advances in Complex Systems, [Link] [9] G. M. Whitesides and R. F. Ismagilov, Complexity in Chemistry,Science, April 2 1999, Vol. 284 No. 5411, pp. 89-92.
Weather
Simulation Based Approaches for Systems Engineering
16
Characteristics and challenges Defining characteristics
Sensitivity to initial conditions Emergent behavior Composition of components Uncertain boundaries Nesting State memory Non-linear Feedback loops
Challenges
Complex systems difficult to model Models of complex systems difficult to validate
Simulation Based Approaches for Systems Engineering
17
Sensitivity to initial conditions
System state x Predicted value of system state over time
Wide range of predicted states
Initial state x0 Time t Initial time t0 Prediction time t1
Complex systems evolution highly sensitive to initial state. Small differences in state become magnified over time. [10]
[10] L. Smith, Chaos: A Very Short Introduction, Oxford University Press, Oxford England, 2007.
Simulation Based Approaches for Systems Engineering
18
Sensitivity to initial conditions
Challenges Implementation side effects Sensitivity consistency Input data precision Mitigation Ensemble forecasting [10]
Modeling
Increased trials Broad results distributions [11] Validation Sensitivity analysis [12] Input data precision Precision compensation
Distributions
[11] C. H. Brase and C. P. Brase, Understandable Statistics: Concepts and Methods, Houghton Mifflin, Boston MA, 2009. [12] O. Balci, Verification, Validation, and Testing, in J. Banks (Ed.), Handbook of Simulation: Principles, Methodology, Advances, Applications, and Practice, John Wiley & Sons, New York NY, 1998, pp. 335-393.
Ensemble forecasting
Simulation Based Approaches for Systems Engineering
19
Emergent behavior
Behavior not explicitly encoded in agents or components emerges from interaction of agents or components with each other and environment. [13]
[13] G. Williams, Chaos Theory Tamed, Joseph Henry Press, Washington DC, 1997.
Simulation Based Approaches for Systems Engineering
20
Emergent behavior
Challenges Incomplete observation Indirect representation Overabstraction Face validation unreliability Test case design Mitigation Increased observation Explicit modeling focus
Modeling
Validation
Structured face validation [14] Scenario space search
x2
x1
[14] G. Rowe and G. Wright, Expert Opinions in Forecasting: Role of the Delphi Technique, in J. Armstrong (Ed.), Principles of Forecasting: A Handbook for Researchers and Practitioners, Kluwer, Boston MA, 2001.
Scenario space search
Face validation unreliability
Simulation Based Approaches for Systems Engineering
21
Composition of components
Complex systems composed of interacting components. Models of complex systems composed of submodels.
Simulation Based Approaches for Systems Engineering
22
Composition of components
Challenges Interface compliance Architecture selection [15] Model correlation [16] Mitigation Interface analysis [17] Conceptual model comparison Known problems [18]
Modeling
Validation
Weakest link validity Uncertainty estimation [20] Error location Multivariate statistics [21] Statistical method unsuitability Composition validation Validity under composition [19]
[15] M. Shaw and D. Garlan, Software Architecture, Perspectives on an Emerging Discipline, Prentice Hall, Upper Saddle River NJ, 1996. [16] M. Spiegel, P. F. Reynolds, D. C. Brogan, A Case Study of Model Context for Simulation Composability and Reusability, Proceedings of the 2005 Winter Simulation Conference, Orlando FL, December 4-7 2005, pp. 437-444. [17] O. Balci, Verification, Validation, and Testing, in J. Banks (Ed.), Handbook of Simulation: Principles, Methodology, Advances, Applications, and Practice, John Wiley & Sons, New York NY, 1998, pp. 335-393. [18] D. Gross and W. V. Tucker, A Foundation for Semantic Interoperability, Proceedings of the Fall 2007 Simulation Interoperability Workshop, Orlando FL, September 16-21 2007. [19] E. W. Weisel, R. R. Mielke, and M. D. Petty, Validity of Models and Classes of Models in Semantic Composability, Proceedings of the Fall 2003 Simulation Interoperability Workshop, Orlando FL, September 14-19 2003, pp. 526-536. [20] W. L. Oberkampf, S. M. DeLand, B. M. Rutherford, K. V. Diegart, and K. F. Alvin, Estimation of Total Uncertainty in Modeling and Simulation, Sandia National Laboratories, SAND2000-0824, April 2000. [21] O. Balci and R. Sargent, Validation of simulation models via simultaneous confidence intervals, American Journal of Mathematical and Management Science, Vol. 4, No. 3-4, 1984, pp. 375-406.
Simulation Based Approaches for Systems Engineering
23
Challenges from modeling and simulation: Model composition
Simulation Based Approaches for Systems Engineering
24
Composability
Composability. The capability to select and assemble simulation components in various combinations into simulation systems to satisfy specific user requirements. [22]
Component Repository Simulation A
1 1 2 2 3 3
Simulation B
1 1 3 3 52 52
3 3 8 8 22 22 N N
...
N N
[22] M. D. Petty and E. W. Weisel, A Composability Lexicon, Proceedings of the Spring 2003 Simulation Interoperability Workshop, Orlando FL, March 30-April 4 2003, pp. 181-187.
Simulation Based Approaches for Systems Engineering
25
Syntactic and semantic composability
Syntactic composability The components fit together.
Semantic composability The components fit together in a meaningful way.
Simulation Based Approaches for Systems Engineering
26
Validity under composition
[23]
Compositions of valid models considered for classes of models and validity relations Classes of models
Linear Affine Algebraic Elementary Computable
Classes of validity relations
Equivalence Step metric Trajectory metric
[23] E. W. Weisel, R. R. Mielke, and M. D. Petty, Validity of Models and Classes of Models in Semantic Composability, Proceedings of the Fall 2003 Simulation Interoperability Workshop, Orlando FL, September 14-19 2003, pp. 526-536.
Simulation Based Approaches for Systems Engineering
27
Given two models, separately valid under some validity relation, is their composition necessarily valid under the same validity relation?
Validity relation
Equivalence Step metric Trajectory metric
Model
Linear Yes Affine Yes Algebraic Yes Elementary Computable Yes Yes
Yes
Yes
No
No
No
Conditional Conditional
No
No
No
Most useful models here
Simulation Based Approaches for Systems Engineering
28
NP-completeness theory, in a nutshell
Classes of problems P: solvable in polynomial time NP: not solvable in polynomial time? NP-complete: as hard as every NP problem To prove a problem is NP-complete, show that solving it solves a known NP-complete problem NP-complete problems computationally intractable
P A B C D NP V ? NP-complete U S T
Simulation Based Approaches for Systems Engineering
29
Selecting models to compose is NP-complete
Variants of Component Selection Bounded Unbounded Nonemergent
[24]
Emergent
Antiemergent
NP-complete NP-complete NP-complete NP-hard NP-hard General Component Selection NP-hard, NP-complete with oracle [25] NP-hard
[24] E. H. Page and J. M. Opper, Observations on the Complexity of Composable Simulation, Proceedings of the 1999 Winter Simulation Conference, Phoenix AZ, December 5-8 1999, pp. 553-560. [25] M. D. Petty, E. W. Weisel, and R. R. Mielke, Computational Complexity of Selecting Models for Composition, Proceedings of the Fall 2003 Simulation Interoperability Workshop, Orlando FL, September 14-19 2003, pp. 517-525.
Simulation Based Approaches for Systems Engineering
30
Summary
Simulation Based Approaches for Systems Engineering
31
Summary Systems engineering depends on models MBSE methodologies and tools Challenges from M&S may impact MBSE
Models have always been used in engineering MBSE is applied modeling
Simulation Based Approaches for Systems Engineering
32
End notes More information
Mikel D. Petty, Ph.D. UAHuntsville Center for Modeling, Simulation, and Analysis 256-824-4368, pettym@[Link]
Slides: [Link] Questions?
Center for Modeling, Simulation, and Analysis
