Introduction
Computer simulation has become one of the central methodologies of modern scientific inquiry. Alongside theoretical analysis and physical experimentation, simulation provides a third way of understanding complex systems. Where mathematical analysis seeks exact solutions and experimentation observes real-world behaviour, simulation constructs a computational model of reality and explores its behaviour through repeated virtual experiments.
Stanisław Ulam 
John Von Neumann 
ENIAC (Electronic Numerical Integrator and Computer)
The idea of solving problems through simulated repetition dates back to the 1940s, when Stanisław Ulam considered the probability of winning a solitaire game. Rather than attempting to derive a complicated analytical formula, he proposed repeatedly playing the game and estimating the probability empirically. Together with John von Neumann, this reasoning developed into what is now known as the Monte Carlo method: using randomness and repetition to approximate solutions to problems that are otherwise analytically intractable. With the emergence of electronic computing, this approach transformed from a clever thought experiment into a powerful scientific tool.
At its core, simulation is the use of a computer to imitate the operation of a real-world stochastic system in order to study its behaviour.
The system of interest may be physical, biological, economic, or entirely abstract. We may simulate the lifetime of a machine component, the arrival of customers in a queue, rainfall patterns over time, the spread of an infectious disease, or the evolution of a financial portfolio. In each case, the goal is not merely to generate numbers, but to gain insight into variability, uncertainty, and long-run behaviour.
A simulation study always rests on three conceptual pillars: a model, randomness, and repetition.
The model provides a mathematical or logical representation of the system. Randomness is introduced to reflect uncertainty in inputs or mechanisms. Repetition allows patterns to emerge from noise, enabling us to estimate quantities such as probabilities, expectations, and distributions. Without repetition, randomness obscures structure; with repetition, structure becomes visible.
Unlike analytical methods, simulation rarely produces exact answers. Instead, it produces approximations whose accuracy improves as the number of replications increases. This fundamental idea underlies much of computational statistics and is formalised by results such as the Law of Large Numbers and the Central Limit Theorem, which justify the convergence of simulated averages to theoretical quantities.
Simulation is especially valuable when systems are too complex for closed-form mathematical analysis.
Real-world processes often involve nonlinear interactions, high-dimensional dependence, heterogeneous agents, time-varying parameters, and complicated feedback mechanisms. In such settings, deriving exact formulas may be impossible or impractical. Simulation trades algebraic elegance for flexibility: we sacrifice exactness in exchange for the ability to model reality more faithfully.
Modern computing environments such as R and Python make it possible to design, implement, and analyse simulation studies efficiently. However, effective simulation requires more than programming skill. It requires careful problem formulation, thoughtful modelling assumptions, validation of computational implementations, and rigorous statistical analysis of output. A poorly designed simulation can be as misleading as a flawed mathematical proof.
This chapter introduces the foundational ideas that underpin all simulation methods studied in this unit. We begin by formalising what is meant by simulation and how simulation studies are structured. We then examine the essential role of random number generation, since all stochastic simulation ultimately depends on our ability to generate high-quality random variables. Finally, we review the probability concepts that provide the theoretical justification for simulation-based inference.
Simulation is not merely a computational trick. It is a way of thinking about uncertainty, experimentation, and modelling in the presence of randomness. Understanding its scope and its limitations is the first step toward using it responsibly and effectively.