Overview

There are three main modules to the course, focusing on:

• analytical methods, including Markov chains, queueing theory, mean value analysis, asymptotic analysis, fluid modeling, Petri nets, and stochastic reward nets;
• simulation methods, including discrete-event simulation, time-driven simulation, Monte Carlo simulation, fluid simulation, rare-event simulation, parallel and distributed simulation, statistical methods, simulation verification and validation, pseudo-random number generators, and random variate generation; and
• experimental methods, including experimental design, system validation, prototyping, benchmarking, statistical analysis, ANOVA, as well as presentation and interpretation of results.

Several papers from the literature will be used as case studies throughout the course to illustrate a variety of performance evaluation techniques, both in isolation and in combination. These papers will be drawn from a variety of computer systems areas, including computer architecture, database systems, enterprise systems, file systems, media streaming, peer-to-peer systems, telecommunications, Web-based systems, and wireless networking.

Course Syllabus and Schedule

1. Introduction and Overview (1 week)
   Course outline. Administrative details. Course objectives. Examples. Background basics in statistics and probability.
2. Analytical methods (3 weeks)
   Markov chains. Erlang formula. Basic queueing theory. Stability requirements. Open queueing networks. Closed queueing networks. Mean value analysis. Approximate MVA. Asymptotic analysis. Fluid modeling. Petri nets. Stochastic reward nets.
3. Simulation methods (3 weeks)
   Time-driven simulation. Discrete-event simulation. ns-2. Monte Carlo simulation. Fluid simulation. Rare-event simulation. Parallel and distributed simulation issues. Random number generation. Simulation verification and validation.
4. Experimental methods (3 weeks)
   Experimental design. Empirical measurement. Instrumentation. Prototyping. Benchmarking. Internet measurement. Network emulation. Statistical methods. Analysis of variance. Presentation and analysis of results.
5. Student Presentations (2 weeks)
   Presentation and discussion of selected papers from the current research literature, featuring performance evaluation case studies of computer architecture, databases, enterprise systems, media streaming, peer-to-peer systems, Web-based systems, and wireless networks.