Performance Engineering

Module aims

Performance engineering is the area of computer science ensuring that IT systems are responsive, scalable, and efficient. The course aims at introducing fundamental principles and techniques used in performance management of IT systems.

Learning outcomes

 * Knowledge and understanding:

  • understand key performance metrics in enterprise systems
  • describe the performance lifecycle of enterprise applications
  • recognize standard benchmarks used in the industry
  • explain the software architecture of an enterprise benchmark
  • recognize types of enterprise applications and workloads
  • understand tradeoffs when developing software systems
  • recognize critical vs. non-critical paths


* Intellectual skills:

  • identify an efficient testing strategy for a system
  • specify test runs using probabilistic graphs
  • specify an experimental plan for a system
  • forecast performance of an enterprise application
  • assess the impact of data, code and hardware on performance
  • select appropriate tools & techniques for performance engineering

* Practical skills:

  • design and run an enterprise benchmark
  • analyse test data results
  • design an experiment to tune computing performance
  • assign computing resources to meet performance requirements
  • quickly identify performance bottlenecks in complex software systems
  • apply static optimization, meta-programming, just-in-time compilation and hardware acceleration

Module syllabus

 1) Introduction to performance engineering.
- Reference case studies:
-- Stream processing systems
-- Relational data management systems
-- Computer games


2) Profiling and tuning
2.1) Introduction to profiling and micro-benchmarks.
- Profiling systems to find bottlenecks
- Micro-benchmarking critical sections
- Static analysis of critical sections
- Hardware-conscious cost modeling

2.2) Addressing Bottlenecks
- Metaprogramming
- Just-in-Time code generation
- Hardware Accelerators
- Improving Cache-Locality

2.3) Design of experiments
- Capturing test results using response models
- Designing and running a large number of experiments.


3) Performance in distributed systems
3.1) Load-testing an enterprise application
- SPEC and TPC benchmarks.
- Case studies: SPECjbb2015, Apache JMeter.

3.2) Workload characterization
- Workload analysis. Web log mining.
- Resource demand characterization. What-if analysis in distributed systems.

3.3) Resource management
- Auto-scaling in the cloud. Reactive and proactive autoscaling. Load balancing.
- Case study: Apache Cassandra auto-scaling.

Pre-requisites

There are no strict pre-requisites.

Teaching methods

Profiling and tuning; Load testing; Exam Questions

Module leaders

Dr Holger Pirk
Dr Giuliano Casale