Performance Engineering

Module aims

Enterprise systems are complex distributed applications, which implement common business functions such as e-commerce, resource planning, customer relationship management, among others. Performance engineering is the area of computer science focussed on ensuring that enterprise systems are responsive, scalable, and cost-effective. 

The course aims at introducing fundamental principles and techniques used in the performance engineering practice. The problems discussed throughout the lectures are common in industrial ICT practice. The course aims in particular at teaching:

  • Mechanisms and architectures to control performance, such as cloud auto-scaling
  • Benchmarking techniques used to quantify the capacity available to a system and its performance
  • Techniques to design experiments aimed at tuning performance in an automatic manner  
  • Techniques to model and forecast the performance of an enterprise application
  • Decision-methods to allocate resources in cloud-based applications

The course this year will also give chance to the students to learn and practice using the Microsoft Azure cloud, thanks to an education support programme sponsored by Microsoft. This will provide an opportunity to familiarize with performance engineering practices for cloud-based systems. 

Learning outcomes

Knowledge and understanding: 

  • understand key performance metrics in enterprise systems
  • describe the performance lifecycle of enterprise applications
  • recognize standard benchmarks used in the industry
  • explain the software architecture of an enterprise benchmark
  • recognize types of enterprise applications and workloads 

Intellectual skills: 

  • identify an efficient testing strategy for a system
  • specify test runs using probabilistic graphs
  • specify an experimental plan for a system
  • forecast performance of an enterprise application

Practical skills: 

  • design and run an enterprise benchmark
  • analyse test data results
  • design an experiment to tune the performance of a computing system
  • assign computing resources to meet performance requirements

Module syllabus

Introduction to performance engineering

  • Enterprise applications
    • lifecycle
    • common and emerging architectures
    • introduction to cloud computing
    • Microsoft Azure case study
  • Quality-of-service (QoS)
  • Service-level agreements (SLAs)

Part 1: Performance testing

  • Benchmarking and load-testing
    • SPEC and TPC benchmarks
    • case study: SPECjbb2015
    • anatomy of a multi-tier application benchmark
  • Experimental performance tuning
    • capturing test results using response models
    • design of experiments
    • black-box tuning of a computer system

Part 2: Resource allocation

  • Workload analysis
    • batch and interactive workloads
    • customer-behaviour model graph
    • client-server interaction diagrams
    • resource demand characterization
  • Analytical models of resource capacity
  • Automated resource management
    • Load-balancing
    • Auto-scaling in the cloud

Pre-requisites

There are no strict pre-requisites.

Teaching methods

Lectures and exercises. Assessed courseworks will be given throughout the course, which will involve experimental work with the Microsoft Azure cloud. 

Assessments

 * This is a level 7/M course

Reading list

Supplementary