What is benchmarking?
The process of benchmarking can be defined as “a structured approach to identify actions that lead to superior performance”. Benchmarking is not merely a comparison of performance data or a creation of league tables. Performance measurements, for example, deliver little benefit on their own, but they stimulate productive questions and lines of enquiry for more in-depth analysis and research.
In the public transport benchmarking groups, there has been a strong focus on results that can be implemented, performance improvement, and on informing strategy. For this reason benchmarking group members undertake a variety of detailed case studies each year in wide-ranging but focused areas of bus management. Many studies identify best practices in operations and management; others offer key insights that can drive strategy and policy and provide information to support better dialogue with city governments, regulators, and other stakeholders.
The benchmarking process of the Railway and Transport Strategy Centre
Fundamentally, the benchmarking groups provide a forum for organisations to share their experiences and exchange information. So what principles regarding the process have led to successful benchmarking?
Firstly, each benchmarking group is owned and run by its participants. This ensures that the focus is directed towards their highest priority needs of the members and directs efforts towards the areas which will produce the greatest benefits.
Secondly, a long-term approach to benchmarking pays off. It can take several years and iterative cycles to achieve benchmarking indicators that are comparable and reported on a consistent basis. One-off benchmarking studies are rarely successful for this reason.
Thirdly, and very importantly, each group operates within a strict confidentiality agreement. This allows for full data and information exchange within the group but not externally. This overcomes political and commercial sensitivities.
Each benchmarking process uses approximately 35 Key Performance Indicators (KPIs), which measure the performance of the organisation through the balance scorecard categories. The indicators are comprehensive so as to represent all the different parts of the business, yet concise enough to be able to be used effectively by an organisation. The data set behind this set of indicators is collected to standard definitions by all the group members on an annual basis.
The performance of a public transport organisation is impacted by factors both within and outside management control. It has therefore been important to understand external effects on the organisation's performance and the KPIs are often normalised to account for the local cost of labour, for example, or the density of the city. Statistical analyses have been used to provide a greater understanding of the results, while time series analysis has allowed for trends in reliability to be identified. This helps identify which members are changing their practices and shows what improvements are relatively achievable. Where clear differences or improvements in performance have been identified, detailed analysis is carried out, through case studies. These include data analysis as well as interviews with functional managers and staff.
In many cases best practice may lay outside the two respective industries, so other public transport operations and other industries are reviewed for relevant practice. Workshops involving functional experts from each of the participants are particularly useful in enabling reasons for superior performance to be identified.
Each benchmarking group has its own secure website, which can be accessed by the participants to download all available reports and presentations and through which they can communicate with eachother through an on-line forum.