A new set of outcomes and indicators helps commissioners and the public see how well local services compare to the rest of the country

How can clinical commissioning groups (CCGs) see how well they are improving healthcare? There is no shortage of data available to be analysed and combined to produce rankings or maps. These will show variations in performance across the country but the significance of these statistics can be obscure. How can we robustly measure the effectiveness of this new layer of healthcare management?

Primary care trusts (PCTs) have previously been assessed under various frameworks including star ratings and later the World Class Commissioning Framework. These have tended to measure processes and financial outcomes and became rather distant from service delivery.

The NHS as a whole has been issued with an outcomes framework by the government that lays out how performance will be measured throughout the English NHS.  NHS England has taken aspects from this and incorporated them into the interim new CCG Assurance Framework which also includes sections on financial management, delivery of the NHS constitution and general capability. This will be assessed quarterly by NHS England.

The outcome indicators have gone through a number of versions over the last year and have previously been known as the Outcomes Indicator Set and earlier the Commissioning Outcomes Framework (COF). 

Some of the areas covered by these indicators may take many years to respond to any changes made by the CCG. Where possible they will be updated quarterly but in some cases it will take years to update the figures. Only where figures are available will these indicators be assessed.

The quality premium, worth up to £5 per patient, will be awarded for a subset of these targets. The outcomes indicators are a major part component of the calculations for the premium. The value of the premium increases with the outcome indicators but may be reduced if any of the constitutional targets are not met.

Explaining the indicators

The national outcomes are split into five areas and this is reflected in the Assurance Framework, although there is inevitably some overlap between them:

Area 1: Prevent people from dying prematurely.

Area 2: Enhance quality of life for people with long-term conditions.

Area 3: Help people to recover from periods of ill health/injury.

The ‘happiness’ section

Area 4: Ensure people have a positive experience of care.

Area 5: Protect patients from avoidable harm.

The indicators themselves have some similarities to the indicators in the quality and outcomes framework (QOF). They have been developed in association with the National Institute of Health and Care Excellence (NICE) and have a population to which they apply – the denominator – and a number of people who ‘pass’ the indicator – the numerator.

They differ from the QOF as there is no attempt to calculate points to compare one area with another. As we will see later there is not even consistency about whether the final outcome should be high or low. Most of the indicators are concerned with avoiding an undesirable event such as premature death, avoidable hospital admission or hospital-acquired infection. However, patient satisfaction should be positive.

There will likely be wide social and economic variation between CCG populations, and many of the outcomes will vary with them. The criteria for success in most of the indicators is based on relative performance from previous years rather than setting an absolute level of success. 

One further difference from QOF is the many different sources of data that are used. QOF depends almost entirely on data that is produced by a single general practice. 

This is commonly monitored by the practice, almost on a daily basis.

The CCG outcomes indicators look much more widely at health outcomes and use a greater range of information sources. Data from hospital episode statistics (HES) will be used alongside data from the Office of National Statistics (ONS), Public Health England and the Friends and Family Test.

As some of these report annually, and often quite far in arrears of their data collection, it will be difficult for CCGs to monitor their progress accurately from day to day. They will be very dependent on reports from NHS England to assess their status.

When the Care.Data dataset starts to be produced it’s likely to be used more frequently in the framework. This will be updated more frequently although it will be at least 2014 before this can used for these indicators.

Working out potential years of life lost

The first area, preventing premature mortality, has a single overarching indicator: “Potential years of life lost from causes considered amenable to healthcare” and introduces a new abbreviation PYLL. This is to be based on statistics from the ONS. All mortality with a primary diagnosis on a given list of OCD10 codes will be counted in the numerator of this indicator. In most of these cases a mortality of zero would be highly peculiar. They are not entirely preventable rather it is thought that these are conditions which could be influenced by health care.

Each death is then multiplied by the life expectancy at the age of their death. Thus a death at age 50 will have a rather greater effect on the final outcome than a death at age 70. The aim is to give a measure of years lost, in a similar way to the use of quality-adjusted life years to measure years gained by treatment.

This is not the same as having an overall national life expectancy. The statistics take account of the fact that expected age of death of a 50-year-old is rather lower than that of an 80-year-old. For this reason increasing average life expectancy by one year will reduce PYLL by less than one year per patient. It is by the life expectancy calculation that the adjustments for age and sex are made. The criteria for the Premium are that the total life years lost should fall by 3.2% between 2013 and 2014 – an ambitious target.

Exploring the indicators

Beneath the main mortality indicator are a number of more specific indicators. These concentrate on patients with specific diseases, with several concerning mortality directly. Mortality rates for patients under 75 from cancer, respiratory and cardiac disease each have their own indicators although there is little detail about how these will operate at the moment. These indicators also will not count towards the premium.

In previous drafts there have been a series of indicators about the mortality of patients with mental health problems. None of these have made the final cut and in fact there are no mental health indicators at all in the outcomes section of the framework and only one indicator in the rest of the framework.

For long-term conditions the most general indicator is ‘the health status of patients with long-term conditions’. The exact details have not yet been published, but the current proposal is to use the GP Survey to collect data on health status. For a couple of years questions have been included which allow patients to identify themselves as having a long-term condition. If they do, they are then directed to complete the EQ-5D™ questionnaire, a six-question survey largely about functional impairment by self-reported measure.

The results will be influenced by services but also by expectations. The indicator will be a measurement of how well services and support live up to patient expectations that will, in turn, be influenced by the social and cultural backgrounds of the patients. These indicators will not be used in checkpoints or the quality premium.

The proportion of patients feeling supported to manage their conditions will also be recorded as will the diagnosis rate for dementia.

Long-term conditions

For the purposes of checkpoint assessments and the quality premium, a single indicator will combine both long-term conditions and acute care. Admissions to secondary care are the primary focus. There are four components. Admissions for chronic conditions suitable for ambulatory care and acute conditions which are not thought to require admission will be counted as will emergency readmissions within 30 days of discharge. The final group will be admissions for asthma, diabetes or epilepsy in patients under 19 years old.

A CCG will earn that proportion of the quality premium if there is no increase in these admissions based on hospital episode statistics (HES). The baseline population will be reported by the ONS.

Other indicators that will not be included in the premium assessment are the number of admissions with lower respiratory tract infections in children and measurement of health gain in patients undergoing surgery. This is likely to be assessed through patient reported outcome measures (PROMS) with questionnaires before and after surgery for hip and knee replacements, inguinal hernias and varicose veins. Varicose vein surgery is pretty rare these days so its inclusion here is a little surprising.

Positive experiences of care

The area about positive experiences of care is unsurprisingly largely drawn from another selection of patient questionnaires. These will look at primary care as well more specifically at those patients who have been treated at or admitted to hospital. The GP survey is already conducted on a rolling basis but neither of these will contribute to the Premium.

It is based on roll-out of the friends and family test and subsequent improvement in the scores. This should be provided in all acute providers and the improvement should be shown between 2013 and 2014. 

Yet a single CGG may use several providers, and a single provider is likely to provide services to several CCGs. As the question will be administered anonymously by the provider and collecting extra data such as postcode is not allowed it is unclear how this information will be allocated to CCGs.

Avoidable harm

The avoidable harm section looks at the number of healthcare associated infections reported to Public Health England. Infections with MRSA and clostridium difficile each have their own indicators and count separately towards the Premium. C Diff cases should be below a threshold (yet to be defined) and there should be no cases of MRSA bacteraemia at all. A single case in a CCG could cost the CCG over thirty pence per registered patient – a considerable sum.

Most of the indicators will depend heavily on locally collected and coded data. There have already been too many examples of coding changes being used to improve statistics in the NHS but it would not do to be too cynical.

Attention will certainly be paid to what is put on death certificates and how other data is submitted but this is more likely for locating and removing current anomalies.

It is likely that CCGs will pass on some of these indicators to acute providers as they negotiate contracts. For indicators that are not part of the premium or the quarterly assessments the pressure will be rather lower. For others, the lack of nationally set thresholds may reduce the pressure for dramatic change – for better or worse.

The timescale for some of these indicators may also reduce pressure for coding changes. Services commissioned could take years or even decades to show changes in mortality. Changes to patient perceptions of services and support can change more quickly but will likely lag behind changes to services on the ground. Even the statistics will take some time to put together. At the time of writing, the latest figures available for mortality are only up to the year 2010. The latest diabetes audit is for 2011. While some of the other data is updated more regularly these are indicators for the long term. While the targets attached to the commissioning premium are likely to be an immediate priority for CCGs the outcomes indicators in this framework should help to shape the strategy of the group.

With luck and by keeping politics to a minimum these indicators should help people assess their CCGs as well as help CCGs to learn about their populations.