Evaluation of the Integrated Care and Support Pioneers Programme in the context of new funding arrangements for integrated care in England

Better coordination within the health care sector, and between the NHS and local government, in the commissioning and provision of services has long been a policy aspiration in England. It is frequently stated that integrated care should lead to more person-centred, better coordinated care, deliver more efficient services that improve outcomes for individuals, and provide better value for money. The national programme of Integrated Care and Support Pioneers aspired to encourage the ‘most ambitious and visionary’ local areas to become integration Pioneers to drive change ‘at scale and pace, from which the rest of the country can benefit’.

In 2013-15, PIRU was commissioned to carry out two projects to explore this:

• to convene an expert group to provide advice to the Pioneers on a set of candidate indicators of integration performance based on routinely collected data that they could use individually and collectively to monitor their progress.

• to carry out a largely qualitative early evaluation focusing on progress of the 14 first wave Pioneers in the first 15-18 months in relation to their initial integration objectives.

The early evaluation was carried out between January 2014 and June 2015. The aims were to: 

• Describe and understand the scope, objectives, priorities, plans and management of the 14 selected Pioneers 

• Describe the mechanisms and ‘intervention logics’ adopted by the Pioneers to deliver those plans and priorities 

• Identify the financial incentives, contractual forms and budgetary innovations put in place to implement the Pioneers’ plans 

• Describe how the Pioneers’ Better Care Fund (BCF) plans begin to be implemented 

• Assess the extent to which Pioneers are able to address previously identified barriers to the integration of care 

• Examine progress of the Pioneers in the first 12 months in relation to their first year integration objectives. 

In July 2014, the Department of Health tendered for a long-term evaluation of the Pioneer programme which was won by a team from PIRU.

The overall aim of the longer-term evaluation as to assess the extent to which the Pioneers were successful in providing ‘person-centred coordinated care’, including improved outcomes and quality of care, in a cost-effective way. The evaluation was designed to help build the evidence on what works best in delivering quality integrated care in different contexts. All 14 first wave and 11 second wave Pioneers were included. The evaluation consisted of three interdependent work packages (WPs): 

• WP1: Pioneer level process evaluation and (limited) impact evaluation
• WP2: Scheme/initiative level impact and economic evaluation
• WP3: Working with Pioneers, national policy makers and partners, patient/user organisations and experts to derive and spread learning

WP1 aimed to: 

• Understand how Pioneers were pursuing their aims, and their experiences of making integration-related service changes. This involved surveys and interviews with a panel of managerial and delivery staff from all 25 Pioneers and other local stakeholders, such as patients/service users. The surveys took place on a regular basis throughout the life of the evaluation.

• Compare differences, at Pioneer level, between Pioneers and matched non-Pioneer areas in England in terms of changes in key indicators of the extent of care co-ordination and its consequences. This was done using relevant, routine, national and local data sources. 

• Identify similarities and differences between the approaches to system-level (macro) integration adopted by the Pioneers and those in the NHS’s ‘Five Year Forward View’.

WP2 aimed to: 

• Assess the cost-effectiveness of systemically important integration initiatives undertaken by Pioneers, using a range of designs, such as quasi-experimental, depending on circumstances. 

• Understand how the initiatives’ impacts are affected by context, by undertaking qualitative interviews and focus groups with key stakeholders in the Pioneers participating in the cost-effectiveness analyses. 

• We undertook an evaluation of community-based integrated health and social care multi-disciplinary teams within two Pioneers. 

WP3 aimed to: 

• Synthesise findings from WP1 and WP2, by assessing the extent to which integration initiatives’ achievements (WP2) can be related to the higher level approach taken by Pioneers (WP1). 

• Derive and spread evidence-based insights from WP1 and WP2 to improve integrated care, as well as feeding this information back into the research process to encourage reflexive learning. This included regular interactive workshops involving the research team, Pioneer and patient/user representatives, and other key experts and stakeholders. 

The findings from the evaluation broadly reinforce rather than challenge the findings of evaluations of the series of other nationally initiated, time-limited, integration pilot programmes in England since the mid-2000s.

Work package 1

Annual key informant surveys

The findings from this part of the evaluation largely confirmed and refined those of the early evaluation rather than uncovering previously unacknowledged aspects of the drive at local level to improve health and social care coordination.

The Pioneer programme appeared to be distinctive in terms of the level of local authority engagement both in leading applications for Pioneer status and active engagement in Pioneer activities. While the local authority involvement was not necessarily on equal terms with the local NHS, local authority staff were less likely to report being marginalised than as reported in the ensuing New Care Model Vanguard programme. However, NHS concerns were clearly very significant in the way that the Pioneers’ objectives evolved and some differences in priorities were reported between the NHS and local authority. The early evaluation findings had further suggested that the Pioneers’ initial more locally derived goals (albeit influenced by national policy direction) focused primarily on improving user and carer experience by striving to make care person-centred and seamless had been relegated to the second tier behind NHS goals set nationally focused on reducing unplanned acute hospital admissions and reducing delayed discharges, especially of older frailer patients. This narrowing of focus continued into the period of the longer term evaluation.

The third annual survey in autumn 2018 reinforced the finding from the early evaluation that the most frequently reported manifestation of the commitment to health and social care horizontal integration at the level of service delivery across the Pioneers continued to be the community-based multi-disciplinary team (MDT), usually organised to serve groups of general practices. These MDTs predominantly focused on trying to coordinate the care of older people with multiple long-term health conditions living in their own homes. To varying degrees, they worked to avoid unplanned hospitalisation and facilitate timely discharge home of such people. These findings provide a rationale for why Work Package 2 focused on community MDTs.

There were few specific reports in the annual surveys of ‘substantial’ progress even three years into the life of the Pioneers. Although the Pioneers were selected in principle from among those areas most likely to be able to make swift progress with integration from which other parts of the country could learn, sites did not generally shift from reporting progress in planning to reporting some progress in meeting their integration objectives until at least three to four years into the programme. This reinforces the findings from the early evaluation and other similar evaluations undertaken before and after the start of the Pioneer Programme that the aspirations of national policy makers to make rapid progress with health and social services integration are unlikely to be realised.

There was some evidence from the surveys that the reported ‘barriers’ to implementing integrated care were gradually diminishing in salience and/or being overcome to varying degrees by the time of the COVID-19 pandemic. This suggests that the Pioneers were developing skills in managing the barriers to integration and that the balance between barriers and enablers was beginning to shift in favour of enablers. Indeed, the reported experience of the early pandemic period in the final key informant survey reinforced this (see below). However, a number of obstacles that were reported during the early evaluation remained significant throughout. In particular, routine access to linked NHS and social services data to enable the care of individuals to be more easily coordinated and monitored across agencies was highlighted repeatedly. Such issues required action at national level which had not been forthcoming despite assurances at the start of the Pioneer programme that becoming a Pioneer would enable sites to avail themselves of greater flexibilities enabled by national agencies. Financial constraints varied in salience from year to year but were always remarked on as a barrier to better service integration. Another perennial limit on progress was the fact that so many of those involved in advancing local integration efforts also had other full-time or near-full time jobs.

In line with much other research on service integration, good working relationships, strong local leadership, and the active involvement of local providers were consistently said to be key facilitators of progressing and sustaining better horizontal integration between the NHS and social care, while having local voluntary organisations actively involved increased in importance as a facilitator over successive key informant surveys.

Analysis of indicators of health and care integration

There was some evidence from the analysis of routine hospital admission data that while all parts of the country experienced a rise in unplanned hospital use in the decade 2010 to 2020, places that had taken part in successive pilots (Pioneers followed by involvement in the New Care Model Vanguards) had a lower rate of increase than places that had not. By contrast, there was no sign that Pioneers were better able to reduce the rate of delayed transfers of care than other parts of the country.

Interpreting comparisons of Pioneers and non-Pioneers using routine data

The design of the Pioneer programme and the limitations of routine data systems made comparing the performance of Pioneers with similar non-Pioneer areas difficult. For example, it was unclear whether the Pioneers’ efforts to improve local health and social care coordination at system and service delivery levels were any different from activities that might be taking place in other parts of the country. This made identifying a suitable counter-factual very difficult. It also made interpretation of any Pioneer-non-Pioneer differences tricky. For example, analysis of patterns of unplanned hospital use among the first wave of Pioneers showed a smaller increase in emergency admissions relative to non-Pioneer areas in the first year of the Programme that was not sustained in the second year. A lower rate of increase in unplanned admissions was also seen for Wave 2 Pioneers in the last year of the Programme. The latter finding aligns with the analysis of the cumulative impact of both the Pioneers and Vanguards in areas which had taken part in both programmes but the apparently transitory effect in the first year in Wave 1 Pioneers is harder to explain.

Work package 2

Evaluation of community-based, health and social care integrated, multi-disciplinary teams (MDTs) in two Pioneers

Observations of MDT meetings and interviews with staff showed that they saw the value of working within a MDT structure in terms of the following: the ability to share information about patients, often in ‘real-time’; learning about services, processes and decision-making of other participating agencies; planning strategies for patients and/or their carers that services found difficult to engage; managing risk; and being supported by colleagues when faced with managing often distressing and stressful cases for which there were few if any solutions. It was notable that the team meetings took place entirely without patients or informal carers present. The MDTs generally saw their roles as operating ‘behind the scenes’.

Staff described the MDT as a leveller of traditional professional hierarchies, as a mechanism for collective problem-solving and as a means of sharing responsibility for patient care in which individual members of the team were supported by others, each of whom lent a distinctive perspective to enhance delivery of care.

Frontline staff perceptions of benefits to patients/users overlapped with patients’/users’ views on the benefits of well-coordinated care. The perspectives of frontline staff in the two Pioneers participating in the MDT evaluation were echoed in the findings from the strategic staff (as well as observations of MDT meetings).

Interviewees identified some similar challenges to implementing integrated care in both sites. They particularly emphasised staff turnover, often related to uncertainties about future funding, and the lack of shared information systems as being among the most problematic operational difficulties. System leaders valued national policy frameworks as potential enablers of integrated care but also recognised the role of local contexts in shaping their adoption. Although interviewees pointed to benefits emerging from multidisciplinary working (such as its potential for delivering more holistic care, fewer instances of work duplication, speedier access to care and enhancement of home care provision) they were concerned that such benefits might not be captured by commonly used performance metrics and, thus, the value of MDTs might be under-estimated.

There were some common characteristics of health and care services, together with professional behaviours and attitudes, valued by patients and carers as contributing to perceptions that care was, to at least some extent, person-centred. For example, patients and their informal carers appreciated components of care (such as aids and equipment) which supported them to live independently and with dignity in their own homes. Other valued aspects of care included timely access to care, continuity and consistency of care delivery, having a named point of contact in the MDT or elsewhere together with effective information sharing and communication between services and also with them. Staff whose attitudes and behaviour suggested their needs mattered were also appreciated as was access to staff who could be relied on to respond to emerging concerns, or proactively reach out to them. These findings accord with much previous research.

Where patients and carers experienced difficulties accessing services or professionals and/or receiving adequate time with them, where they had to be proactive in attempting to obtain the support or treatment wanted or perceived necessary, or where they experienced poor information sharing and communication, inflexibility of services or quality concerns, they sometimes reported feeling let down, abandoned, or angry. Most patients supported by MDTs relied on informal help even when they were receiving formal services.

Despite strong similarities between the MDTs in their understanding of the nature of a MDT, the ways of working and precise roles of each MDT were adapted to the context and the needs of the population they served. There was some difference of emphasis between teams that primarily fulfilled a coordinating function and those that primarily provided direct care and support.