SIBs bring promises of extra cash and reduced fragmentation but also concerns about private sector interests, practicality and governance.
What does research tell us about Social Impact Bonds (SIBs) and their applicability to health and social care? That’s our big question as we review the issues raised by our experts in their thoughtful commentaries over the past few weeks.
It’s clear how the mood music around SIBs has appeal, notably concerning two issues that bedevil health and social care – funding shortfalls and fragmentation. Ring-fencing funds and staff to achieve specific outcomes is a key SIB feature, particularly highlighted in the contribution from Ben Jupp of Social Finance. Those in health and social care who feel frustrated by juggling cash for competing tasks – having to “rob Peter to pay Paul” – would like to get on with their jobs, unencumbered by unexpected resource cuts.
Likewise, part of SIBs’ appeal is the feeling that they address chronic fragmentation of service delivery, particularly between health and social care. In principle, SIBs incentivise everyone to work in harmony on behalf of the patient/user.
Some of our commentators have cast doubt over these possible benefits. Ring-fencing of spending and staffing can, for example, be achieved by other means, such as minimum nurse staffing ratios. Our contributors also highlight that SIBs can’t cure the structural problem of health and social care fragmentation, though they can be helpful on a case by case basis.
We would like to draw attention to three further issues raised by our commentators: the public’s general discomfort with the idea of private interests in NHS provision; the practicality of outcome-based, debt-funding in health and social care compared with traditional forms of payment; and the governance challenges created by SIBs as they reintegrate the purchasing and providing roles that have been separated in the English NHS since the 1990s.
On the other hand, we suggest that SIBs can contribute to mitigating some longstanding issues facing health and social care. These include the need for a better focus on prevention and outcome delivery; personalising services around the user/patient; and better partnership in health and social care planning and delivery between purchasers and providers.
1. Discomfort with private financing of healthcare
The early UK SIB-funded programmes have typically dealt with issues outside the mainstream of health and social care provision, such as youth unemployment and probation services. SIBs in health and social care have mainly focused on preventive interventions to improve health (e.g. social prescribing). We’ve yet to see proposed applications that address core NHS treatment activities such as funding hospitals and primary care. Such proposals might be highly controversial to the public.
The NHS retains a unique status for voters, unlike, for example, tertiary education, where private fees are widely considered to be acceptable, or social care, which is means-tested. The NHS is a universal service. There is a general belief that it should be paid for publicly. There is also less pressure to find alternative sources than there is, for example, in the case of some local authority social services because the NHS has been better protected from financial austerity than other public services.
2. Important practical and effectiveness questions
Our second theme concerns practical questions about the effectiveness of SIBs. Alex Nicholls and Mildred Warner have drawn attention to the weakness of the evidence in this area. There is a lack of evidence to demonstrate that SIBs provide better services that are more cost-effective or more likely to save money than those delivered using conventional funding methods. A further issue is how to make cashable any notional savings generated by improving user outcomes: for example, being able to close wards and sack nurses as a result of a successful SIB would not be a popular approach in such a highly politicised public service. Unlike many social services, such as problematic families, the public generally regards spending more on the NHS as a positive thing, so there may be less political support to pursue “savings” in health.
Healthcare commissioners in the UK are well versed in scrutinising cost-effectiveness evidence but healthcare lacks the interoperability of information systems that aids oversight of the outcomes and costs arising from interventions. So, for example, it’s currently easier at the Department for Work and Pensions to assess a SIB’s impact on employment rates among NEETs (Not in Employment, Education or Training) than for the NHS to track whether a preventive health SIB has reduced use of primary and hospital care.
Healthcare commissioners may also be concerned that SIBs typically have high set-up/transaction costs and quite opaque management costs. They may be reluctant to pay for outcomes when the sources of money needed to pay intermediaries and other actors lack transparency. This caution about SIBs is well founded given the continuing costs of previous PFI projects, which created new facilities quickly but left the NHS with debt and concern among some that providers may have secured poor value for money.
3. SIBs and governance issues in healthcare
Thirdly, there is a governance challenge that may accompany the adoption of SIB-funding for health and social care services. SIBs are attractive because they are designed to bring stakeholders into partnership in pursuit of a set of goals around a particular client group. This may imply, for healthcare, a degree of reintegration of purchasing and providing. This may be problematic as it appears that a genuine market in intermediaries is currently lacking.
We also do not yet have a recognised governance framework setting out how intermediaries and the other parties should operate at particular stages of the SIB development process. This is of crucial importance for policymakers as SIBs currently develop in a space that lacks agreed rules and processes. It will be important to resolve these governance issues and create an explicit framework for managing potential conflicts of interest.
Digging a little deeper, the SIBs debate should remind us of the need to maintain a healthy scepticism about all providers, even if they are from the Third/not-for-profit sector. They should be subject to due scrutiny, like any other providers of contracted-out services.
There are important questions and doubts about the wisdom of widespread application of SIBs to health and social care. Nevertheless, we should be careful and avoid prematurely rejecting a still largely untested innovation that could have niche applications for entrenched social issues. It is also worth remembering that traditional health and social care procurement models are far from perfect.
SIBs are rightly taken seriously because they help to question services that have, through bureaucracy, size and provider-domination, sometimes failed to meet the needs of patients or the requirement for cost-effectiveness. But there are important questions to be asked about the transparency, costs and accountability of SIBs. Our commentators have shown that SIBs are not a panacea and that they bring unanswered questions with them. More robust and more plentiful evidence would be needed to justify widespread roll-out of SIBs in UK health and social care.
Stefanie Tan and Alec Fraser are Research Fellows at the London School of Hygiene & Tropical Medicine and part of the Policy Innovation Research Unit (PIRU) at LSHTM.
Nicholas Mays is Professor of Health Policy at LSHTM and Director of PIRU.
Acknowledgement and disclaimer:
This is an independent commentary commissioned and funded by the Policy Research Programme in the Department of Health. The views expressed are not necessarily those of the Department.
“There is strong encouragement for the NHS to make better use of innovation, but we’ve not known how our system performs internationally. Now we are developing a potential measure.”
The Accelerated Access Review (AAR) – which aims to help speed the introduction of health innovations into the NHS – was published last October. Readers with long memories may recall a string of previous reports all tackling the need to improve the UK’s health innovation system: the House of Commons Health Committee’s inquiry into the use of new medical technologies (2005) , Best Research for Best Health (2006), the Cooksey review (2006), Innovation for Health (2007), Our NHS Our Future (2012), Innovation Health and Wealth (2012).
The AAR calls for a broader remit for NICE, to include more medtech and digital health innovations. Other recommendations are for closer alignment between regulatory requirements and processes, and for once-only assessment by NHS England of innovations not referred to NICE. Simpler and swifter procurement processes are part of the future agenda. And in an implicit nod towards the work of Clayton Christensen, the AAR suggests there should be some funding to support the commercialisation of disruptive innovations that have the potential to significantly improve care pathways.
These are all sound recommendations, but they are only the beginning. Recommendations now need to be turned into specific actions with budgets, owners, milestones and deliverables.
The good news from the AAR is that consensus seems to have broken out, with a genuine belief by government and NHS leaders in the potential of innovative technologies to improve patient care. The consensus extends to a call for more flexible approaches to pricing and reimbursement which can support the adoption of innovations.
The bad news is that the NHS is in deep financial trouble. For all the Churchillian rhetoric of healthcare leaders – “we have run out of money, now we must think” – and exhortations that a crisis is a great opportunity for innovation, the distinct tendency in the NHS is to hunker down, deal with the immediate concerns and shuffle innovation into the “too difficult” pile.
Goals, targets or crisis can of course concentrate the mind and, when coupled with the right kind of support and incentives, generate useful new ideas – a good example was Scotland’s Unscheduled Care Collaborative Programme in the mid-2000s. However, the magnitude of the challenge of modernising the NHS, whilst continuing to run services with shrinking budgets, seems overwhelming. We know we need innovation, we know what kinds of innovation we need and we have a pretty good idea what is the potential impact of many innovations on improving care – the problem is introducing and establishing them in the NHS.
The AAR, if its recommendations are implemented, might help to speed up the flow of some new technologies into everyday practice, but it will not tackle the other layers of innovation that are needed. These were nicely described by the Health Foundation in 2015 as five interlinked components of a framework for change, all of which require innovative thinking – population health management, ways of delivering care, process improvement, active cost management, and scientific discovery, technology and skills. The report argued that all these are capable of delivering improvement over different timescales. Interestingly, the last of these – science and technology – is felt to have the most limited potential impact.
But just how bad is the NHS at innovation? There is much anecdote and received wisdom – “the UK is great at generating innovations but poor at adopting them”, “the NHS is always a late adopter”, “developing countries health systems are where the really interesting new ideas are”, “the USA over-adopts healthcare technology”.
Much seems to depend on perception. The 2016 Global Diffusion of Healthcare Innovation study surveyed 1,356 frontline health workers and healthcare leaders in six countries and found that the USA and UK were ranked as the two most important sources of innovation, but there were variations partly according to location – India was perceived as by far the most important by interviewees in some countries.
The volume of research on healthcare innovation processes has grown over the last fifteen years or so. We know what constrains the adoption of innovation and we know what innovators need to support them from research to commercialisation.
What we lack, though, is any kind of assessment of where different countries stand in relation to each other in their healthcare innovation capacity – how good they are at adopting innovative solutions developed elsewhere and originating innovations themselves. None of the reports on the state of healthcare innovation in the UK described above provide any real insight into our performance compared to other countries. Apart from limited work by Deloitte, PWC and Accenture, there has been no attempt to create an international comparative healthcare innovation index (the Global Diffusion of Healthcare Innovation is not an index because it teases out perceptions of innovativeness).
There are many indices or composite indicators measuring aspects of national performance in innovation in general. Indicators include measures of entrepreneurship, technology development and scientific research, innovation in general, and innovation in public sector organisations. However, these remain underdeveloped in relation to health.
A carefully constructed global health innovation index would be valuable in two ways:
- It would focus attention on the relative performance of the UK’s (or any other country’s) health system, pinpointing its strengths and weaknesses, identifying bottlenecks and issues for attention. This would enable policy makers and the healthcare sector to identify and prioritise the levers that can have the greatest impact on innovation.
- It would help medical technology and life sciences companies by providing an understanding of the attractiveness of each country’s health system as a market for products, as a source of innovations, and as a location for R&D.
A project funded by PIRU has carried out the groundwork needed to develop a global health innovation index. We reviewed existing global innovation indices and explored the conceptual, methodological and practical issues that must be addressed. We draw three conclusions from our review.
First, one should proceed with caution when deriving policy, research or other implications from composite indicators. Before we can be confident of their implications for the comparative performance of countries or regions and establish benchmarks to underpin policy or other decisions, it is essential to understand how innovation indicators impact on innovation processes. This in turn requires indices to be underpinned by a clear and strong theoretical framework.
Second, there are a number of data issues which have to be addressed. To explore the dynamics and evolution of health innovation across countries through an index requires panel data (collected over time). While the availability of cross-sectional data (between countries) is fraught with difficulties, the use of panel data is even more so. Another issue is variation at the regional or local level. Policy makers are interested not only in their national scores, but in whether different regions in their country perform differently.
Finally, policy makers should be careful not to draw conclusions about the relationship between composite innovation indicators and other indicators of policy interest such as health outcomes. While it may be of interest to correlate innovation index scores with those of an index of health outcomes, we must always remember that correlation does not mean causality – introducing a new drug or device may well lead to better health outcomes, but an index will not tell us this.
We are now hoping to create a prototype index for a few countries. This involves investigating data sources and more work on what the index is measuring, for whom and for what purpose.
Anyone interested in finding out more should contact James Barlow at Imperial College Business School, email@example.com
James Barlow is a Professor of Technology and Innovation Management (Healthcare) at Imperial College Business School and a member of PIRU. His latest book, Managing Innovation in Healthcare, was published by World Scientific in January 2017.
- House of Commons Health Committee (2005) The Use of New Medical Technologies Within the NHS: Fifth Report of Session 2004–2005.
- Department of Health (2006) Best Research for Best Health: A New National Health Research Strategy: The NHS Contribution to Health Research in England.
- HM Treasury (2006) A Review of UK Health Research Funding.
- Strategic Implementation Group & Healthcare Industries Task Force (2007) Innovation for Health: Making a Difference.
- Department of Health (2007) Our NHS Our Future: NHS Next Stage Review – Interim Report.
- Department of Health (2011) Innovation Health and Wealth, accelerating adoption and diffusion in the NHS.
- Dattée B, Barlow J (in press) Multilevel organizational adaptation: Scale invariance in the Scottish healthcare system. Organization Science; Dattée B, Barlow J (2010) Complexity and whole-system change programmes. Journal of Health Services Research & Policy 15, S2, 12-18.
- The Health Foundation (2015) Shaping the Future. A Strategic Framework for a Successful NHS.
- Deloitte (2012) Innovation Indicators for Healthcare in Emerging Countries. Understanding and Promoting Innovation in Emerging Markets.
- PWC (2011) Medical Technology Innovation Scorecard. The race for global leadership.
- Accenture (2013) Measuring the Unmeasurable. A New Framework for Assessing Healthcare Innovation.
- Cravo Oliveira T, Barrenho E, Vernet A, Autio E, Barlow J (2017) Developing a Global Healthcare Innovation Index. PIRU Reports 2017-20. http://www.piru.ac.uk/publications/piru-publications.html
The experience of Social Impact Bonds can inform a wider set of relationships to help address social needs, argues a key innovator in the field.
The health and social care system I work in has rarely been in greater need of transformative investment. As demographic and financial pressures threaten to pull it apart, better responses to people’s needs are urgently required.
In my experience, developing more community focused and preventative health and care services will rest on approaches which combine both innovation and rigour. It will rely on strengthening the capacity of services to understand needs, learn from others, adapt and implement change. It will take an approach that looks beyond the annual budgeting of the NHS.
Many Social Impact Bonds seek to change systems
To me, and my colleagues at Social Finance, Social Impact Bonds (SIBs) have always been one tool to support such transformative responses. Take, for example, the Reconnections Service in Worcestershire. It aims to address chronic loneliness, responding to a need that was highlighted by older people in the county themselves.
Establishing Reconnections has involved designing a new service, pioneering long-term cost-benefit analysis on the impact of chronic loneliness, and helping draw together a partnership with a network of half a dozen voluntary and community sector organisations. For the range of commissioners involved, an outcome-based contract was attractive: making payments only if and when the new approach is effective. For the service itself, an outcome-based contract has also enabled flexibility and innovation.
The first year of Reconnections has been hard. Service providers, volunteers and investors have all had to work in ways they had not expected. We have had to learn and adapt. But, slowly, the six charities and community organisations are developing better responses to needs that have often been neglected for far too long, to the detriment of individuals and the health and care system as a whole.
The value of Social Investment Partnerships
Our experience of developing the Reconnections Social Impact Bond and other transformative programmes has also highlighted the value of creating broader Social Investment Partnerships (SIPs) in health and care.
Such Social Investment Partnerships encompass approaches to address social needs which include Social Impact Bonds but also wider forms of collaboration where commissioners, providers and investors share risk and pool knowledge and experience in the design and delivery of services. Highlighting partnerships reflects the need for commissioners, investors and providers to work collaboratively and flexibility, rather than the more adversarial separation between ”purchasers” and ”providers” of services that has characterised the NHS in England for the last 30 years.
For example, in a Social Investment Partnership to improve community-based end of life care, we have been working collaboratively with commissioners and providers for six months to scope out the need jointly. The investors and commissioners have then jointly considered which organisation would be best placed to develop the service; sometimes through a competitive process, sometimes by just building on an existing service.
Alternatives to outcome-based contracts
In such partnerships, we have also found that an outcomes-based contract is only one way to transform services. A focus on the underlying outcomes enabled by a service is important: an outcome-based contract can have significant advantages in terms of allowing service flexibility and providing focus. But for other areas, it’s impossible to capture the full range of people’s needs with a focus on a small number of objectives.
That is why, for example, in our partnerships to develop better Shared Lives social care (the Shared Lives Incubator), we have invested directly in the provider without an outcomes-based contract. Creating the conditions in which someone wants to share their home with another person in need of care is difficult to stipulate in a contract which prioritises one or two impacts. So the investors and commissioners jointly identify the provider, based on their overall qualities and experience. The investor then provides the upfront funding for developing the local service and repayment is made simply as proportion of the service revenue, if and when it grows. In other partnerships, risks are likely to be shared on the basis of a capitated budget – a set amount of funding for the population.
Drawing in investment from a range of sources
Finally, I have found that the concept of a “Social Investment Partnership” draws attention to the need for new resources and capacity and a long-term approach when tackling social challenges, rather than focusing on a particular type of financing. In some of the partnerships which we are building, such as those that develop better employment support for people with health conditions, the source of the funding is a mixture of public sector, charitable and external socially motivated investment. In nearly all such partnerships, we are combining financial investment with access to clinical and operational support. For example, because Health and Employment Partnerships operates across multiple areas, it has the scale to able to employ clinicians and service managers with deep experience of managing similar services before and is helping to manage NHS innovation funding as well as social investment.
I’m proud of the pioneering approaches that my team and colleagues across the sector are engaged in through Social Impact Bonds such as Reconnections. By broadening the approach to developing Social Investment Partnerships, we now have the opportunity to enable a wider range of contractual forms, further blur traditional adversarial relationships, and draw in resources from a number of sources.
These partnerships, whether traditional Social Impact Bonds or a not, are united by a passion for supporting communities and the most vulnerable in particular. They also recognise the importance of both innovation and rigour to achieve social change and ensure that organisations and systems have the capacity to adapt.
Ben Jupp is a Director at Social Finance. He was Director of Public Services Strategy at the Cabinet Office. He has also been Director of the Office of the Third Sector in the Cabinet Office.
There’s no shortage of gadgets. But users and carers require help to make the most of what is available. Industry also should focus more on understanding what people really need from assisted living technologies, argues a new PIRU report.
In 25 years’ time, when many of today’s middle-aged population can expect to be living with dementia, technology will play a big part in their care. It will help them to stay in their homes longer and to avoid being institutionalized. It will also support their carers.
But right now, this isn’t happening enough, even though there is no shortage of gadgets developed by industry. “Why is this?” we ask in our report entitled “The Case for Investment in Technology to Manage the Global Costs of Dementia”. What can be done to accelerate the adoption of assisted living technologies, given the high levels of need among those who have dementia, the stress and isolation of their carers, and the urgency with which the cash-strapped health and social care systems need to control demand for costly treatment and support?
Slow adoption is partly explained by the particular characteristics of those whom we expect eventually to adopt these technologies – people with dementia themselves, their carers – and the local authorities and NHS bodies that might facilitate change. More on these issues in a moment.
Supply-side industry issues
However, responsibility also sits with those who develop the technologies. Research programmes which have looked into the use of assisted living devices continue to find that industry fails to optimise the user interfaces of technology, whether for health and care professionals or the general public.
That said, industry ought not to take all of the blame. It does, given the right circumstances, create and sell low-cost, user-friendly health-related technologies such as Fitbits, which many older people feel comfortable adopting. And most older people have mobile phones. So why doesn’t the large but niche market for people living with dementia show a similar level of growth and development?
Partly, it is because industry has not been as sensitive to this market as it has been to more able-bodied and -minded consumers. It can produce some really helpful, easy-to-use innovations, such as weight-sensitive mats that tell carers remotely whether someone has got out of bed and is walking about. However, as far as more sophisticated devices are concerned, research shows that suppliers often still tend to blame users when technology doesn’t work well. They should instead ask themselves more searching questions that would identify often simple design difficulties in their products.
Suppliers are not necessarily fully cognisant of the need to match technology to the sensory and tactile skills that older people have. Major companies are sometimes naïve about both the basics of what ageing means for individual abilities and the markets for a product. Developers in the ICT industry are still seduced by bigger, more obvious markets in which less trouble has to be taken in show people how to use the technology.
Despite all the effort put into user-led design, there is also still a big gap between research and development and engaging users to ensure that products are right from the outset. There are also plenty of examples where researchers and companies appear to develop a solution that’s in search of a problem.
Lack of support for users and carers
Nevertheless, despite these supply-side concerns, perhaps the biggest issues with regard to dementia care and assisted living technologies are on the consumer or demand side. We can’t be scientific, but perhaps about 80 per cent of the challenge is human, organisational and funding, compared with 20 per cent of the problem being about developing the technologies in the first place.
Funding issues are acute in the UK, where, like in most countries, the health and social care systems are fragmented. That means the costs and benefits of assisted living technologies fall into different silos, which can block take-up of innovations.
From previous research we reckon that an eighth of the costs of dementia care fall on the NHS, a quarter on social care services and the rest on informal carers. As a result, it is sometimes difficult to persuade one system to fund technology when the cost savings might accrue elsewhere. This sharing of cost is particularly difficult in the UK because social care is means-tested while healthcare is not.
An underlying issue, which our report also highlights, is that these technologies can be challenging for everyone, but particularly for older people, many of whom are not competent or confident with what for them are new technologies. That is particularly true for people with cognitive difficulties arising through dementia.
Carers will generally not have the cognition issues – although many carers are quite old themselves, and may have their own health issues - but they may not be familiar enough with the technologies to take on new challenges and tasks within the care setting. Older carers in particular might not have spent much of their working lives using ICT. They may perceive broadband to be expensive, even though the actual cost is not particularly high.
People need reliable advice and support
Often advice and support that’s available from the local authority is neither confident nor competent. The Care Act 2014 requires support for carers, but this duty has coincided with a sharp reduction in social care funding and availability, so that implementation has not matched the ambition.
Our report finds that local authorities are not good at sign-posting technologies to carers or at providing other associated supports. This concerns not just the monetary costs of purchasing technology but the costs of learning new skills. Such help would require fresh investment by councils.
However, this support could be vital to carers who need sound advice around technologies that they themselves might buy. They are not alone in being in the dark: a whole range of people in the system - including commissioners and care homes - lack knowledge about what is available, and what it might achieve.
That is why we argue that the public sector should take on an information broker role, offering a source of unbiased advice on assisted living technologies. A council might even use its purchasing power to negotiate attractive deals with suppliers, so that people feel assured that they are buying the right product at the best possible price.
There are examples of healthcare professionals – including dentists, GPs and pharmacists - providing this kind of advice to vulnerable patients and carers, even if their own service cannot foot the actual bill. The Third Sector has also long provided important consumer support, going right back to the founding of the Consumers’ Association. Some of the charities that support older people are well-positioned to take on such a role.
It may well be that industry – keen to expand its own markets – has a part to play in easing this problem. It should consider how it can support delivery of unbiased information and support. Carers and patients need all the assisted living help that they can get. They can’t wait for another 25 years.
Martin Knapp is Professor of Social Policy, Director of the Personal Social Services Research Unit at the LSE, and Director of the NIHR School for Social Care Research.
James Barlow is Professor of Technology and Innovation Management (Healthcare) at Imperial College Business School.
Both authors are members of the Department of Health’s Policy Innovation Research Unit (PIRU). Their report, written with several other colleagues, “The Case for Investment in Technology to Manage the Global Costs of Dementia”, is published by PIRU.
Supporting change through social policy is an iterative process which develops participatory relationships with users. Social Impact Bonds fail on this front, rendering them unfit for purpose.
SIBs are intrinsically unsuitable for programmes that seek to transform people’s lives. That’s because such transformative interventions typically succeed only when they involve partnership with the people who are in need of help – and that’s not one of SIBs’ strengths.
SIBs are generally unaccountable to service users; they usually answer to funders. That’s their core failure and limitation. An inadequate relationship with service users means that SIBs typically lack sufficient understanding of – and sensitivity to – the causes of social problems that is required to achieve transformative change.
We should face up to these issues because, although there are only 32 SIBs operating in the UK, there is a considerable amount of enthusiasm for them. They have, and continue to receive, a lot of vocal and financial support from government.
Difference between technical and transformative interventions
In social policy, it’s helpful to distinguish between technical and transformative interventions. The former tend to be simple responses that encourage relatively uncomplicated changes and behaviour. Often they are about increasing service use, such as, for example, raising the uptake of vaccinations. SIBs could be suitable for such programmes and randomised controlled trials could be one way to evaluate them.
In contrast, transformative interventions try to generate profound changes in the dynamic between conditions and actions. So they must engage the agency of service users or the recipients of those interventions. These interventions address complex problems, often with multiple stakeholders, and try to ameliorate poor conditions. They’re very much contextual. Crucially, they’re about doing things with people rather than doing things to them. So, they might involve trying to reduce or tackle obesity and the problems around that very complex issue. Or they might seek to reduce health inequalities which are a major problem within the UK.
Recipe for social transformation
For social interventions to be considered transformational, they must engage with the means and the identities of those they seek to help. They should cultivate capability for independent living. There shouldn’t simply be a focus on producing outcomes. A good way of thinking about this is to consider education policy: it should be about cultivating independent and critical thinking rather than focusing simply on outcomes and achieving certain rates for test passes.
This approach should encourage us to rethink best practice. Where transformation is concerned, it shouldn’t be about uniform standardisation. Policy should respond to varying conditions and individual circumstances. What works in one area may not work elsewhere. We should resist the urge to scale up interventions indiscriminately.
Different approaches to evaluation
Evaluations should, in turn, provide insights that help us understand these very complicated human interactions and social conditions. Good policy development and evaluation should be about understanding interactions between agents and the subjective experiences of service users. It should be about getting to the bottom of why things work, which is a difficult and challenging issue.
There tend to be two approaches to evaluation. One tradition - let’s call it “positivist” - is exemplified by much of the evidence-based policy movement. It tends to favour randomised control trials in trying to attribute causality.
An alternative tradition – let’s call it “phenomenological” or “interpretivist” - regards policy in relation to complex social problems as less linear and more of an iterative process. It’s based more on reasoning from incomplete or imperfect data. It highlights, to some extent, the limitations of the evidence-based policy movement.
A great deal can be learned from the field of public health. It has a long history of engaging in, developing and evaluating complex social interventions. Indeed, the Medical Research Council (MRC) has issued guidance on how to develop and evaluate complex interventions. The MRC says it’s an iterative process and that the first step should be to understand subjects’ views and how they understand their own condition and treatment.
The political nature of social policy
So social policy, when trying to do anything complex or transformational, shouldn’t try to reduce issues to mechanistic terms such as providing inputs and then producing outputs. We should also recognise that the design of social policies is a political process. Indeed, simplifying complex conditions and problems into depoliticised inputs and outputs, far from being apolitical, is actually a form of ideology itself.
Nevertheless, SIBs tend to be represented as a technocratic and depoliticised approach to policy. They involve shifting responsibility for welfare policy away from government. In this vision, the role of government is diminished, becoming about purchasing or brokering public services.
SIBs are highly political
We would, instead, argue that SIBs are highly political in two key respects. First, they turn public services into commodities, rather than being services to citizens that require a relationship with those citizens. So SIBs “commodify” public services. Second, they give financial value to those fictitious commodities. So SIBs change what were non-monetary social relationships into financialised commodities.
This process diminishes and dehumanises service users who become reclassified as customers. This diminution is not immediately obvious because it is masked by language. Indeed the language of consumer culture – in which the key agent is the customer who is always said to be “right” – might seem to raise rather than diminish the status of the service user. But this is actually a fraudulent representation.
A real customer has control of the cash, providing him or her with consumer sovereignty. In reality, with SIBs, there is very limited consumer sovereignty because citizens’ welfare entitlements become transformed into contracts between organisations and stakeholders.
In this change, the quality of these services is altered. The previous public sector ethos is lost, replaced by a profit incentive. As a result, the service user, instead of being at the heart of everything, becomes an externality, a troublesome afterthought to the main activity which is a series of transactions between principal stakeholders.
What next for SIBs?
Where does all this leave us? SIBs encourage a focus on what is measurable. They also prioritise outcomes rather than trying to understand how these effects are actually achieved. We would argue that complex multi-dimensional social problems require a different approach that involves collaborative policymaking. This policy should involve trying to alter conditions for service users’ rather than imposing strict theories of change upon them.
Moving forward from SIBs
Recognising these deficiencies, we propose a different approach which prioritises service users’ perspectives and which is a first step towards co-determination of policy. A good way forward is to use Q methodology, a research tool used in psychology and in social sciences to study "subjectivity" (namely views, opinions, values and beliefs). It was developed in the last century by the psychologist and physicist, William Stephenson.
This approach is a well-established way to identify and describe viewpoints. First, a card sort (of statements) is used to generate data. Then, there is a form of factor analysis which identifies patterns and similarities between individual card sorts. These methods reveal the structure and form of subjectivity on a particular topic, allowing subjective perspectives to be described.
Q methodology has potential uses in policy development and evaluation because it could help to ascertain the relative importance of problems, the causes of problems, as well as potential solutions and policies. However, it’s just one practical, albeit important, way to actualise service users’ perspectives and so improve policy design and evaluation.
Crucially, this approach is aligned with genuine social innovation whose starting point is that it should empower service users. It should enhance their capacities. That’s missing in SIBs. If social policy is really going to make a difference, then we need to replace SIBs with more participatory approaches to policy development.
Dr Stephen Sinclair is a Reader in Social Policy in the Yunus Centre for Social Business and Health, Glasgow Caledonian University
Dr Neil McHugh is a Researcher at the Yunus Centre for Social Business and Health, Glasgow Caledonian University.
Dr Michael J. Roy is a Senior Lecturer in the Yunus Centre for Social Business and Health, Glasgow Caledonian University
In the US, SIBs are the height of social policy fashion, but the risks are clear and numerous.
The dream of Social Impact Bonds (SIBs) sounds pretty exciting. It’s that you can invest in something good for society and at the same time get a market return so everybody wins. The investor wins, the client wins and, because it’s a better programme, society wins.
SIBs are growing around the world and mainly focus on prevention, which is good to see, because cure is often more costly than prevention. There is considerable enthusiasm in the US where 35 states are building programmes to encourage SIBs. Only ten SIB projects are active in the US, but there are probably hundreds in the pipeline.
In Congress, support is coming from both sides of the aisle. The US Government Accountability Office told me: “You know, Mildred, the Republicans like this because it lets the private sector get access to public social welfare dollars and the Democrats like it because it might increase public investment in social welfare”. So it’s a win-win politically.
Large organisations are lining up to act as intermediaries in establishing SIBs and evaluators are honing their skills to get the contracts to assess the outcomes. I’ve even talked to people on the street who have heard about SIBs. “I’d like to invest in that,” they’ll say. “I’d love to put my money in something that’s going to yield a good return for everyone.” But they’re not quite sure what this thing is.
And there’s lots of concern as well. Providers are responding to this new landscape - some are nervous about payment. Academics are probably the most critical. Government managers feel cautious, worried about how much work is involved in putting these things together. Sometimes, officials feel forced reluctantly down the SIB pathway.
Take, for example, the SIB at Riker’s Island prison, designed to reduce the rate at which juvenile offenders return to jail. The Vice Mayor of New York City told me she would not have done the Riker’s Island project as a SIB if she could have funded the program directly. That would have been quicker and easier. But budget limits prevented that and she didn’t want to wait as, each year, more teenagers get caught in the prison system.
So we should take a long hard look at what’s going on. We should make sure that we’re not swept along by a tide of wishful thinking that could leave disappointment in its wake.
SIBs transform social services
This aspect of SIBs is particularly important because they represent a major upheaval in the design and delivery of social services. Typically, SIBs require intermediary management, private investment and some kind of outside evaluation which allows you to link performance to financial returns.
SIBs take public management to a new place because government is no longer at the centre, as in the past, but an intermediary organisation is organising and running things. It’s true that the Government is at the top, calling the shots, defining the structure, defining the goals, but the intermediary is at the centre of the process making everything else happen, linking to the service provider, linking to the outside investors, commissioning the evaluator. The intermediary is becoming a very important actor.
Clients lack a voice
There are some real concerns about the way SIBs work. They tend to focus on areas where the client is pretty weak or voiceless or maybe despised by society. These are areas where we haven’t been able to motivate sufficient public investment in prevention because who cares, for example, about a prisoner and their re-entry into the community?
Yet, ironically, SIBs seem to leave the clients voiceless. These are homeless people, little kids, people who are vulnerable. There seems to be very little thought that we’re not giving the consumer or the client a voice in SIBs. That’s deeply troubling.
I would sum up other concerns as relating to: the suitability of SIBs for complex social problems and solutions; the difficulties of contracting; the transparency of SIBs; whether private investment in SIBs is, or can ever genuinely be, a reality; and the potential impact of private investment on the core values of public services.
Do SIBs suit complex problems?
You’ve got to be careful because social problems are by definition very complex. People are complicated and we live in a very complex world. There’s a lot going on. So focussing on a simple, short-term intervention to deliver a single outcome may not be the best approach.
The Riker’s Island SIB, for example, funded a behaviour modification for young people in prison. Those teenagers were then sent home with very limited skills often to distressed family and neighbourhood situations and to try to engage in an economy that had pretty much left them behind. But they had been taught how to be polite. That’s cheap and it certainly doesn’t hurt. But don’t we need a more comprehensive approach? Aren’t we simply treating the symptom and not the cause? Not surprisingly, some critics have said: “We really need some longer term structural change and this is just a Band Aid”.
Some SIBs are being developed in the US to fund pre-school provision. Indeed the roots for SIBs in the US lie in work done around developing private investment funding models for preschool provision. I can see the appeal. Preschool is inexpensive compared with the costs of early childcare. It’s relatively short term and you can get a good measure of the investment’s efficacy when you see how kids perform in kindergarten. It’s linked with some wonderful long-term improvements in health, education and employment.
When all these savings are calculated, one model found that preschool offers a 17 per cent internal rate of return, which is better than anything the stock market could give. And three quarters of that return is savings to government which offers the chance to get government to pay for SIBs out of savings in future programme costs. Nevertheless, it’s pretty heroic to assume that preschool provision can be credited with producing all the benefits you hope for when a child reaches adulthood.
At this point, it’s also worth remembering that SIBs – like many social policy innovations – are created around the assumption that you can take model programmes that have been proven to work in one place and then scale them up. But have you ever tried to make brownies for 40 people instead of ten? You actually have to change the recipe. You don’t just quadruple the recipe, because it won’t work.
I worry about the complexity of contracts that typically accompany SIBs. I’ve been studying contracting for 20 years in local government in areas like water and garbage collection. Those contracts can be quite complex, but they are simple compared with contracts for social services.
We also know that cost savings from contracting out water and garbage collection are at best ephemeral. In the longer term, markets require management, which can be expensive – especially when, at a local government level, there are not really markets for your public services. I measured how many alternative providers there are for any one of 72 different services that local governments provide in the US. On average, they have one alternative provider. This does not a market make. That’s why local governments do a lot of reverse privatisation or contracting back in – to ensure that there is some element of periodic competition.
High transaction costs
Transaction costs are also high to set up the contracts. Most local officials would say: “If I’m not going to get a ten per cent savings, I’m not going to go out because I’ll spend more than that just designing the contract.” Additionally, because public markets aren’t competitive, there needs to be monitoring, but monitoring is also expensive. That’s especially true for SIBs: they require high quality evaluation because the financial planning is based on delivery against metrics and, ideally, there should be a counter-factual, some sort of comparison group, not subject to the SIB.
SIBs lack transparency – there is a lot secrecy about discussions until the deal is done. People may not hear about a project until it has already been designed and, even then, documents are often unavailable to the public for scrutiny. It makes them hard to study and is a problem for open governance.
SIBs could actually promote inflexible innovation, because they typically involve a model programme for a process which has been proven and which is then locked into a contract. Fortunately, it’s usually only for three to five years, whereas public private partnerships for infrastructure might be 25 or 30 years. But that’s still a constraint on innovation.
Will the private sector invest?
The promise is that SIBs will engage the private sector as an investor in public services and so increase funding for effective programmes and build the political will for policy change. So the long-term outcome could actually be more public spending on these projects once the private investment has shown the value of investment.
That is problematic in the US context because, in seven of the ten projects that currently exist in the US, more than 50% of the finance is being guaranteed by private philanthropy. One of the key investors enjoying this underwritten status has been Goldman Sachs.
I had a student recently from the finance sector who did his Master’s thesis on SIBs. He concluded that the risk of SIBs is too high to attract private capital - a secondary financial market will be required to provide private sector funding. That’s something we should be watching for.
Public values at risk?
Some of the enthusiasm around SIBs is that we’re going to insert efficiency and investment logics to make the social sector better. That sounds interesting. But there is the risk that we will lose some of the values that have underpinned social service policies, such as social justice and citizen empowerment. If financial logics prevail over social values, that could undermine other social goals. And if there are savings from SIBs, why does government have to mortgage those savings to private investor profit, rather than spend it on future investment?
Then there are also those who worry that everything is priced in a SIB. I recently saw the play, “A Curious Incident”, about a child living with Asperger’s Syndrome. In the past, we would have justified spending on such a child as a matter of their right to an education and society’s obligation to accommodate all children in our world. But if we shift towards thinking about the “investible child”, justifying expenditure now by its returns in the future, will that child still get the investment s/he needs? I worry about social Darwinism creeping into social policy, especially as the clients’ voice is largely silent in SIBs.
Meanwhile, what role may intermediaries take in redefining the values of public services? The US already has a largely devolved social welfare state. SIBs may hand intermediaries even more power than they already have to determine what social policy looks like.
New political allies
So there are many challenges. But it’s also important to recognise that SIBs are making a difference, sometimes in unexpected ways. For a start, it’s clear to me that many city managers are trying to ride this wave of marketisation in social policy and get something positive out of it for their communities.
So for example, all the discussion about investment returns on preschool provision is bringing in new political partners to the cause of developing better services for children. Important CEOs in the local communities are saying: “This is important. We want to see investment here”. I remember the head of one of the public social welfare agencies saying: “This is the first time I’ve ever been able to go to city council and tell them that their expenditures are investments, positives rather than negatives.” This was because she was able to talk about the return on investment. SIBs often focus on quite short term returns. This style of language gains political attention, and motivates leaders more than long term returns because, they think: “In the long run, I’m not going to be in office”.
New policy tools
We are also seeing the development of new policy tools. People in the welfare arena are accustomed to rules and rigid regulations governing social services. In contrast, the language of economic development encourages entrepreneurship. In the social welfare world, people find it liberating to be in a culture in which they feel incentivised to deliver a policy rather than simply being governed by regulations and rules.
SIBs aren’t going away. They are growing. In the US they are likely to mushroom, but without the careful study and scrutiny that such a policy reform requires. We should not be blinded by the dream of greater investment and greater program effectiveness. Neither has been proven in the early SIBs. And there are clearly many issues about management and public values. We need to be honest about what really is happening. We should give special attention to how to maintain the important values that underpin our public services.
Dr Mildred E. Warner is a Professor in the Department of City and Regional Planning at Cornell University. Her published articles on SIBs and government services can be found on her website www.mildredwarner.org
Mathematical models can test multiple variables cheaply and quickly, giving early indications of what really matters. That helps in designing pilots and understanding how context can affect a policy’s success, argues Dr Zaid Chalabi.
I was part of a team that evaluated the implementation and cost-effectiveness of the government’s Cold Weather Plan (CWP) for England. The CWP is a guidance document which aims to reduce the thousands of additional deaths that typically occur in England when temperatures plummet. The fall in temperature can increase risks for elderly people as well as those with heart or breathing problems and other chronic conditions.
The plan’s principle is that, when cold weather is expected, the authorities are alerted and they can enact various measures, suggested in the CWP. They might, for example, contact or visit vulnerable patients, check that they have medication, that they are warm and have enough food to last a cold snap. The CWP guidance is quite general and it is up to each local authority to implement the plan in its own way.
Assessing the Cold Weather Plan
There is a lot that we do not know in assessing the CWP’s cost-effectiveness. How fully will each health authority implement the plan? Which aspects will they focus on and what impacts are made by each particular action?
Somehow, we need to know how the CWP, implemented in its various ways, might impact on the health of the population and also avoid hospital admissions to save on costs to the NHS. The costs of elective admissions - postponed if there are weather-induced surges in emergency admissions - also must be assessed. We must evaluate the quality of life, and the years of life extended by the CWP, as part of the final cost-effectiveness calculation.
Clearly all of this is hard to evaluate because there are so many variables. Also the circumstances may not occur often: a decade of mild winters might pass before there is a harsh year. Because the CWP has been running for only a few years, we do not have much data, yet it is also a life and death scenario, so policy makers need advice on how best to implement the CWP in order to deploy limited resources well.
Options for evaluators
What should evaluators do? A meaningful analysis of the outcomes of the CWP would require the ability to compare vigilant health authorities with those less engaged, and to evaluate the CWP over several harsh winters. We cannot afford to wait that long, and it may be unwise to stop some authorities making preparations to protect their populations.
Modelling helps when data is missing
We adopted mathematical modelling as the best approach in such circumstances, where lots of important data are not yet available. It is possible to search the literature or seek expert opinion to make estimates for almost every scenario and action. We can estimate, for example, the benefit to patients’ health of being contacted during cold weather. Theoretical costs can also be factored in for the different options – be it a phone call or a more expensive actual visit. These figures can never be absolutely accurate. But, by building in as much data as possible, the model begins to reveal which variables are significant and which ones do not matter much.
Modelling also provides indications of where the extra costs of the CWP might occur (probably in social services) – and which areas are likely to enjoy savings (probably acute hospitals). In health and social care, implementation of cost-effective innovations is often held up because silo working means there are both losers and gainers. Because the losers may not be compensated, the policy may not be implemented, thus depriving the system of a net gain overall. Modelling can identify who wins and who loses, giving policy makers the chance to equalise the outcome between the silos.
This approach to evaluation, in identifying the many variables and assumptions, also helps us to see gaps in knowledge so we can focus research on gathering any important missing data.
Modelling and other public health interventions
The value of modelling goes beyond policies such as the Cold Weather Plan that can take years to bear fruit. Most public health interventions occur in complex environments and involve multiple variables. It is important to understand the role that context plays in their success.
Models provide an opportunity to vary contexts and see what matters and what does not. They are the obvious precursor to setting up a pilot that can then be designed around key factors that seem to be important, regardless of context. Inserting modelling into the process of early evaluation, before piloting, could be vital if we are to escape the scourge of successful pilots that are not rolled out, because they seem to work only in one place.
Dr Zaid Chalabi is Associate Professor in Mathematical Modelling at the London School of Hygiene and Tropical Medicine.
Social Impact Bonds and similar financial vehicles may seem guaranteed to deliver key values, but they raise important moral dilemmas.
The impulse to use new private sector approaches to funding public services, via Social Impact Bonds (SIBs) and other social investment vehicles, is fraught with ethical dilemmas. We should be cautious before rushing into “marketising” some public services – packaging them up as commodities that can be provided for a price by any supplier judged appropriate.
It’s easy to miss these problems because many SIB advocates hold strong, high-minded and hopeful moral positions. They’re often tired of just trying to do the right thing: they want to achieve real, measurable improvement in people’s lives.
SIBs and Utilitarianism
The views of those supporting Social Impact Bonds are typically rooted in utilitarianism which attaches moral value to the consequences of actions, not merely to the intentions. This approach contrasts with deontological theories, which hold that no matter how morally good their outcomes, some choices are always wrong. The utilitarian tradition can also be contrasted with virtue ethics. This focusses on what kind of people we want to be so we can work out which actions are allowable.
In contrast to deontological theories and virtue ethics, utilitarianism, which underpins the SIBs’ approach, dwells on outcomes. One popular form of utilitarian theory is “prioritarianism”, which attaches higher moral value to improving things for particular groups of people, usually the neediest and worst off. The goodness of this outcome confers rightness on whatever was done to arrive at that outcome.
At first sight, adopting such a consequentialist approach may seem to be a no-brainer. Who could argue with a standpoint that’s focussed on using SIBs to provide concrete help for the most vulnerable? But this seemingly good sense can sometimes blind us to real dilemmas posed by SIBs, and similar examples of marketisation in social services.
For a start, what timeframe should be used to assess outcomes? SIBs are normally planned to deliver their benefits in between two and seven years. These are timeframes after which a private equity fund, or venture capital fund, would expect to exit a programme and realise a return on their investments. But there is concern that such timeframes may not capture all the main benefits – and the dis-benefits or negative externalities - that might manifest themselves subsequently.
Is it possible, in any case, to measure all the outcomes? And, if we can, is it always possible to establish a causal link between an intervention or service, provided via the SIB, and the outcomes measured subsequently? Such uncertainties make it difficult, sometimes, to assess the ethical value of the SIB approach.
Should some social problems be free of markets?
For reasons not to marketise social services, I draw on work by Debra Satz, a philosophy professor at Stanford University and author of “Why Some Things Should Not Be for Sale: The Moral Limits of Markets”. Satz is a critic of “noxious markets”, which, she says, tend to feature vulnerable people who have little say in what is happening to them and who may be damaged by being involved in such a market. These “noxious markets” are, she finds, often characterised by excessive profits. She worries about marketisation undermining our civic values and society in such cases.
I also draw on the work of Michael Sandel, a Harvard University politics professor, whose book, “What Money Can't Buy”, studies the moral limits of markets. He invokes what I call an “icky feeling” about markets in certain goods on the grounds that the exchange, and the nature of the goods, may be devalued by using a market.
These commentators challenge us to consider whether we feel comfortable with a market for kidneys or for other human organs. Would we go along with a market for babies? Would we like to trade them? Would it devalue humankind if we were to do this? How about prisoners? Is it morally acceptable to profit from the misery of others? Is it OK if we “fancy a flutter” with someone else’s well-being, as the Economist asked in an article about the first Social Impact Bond, designed to reduce reoffending rates among former prisoners in Peterborough?
Even if one accepts that SIBs and other forms of marketised social investment are the best that can be done in a difficult world, there are other important issues to address.
The risks of marketization
First, could this kind of marketisation exclude some deserving recipients? Take, for example, help for potential paedophiles. It might not be easy for this group to gain access to resources, using these kinds of mechanism. Which social investor would be prepared risk their brand reputation through such an association?
Another important issue is whether marketising social interventions might affect staff motivation. Describing activities in terms of profit-making can undermine the reasons why people want to provide a service. For example, Richard Titmuss’ classic 1970 study compared voluntary blood donation in the UK with the system in the US that offered donors money in exchange for blood. It concluded that paying for blood reduced not only the quality but the supply of blood because some donors’ motivation was potentially damaged by marketisation.
And lastly, we should also consider possible ill-effects on recipients. How does it feel when a recipient’s problems are generating profits for someone else? How does it feel to be a profit-centre? In short, we should recognise that marketisation can create many potential issues affecting relationships for volunteers, staff and those who receive a social intervention or service.
There are also risks that market systems which replace public values can lead to behaviours that actually worsen social problems. Take, for example, private prisons in the US: providers have spent millions of dollars lobbying for increased rates of incarceration and extending custodial sentences.
Ways to tackle the risks
Where do all these risks leave the question of marketisation, particularly if it looks to be the only way to attract resources? We shouldn’t bury our heads in the sand and conclude that there is nothing that we can do. If some degree of marketisation is inevitable, we should address potential issues.
It may, for example, be better to view markets as complementing rather than supplanting non-profit social services. One can use them around the edges of traditional provision, maintaining an awareness of the social and political implications implicit in these market processes and controlling their worst excesses. This approach would militate against SIBs, and similar approaches, becoming more than an adjunct to more mainstream delivery systems.
One possibility is using experts to constrain profit-based decision making, thus placing moral limits on the market: in other words, regulation of approaches such as Social Impact Bonds. It’s also important to improve the accountability of providers as much as possible, particularly to those receiving services: regulation can reduce power differentials between providers and recipients. There should also be transparency about levels of profit and the structure of deals. Broader social impacts should be measured and monitored.
With these safeguards, the marketisation of social services may be justified even though problems remain, as demonstrated by the behaviour of the US providers of private prisons. Such experience ought to make us reflect about the kind of world in which we wish to live, before we rush into wholesale marketisation. We must not ignore the moral and political implications of actions that might previously have made us feel highly uncomfortable.
Dr Julia Morley is a lecturer in the Department of Accounting at the London School of Economics and Political Science.
Some claimed benefits of Social Impact Bonds remain unproven. But they tackle long-term weaknesses in public service delivery by concentrating on outcomes, early intervention and collaboration.
Governments around the world are increasingly funding health care and other public services through Social Impact Bonds. SIBs claim to deliver better, more cost-effective public services by recruiting private investors who gain their returns by achieving pre-agreed outcomes. Earlier this month, for example, Prime Minister Theresa May promised to fund improved mental health services through this mechanism.
It’s easy to find serious flaws in Social Impact Bonds – particularly with respect to their main selling points. Like many social policy innovations, SIBs are championed for their efficiency and effectiveness, and for having superior - and better evidenced - impact on social problems. Experience so far doesn’t justify these assertions to any great extent.
Claims of greater effectiveness and rigour for SIBs remain unproven. Their supposedly enhanced efficiency – saving public money – looks largely spurious. However, we shouldn’t simply dismiss SIBs because of these doubtful claims. SIBs are a force for some considerable good, albeit in the ways that their champions tend to emphasise less. That’s because SIBs seem in practice to advance three important developments where conventional public services typically struggle: outcomes, prevention and collaboration.
SIBs and key public policy objectives
First, like Payment By Results (PBR), a central principle of SIBs is that they concentrate on the delivery of outcomes, on the consequences of public services rather than on their processes or outputs. This shift represents one of the most important – and potentially impactful – reforms in public service delivery globally. Utilitarianism may have been oversold as a moral philosophy but, as a general approach to deploying public expenditure, there’s a lot to be said for it.
Second – and this is connected to my first point – SIBs are focussed on shifting practice towards prevention and early intervention, because that’s how they can identify long-term savings.
Third, SIBs help increase collaboration in welfare delivery – they are designed to align incentives for multiple stakeholders to deliver a collaborative goal. Cooperation, with everyone pulling together, has bedevilled complex welfare systems. However, it is needed by SIBs if they are to achieve more with less and thus free up savings that can be translated into investor repayments. As a result, their approach offers a breath of fresh air to public service delivery.
So, although the headline claims for SIBs are largely unsubstantiated, some emerging practice is encouraging and much needed. Nevertheless, the verdict for now on SIBs should be “undecided”, given what’s at stake: the supposed benefits of SIBs and similar financial vehicles underpin arguments for the use of private capital in the delivery of public welfare services. I’d advise policy makers to avoid leaping too quickly to adopting SIBs on a grand scale, given what we know so far. Much of this is examined in a book that I’ve co-edited, entitled “Social Finance,” in which leading scholars in the field examine the issues, and in work done by Oxford University’s Government Outcomes Lab.
In addition to the questionable hype about SIBs, there remain further, serious questions about making profits out of vulnerable people, particularly in developing countries. There are also worries about sustainability - that SIBs will fail providers and users once contracts are completed. Finally, as with all PBR approaches, there are doubts about whether SIBs will be value for money in the longer run.
Such a shaky verdict may seem dissonant with a policy option whose popularity is sweeping the world. SIBs are everywhere. If your home country is not on the SIB map, it will be soon. Take Japan for example. It’s poised to join the scores of countries that are laying the foundations for SIBs. At the end of the summer of 2016, its government announced $200m for Development Impact Bonds (DIBs) (see below). So lots of Japanese SIBs are on their way.
Four myths about Social Impact Bonds
Returning to the “hype” I mentioned initially, let’s consider, first, the four core claims, or myths, that underpin the popularity of SIBs: that they are more effective and also more efficient in delivering superior social outcomes; they can make money for investors; and they are more rigorous in their measurement of change.
1. Superior social outcomes?
As far as superior social outcomes are concerned, the most famous UK SIB - for reducing reconvictions among ex-prisoners in Peterborough – did well. Its early performance was promising, suggesting that - once completed - the SIB would exceed its targets and deliver the superior social outcomes that were expected. It was just unfortunate that the experiment was terminated early for other reasons.
Meanwhile, the New Horizons SIB, developed for the Department for Work and Pensions, was meant to reduce numbers of NEETs - young people “not in education, employment or training”. After a rocky start, the SIB outperformed its targets, the contract was paid and the follow-on contract was awarded, albeit at a much discounted price.
There was also qualitative evidence from service users who said things such as: “If it weren’t for that practitioner, I wouldn’t be here today and that’s actually the truth. I’d be dead. Everyone I knew thought I probably wouldn’t make it until the end of the year.” So, there was impact. It might be of a young person who had been excluded from school who said: “Somebody came to see me, worked with me, got me back into school and back into education”. So, the impact was superior at least to what had gone before. But, maybe, there was no help there before, which makes it hard to compare.
So, there is some evidence of SIB successes - exceeding pre-defined targets, around reoffending, educational attainment, the securing of stable accommodation and full time employment. But SIBs have also failed to hit targets in some key areas and there are problems where there has been no baseline.
Sometimes, the performance metrics also seem artificial. In the Peterborough SIB, it’s plain that the particular targets were created to be very achievable because people wanted the programme to succeed. In short, it is possible that improved social outcomes are being achieved. There certainly are evident social outcomes. But it’s more difficult to say that better social outcomes than those achievable by other methods have been proven.
2. Do SIBs save public money?
The claim that SIBs can save public money is important. The financial logic works like this: once a SIB is in place, the cost to government of the status quo is reduced such that when you add the cost of the new intervention - plus a nice payback to investors - there is still a saving to the public sector.
Indeed, it’s a fundamental assumption, in these times of austerity, that SIBs save money. This is the simplest explanation of why SIBs are proliferating: they appear to give you something for nothing, which is very attractive if you are an impoverished service provider or an impoverished government. SIBs seem to get somebody else to pay for something up front, so you pay nothing, or, if you do eventually pay, you pay less than you would have paid anyway. Also, with any luck, repayment will be over a five or seven year period by which time, at the very least, you’re no longer the minister. So the pay back becomes somebody else’s problem.
There is certainly some technical truth in the saving money argument: services delivered now – because someone else is paying upfront – probably have more value than those delayed five years until the Government has the cash. Yet, beyond what’s called the “present net value” argument, the claim that SIBs save public money is probably the most spurious myth of all. In truth, it’s likely that the net cost of a SIB will be higher than the alternative and, even if it is lower, realising the savings may be nearly impossible.
For a start, some SIBs actually increase costs. For example, the Greater London Authority Homelessness SIB aimed to deliver support to people who were outside the welfare system, so it inevitably increased direct costs to government.
SIBs also have very high transaction costs. We underestimate the amount of management time that’s required to run a SIB and we radically underestimate the demands for information that investors make. So there are hidden transaction costs, often borne by service providers.
Realising notional savings is also difficult. If a SIB reduces reoffending how should that be expressed? Do you weigh up the numbers of sacked policeman, closed courts, shut prisons, prison officers put out of work? Realising those savings is clearly not easy. So it’s hard to see how savings estimates ever become a reality. Yet, saving money is one of the biggest sells to government regarding SIBs.
3. Making money for investors
It’s also claimed that SIBs make money for investors. There is some evidence of real returns but a lot of this depends on how the SIB is structured. The Peterborough SIB, for example, required investors to wait a long time to get any payback. In other SIBs, investors get some payback much more quickly, once there are some evidenced results. And, of course, the investor, who might be putting in a £1m, may never actually put all the money in up front. It’s typically staggered over a period. So if an investor has to put in £250,000 every three months, but starts getting back £100,000 after four months, the risk is much reduced and the returns are higher.
Financial risks in SIBs vary greatly. With Peterborough, investors took a 100 per cent risk – they would have lost everything if they had not hit their outcome targets. But this approach is less common than originally envisaged. The SIB at Riker's Island prison in the US, which Mildred Warner discusses in her contribution to this series of blogs, involved zero risk to the investor, Goldman Sachs, whose risk was underwritten by the philanthropists.
A famous social investor, whom I won’t name, was so angry about the Riker’s Island SIB that he practically had steam coming out of his ears. “This is the worst thing that could possibly have happened," he said, “because every single US SIB after this will say: ‘Why should I take any risk because Goldman’s didn’t?’ ” Elsewhere, in Australia for example, there is a two-tier risk model, with lower risk bonds underwritten by the Government and offering a low annual rate of return.
In short, there is evidence of some profitable returns from SIBs. But there is a wide range of experience already among investors from losing money to making money, to making some money and making money under circumstances that feel uncomfortable, and so on.
It is said that SIBs provide better evidence of impact. There is certainly no single approach to measuring impact across SIBs. There is a vast range from randomised controlled trials to SIBs where there is little or no comparison with other delivery options.
Peterborough was rigorous in that it measured reconviction events among its prisoner cohort, compared with the rest on the National Offenders Register. But the vast majority of SIBs lack such rigour. Most offer “validated administrative data”: somebody says this is what the intervention achieves and the commissioner says: “OK, I’ll pay on that” or “I won’t pay on that”. The metrics used also vary from quantitative to qualitative. So this claim about superior metrics is again not really proven.
SIBs do have some genuine strengths on the ground. They are part of a wider shift in programmes towards outcomes which should, if successful, deliver more meaningful public services.
The majority of SIBs that I’ve studied focus on prevention or on an early intervention. Very often they put larger amounts of capital into programmes up front than could be found using conventional funding. This is because programmes, particularly run by the third sector, are often small and don’t have the cash to upscale quickly. So, in speeding up investment and concentrating on early intervention, welfare benefits can potentially be achieved earlier than otherwise.
SIBs also allow collaboration across sectors in a way that is almost unique. In other words, if a SIB works, it should align the interests of the third sector, the government and private investors perfectly. So they should strive to achieve a particular outcome because they would benefit in different ways by that being achieved. This is an important asset for SIBs because the fragmentation of public services is a well-rehearsed obstacle to tackling pernicious problems.
It’s also probably true that SIBs are contributing, at least, to the building of a social investment or finance market – for better or worse – although that market is probably too small at the moment to have a major impact.
There is a big moral question, which Julia Morley discusses in her contribution to this blog series, of whether it is right to privatise, financialise or marketise - whichever language you prefer - social outcomes. This dilemma is perhaps best demonstrated in the case of Development Impact Bonds (DIBs), in which overseas development projects are debt-funded, like SIBs, with payments contingent on achievement of agreed targets. Making profits in this way from tackling the problems of the most vulnerable looks even starker in poor developing countries.
DIBs could be seen as representing a whole new wave of colonialism in which rich financiers say: “Not only can we solve your social problems, but we’re so clever we’re going to make money out of it, and take it back to London and Bonn and Frankfurt and New York”.
It’s also not clear how service providers will sustain their work once contracts for SIBs – or DIBs – are complete. What happens next? This issue affects service users who might say: “I was having this wonderful service and then the SIB stopped. Nobody’s picked up the slack and now I’m left with nothing.” For the service users, as well as providers, this lack of exit thinking is a real problem.
NAO question mark on value for money
But perhaps the biggest question mark brings me back to the beginning – value for money. The National Audit Office struck a note of caution recently in its report, entitled “Outcome-based payment schemes – government’s use of payment by results”. The NAO was mainly talking about PBRs but SIBs sit within that field. It advised caution in the adoption and embracing of such financial approaches. It said the evidence of impact was, at best, mixed and warned about the risk that such approaches might not, in the long-run, offer value for money.
My own overview of the subject suggests that there is very limited evidence for some of the claims made for SIBs. But they do, in reality, offer some innovative approaches – around outcomes, prevention and collaboration – about which public services have long aspired but rarely delivered.
We shouldn’t allow these benefits to be lost. Likewise, we shouldn’t be seduced by the hype.
Dr Alex Nicholls is Professor of Social Entrepreneurship within the Skoll Centre for Social Entrepreneurship at Saïd Business School, University of Oxford and co-editor of the book, “Social Finance”.
Evaluations of national policy pilots are often embarked on with high expectations and end with a sense of frustration on all sides. Policy-makers often expect clearer, and more positive, verdicts from evaluation than researchers are able to provide; researchers hope for their findings to be more influential; and implementers in pilot sites struggle to put in place what they think they are being expected to deliver within the limited timescale of the pilot while wondering what they have gained from either the pilot programme or the national evaluation.
To ease some of these frustrations, we have developed guidance aimed primarily at national level staff involved in policy-making and in initiating policy-relevant pilots and their evaluations. We think the guidance will also be helpful to evaluators. Our advice stems from both experience and analysis of the fate of policy pilots (Ettelt et al, 2015a; Ettelt et al, 2015b). Two observations, in particular, from evaluating policy pilots in health and social care have shaped our thinking.
The first observation is that many times it is not clear what an evaluation is intended to contribute to policy development. This lack of clarity is often a symptom of a deeper problem which has more to do with confusion and conflicts over the reasons for piloting than with the evaluation itself. Indeed, the objectives of the evaluation can be perfectly clearly expressed, and yet it can entirely ‘miss the point’ if the purpose of piloting is not thought through. As we have argued elsewhere, policy pilots can serve different purposes, many of which have more to do with the realities of policy-making, and the dynamics of policy formulation and implementation, than with piloting for the purpose of testing effectiveness (Ettelt et al, 2015a). Different groups involved in a policy pilot can have different ideas about the purpose of piloting. Also, these purposes often change over time, for example, as a consequence of a ministerial decision to roll out the policy irrespective of whether the evaluation has been completed or not. For example, the Direct Payments in Residential Care pilots, which PIRU is evaluating, were rebranded early in the life of the programme to become ‘trailblazers’ as it was decided, ahead of the results of the pilots, that direct payments would be rolled out nationally in 2016 alongside other aspects of the 2014 Care Act. However, the policy context of the ‘trailblazers’ continues to change. As a result, the Department of Health is currently reconsidering whether direct payments should move forward at the same speed as expected earlier.
We think it is important that the goals of such programmes are stated explicitly and that their implications are thought through carefully at the beginning of a pilot programme while it is still possible to make adjustments more easily than later in the process. This is also the time to identify the target audience for the evaluation. Whose knowledge is the evaluation aiming to contribute to? There are likely to be important differences in the information needs and preferences of national policy-makers and local implementers that require some forethought if they are to be addressed adequately.
The second observation is that, under the influence of the mantra of ‘evidence-based policy’, policy-makers increasingly feel that they should prioritise specific research designs for the evaluations of policy pilots, especially experimental designs. Yet, this consideration often comes too early in the discussion about pilot evaluations and is introduced for reasons that have more to do with the reputation of the design as producing particularly ‘valid’ evidence of policy effectiveness than with its appropriateness to generate insights given the objectives of the specific pilot programme. The choice of research design does not make a programme more or less effective. Conducting an RCT is pointless if the purpose of a pilot is to find out whether or not, and, if so, how, a policy can be implemented. In such a situation, the ‘active ingredients’ of the intervention have not yet been determined and thus cannot be easily experimented with. The Partnerships for Older People Projects (POPPs) pilots, conducted in the mid-2000s, are an example of a pilot programme that brought together a large number of local projects (of which about 150 were considered ‘core’), indicating an intention to foster diverse local innovations in care, with an evaluation commissioned and designed accordingly. However, this did not stop national policy-makers subsequently changing direction and demanding a robust outcome analysis from a pilot programme and related evaluation which were both established to meet a different set of objectives.
A similar tension between piloting to encourage local actors to develop their own solutions to problems of service delivery and the desire for definitive (cost-) effectiveness evaluation of ‘what works’ can be seen in other pilot programmes. For example, the Integrated Care and Support Pioneers were selected as leaders in their potential ability to develop and implement their own solutions to overcoming the barriers to integrating health and social care. Yet, the evaluation requirement includes a focus on assessing the cost-effectiveness of integrated care and support. This is extremely challenging in the face of such a diverse programme.
Beyond our two initial observations, the question of ‘evaluability’, which is relevant to all policy evaluation, is particularly pertinent in relation to RCTs and similar experimental designs. RCTs require a substantial degree of researcher control over both the implementation of the pilots (e.g. a degree of consistency to ensure comparability) and the implementation of the evaluation (e.g. compliance with a randomised research protocol). This level of control is not a given, and the influence of researchers on pilot sites is much more likely to be based on negotiation and goodwill than compliance. This does not mean that conducting RCTs is impossible, but that pilot evaluations of this type require a significant and sustained commitment from pilot sites and policy-makers for the duration of the pilot programme to stick with the research protocol, and manage the added risk and complexity associated with the trial.
To help policy-makers to make these decisions and plan (national) pilot programmes and their evaluations better, we have developed a guidance document. ‘Advice on commissioning external academic evaluations of policy pilots in health and social care’ is available as a discussion paper here We are keen to receive comments, addressed to Stefanie.Ettelt@lshtm.ac.uk.
This is an expanded version of an article written for the December 2015 edition of ‘Research Matters’, the quarterly magazine for members of the Social Research Association.
Ettelt, S., Mays, N. and P. Allen (2015a) ‘The multiple purposes of policy piloting and their consequences: Three examples from national health and social care policy in England’. Journal of Social Policy 44 (2): 319-337.
Ettelt, S., Mays, N. and P. Allen (2015b) ‘Policy experiments: investigating effectiveness or confirming direction?’ Evaluation 21 (3): 292-307.