Jasjit Singh, Associate Professor of Strategy at INSEAD and Co-Director of INSEAD Social Entrepreneurship Programme, spoke about measuring social impact at the 2016 DBS Foundation Social Enterprise Summit. Excerpts of his presentation have been adapted and reproduced here.
The case for setting up a car sharing business to create a positive environmental impact is often taken for granted – it would reduce carbon footprint, and there would be less congestion on the roads. But is it really that simple? Consider these questions:
- Who are the people who signed up for this service?
- What happened after they signed up?
- What would they be doing if they did not sign up?
If, for example, the car sharing users would have walked or taken public transport otherwise rather than buying a car, while car owners continue using their own cars, how exactly is the intended environmental impact of car sharing going to come about?
Just assuming that a well-intentioned venture will automatically have the intended positive impact is not appropriate. We need to rigorously evaluate whether the impact does get realised.
This means checking whether the assumptions behind our model for impact really hold true in reality.
Rigorous studies have examined the issue by analysing large-scale data. While some benefits have been proven – carbon footprint has indeed gone down a bit – car sharing has not created quite as big a positive impact as some companies claim in selling their story.
Similarly, in the context of improving the lives of the poor, when evaluating the impact created by your social enterprise or impact-driven business, it’s important to have a clear view of how the impact is generated (your “theory of change”), and the realised breadth and depth of the intended social impact (i.e., the number of people positively affected and the degree of change that has been made).
Without careful analysis, the social impact of ventures often gets exaggerated. For instance, academic research has revealed that the strong claims of microfinance leading to substantial poverty reduction are not supported as convincingly by rigorous impact evaluation studies.
The PlayPump experience
In Africa, a social entrepreneur called Trevor Field introduced the PlayPump Water System to villages in the early 1990s. It was a social innovation meant to alleviate the difficulty girls and women faced in walking long distances to get water, but it ultimately failed.
What happened? The initiative had scaled up too fast before its impact was fully studied, as the agenda had been hijacked by otherwise well-meaning donors who just wanted to see the innovation quickly reach millions of people.
The PlayPump Water System consisted of a merry-go-round attached to a water pump. When children played on the merry-go-round, the movement powered a water pump that drew water out of an underground bore-hole into a water storage tank.
Water could then, in principal, be easily accessed by anybody who needed it. To fund the maintenance of the system, billboards on the water storage tank would provide advertising revenue.
While the project was still in the early stages, with just a few PlayPumps installed, it caught the media’s attention with events such as Nelson Mandela visiting a school with a PlayPump. The innovation’s visibility skyrocketed. Funds poured in, and Field was encouraged to quickly scale up his initiative before its effectiveness had been well-established. By 2008, 1,000 PlayPumps had already been installed.
Watch PlayPumps roundabout water solutions
Only when things started to go really wrong did critics take notice of the flaws. For example, the PlayPumps required children to “play” for a long time – by some calculations as much as 27 hours (yes, 27 hours!) a day to generate water to fully meet daily needs for a community.
They were also terribly inefficient financially, costing four times more than a traditional hand pump capable of pumping more water per hour despite being cheaper. The revenue model based on billboards did not work so well in practice either: companies were not interested in advertising in remote locations where the pumps were needed most.
So, while the PlayPump Water System sounded great in theory, it failed in practice. It had been scaled up too fast, before anybody had bothered assessing the real community-level impact of this water system. Donors were focused on the wrong measurement metric, namely, how many pumps were installed a month – adding pressure to just expand without pausing to evaluate real impact.
The intentionally for creating a social impact is what sets a social enterprise apart from a regular business that just happens to do social good as a by product of its profit-centered drive for selling useful products and services.
But actively monitoring social impact is what will help a social enterprise continuously improve and really achieve the impact it desires. The consequences of not doing so can derail even the best-funded social enterprises.
Here are some steps to help you get started on measuring the social impact of your enterprise.
Connecting strategy and measurement
1. Start by defining the basic value you are creating.
Go beyond good intentions and rigorously pursue making a real difference. Here are some initial questions social enterprises should consider: How are you creating value or fulfilling a need? Who is the customer? Which part of their life’s problems are you addressing? What is your competitive advantage, compared to current alternatives?
A focused needs assessment will ensure you study the current research available; build upon established research, best practice and standards, instead of reinventing the wheel. It will help clarify your enterprise’s mission and impact, and accordingly, the primary metric for measurement later on.
2. Assess resources available and how you would deliver the solution.
After defining the problem, you now have to consider how to operationalise your solution both commercially and to create social impact. This includes studying issues such as affordability, access to the right resources, monetisation and delivery methods.
Once you have defined the intended impact, goals and strategy, next up would be to convey all these to your stakeholders and get them onboard. Your stakeholders could include investors, communities, employees, non-governmental organisations and governments. Early alignment would not only aid in keeping your eyes on your primary goal but also go some way in pre-empting any conflict in goals later on.
Worth noting as well: since it’s hard to predict the final outcome, it’s important to start at a small scale first and assess the outcome before scaling up. So all the talk about “design thinking” and “lean startup” methodologies is truly applicable here.
3. Finally, assess your outcomes
Together with your stakeholders, choose a time horizon and level of rigour all parties agree would be useful and practical when measuring outcomes. Do take into consideration any limitations you may face.
Your measurement and evaluation should be based on your organisation’s primary metric, mission and impact model as defined earlier on. While it might be nice to have other metrics for assessment, there must be no compromise with the primary metric.
Questions you should ask:
What are the immediate signs that things are working out as planned? What are the medium-run and long-run behavioural, social and economic effects that should take place? And finally, and most importantly, what would have happened if the solution was not implemented?
To prove that there has been social impact, you will need to go beyond just the number of people impacted and show that their lives have been made better because of your social enterprise’s work. Iteratively experiment, measure, learn and improve before scaling up.
What distinguishes a social enterprise from a traditional non-profit is asking the question of how far the need or problem can be addressed through a market solution (good for financial viability). But you do not have to do it alone. What kind of complementary activities or partnerships can help fill the gap? Who can help?
Finally, no matter your results, remain patient, humble and inquisitive in problem-solving. Ultimately, measurement should not be seen as a chore. Rather, it’s an effective way to refine your social impact model, and an opportunity to approach stakeholders and exhibit the changes you’re making.
Download the report that captures the reflections and insights shared at the DBS Foundation SE Summit 2016: Hard Truths and Honest Conversations.