Event Details

This event has ended. We have added our summary and insights from the workshop.

DESIAP KL report, Impact Evaluation in Designing Social Innovation: Insights from DESIAP KL Workshop and Symposium, written by Akama, Yee, Hill and Tjahja, was launched in Singapore 2019 at a gathering funded by Northumbria and RMIT University (ECP). This event took place in Singapore because it is a key hub of funding agencies with regional offices that support social impact work in Southeast Asia. Invitations were sent through networks affiliated with DESIAP, such as Asia Venture Philanthropy Network, Social Innovation Exchange and Asian Women Social Entrepreneurship Network. This gathered 9 leading funding agencies in the region to begin discussions on ways to co-explore alternative impact evaluation approaches. The participants that gathered included Kenn Foundation, Lien Centre for Social Innovation, Tondo Foundation, JustCause, Asia P3 Hub, Asian Venture Philanthropy Network and the British Council.

Salient insights from the report were presented as a catalyst for discussion:

• How designing social innovation practices shape the form and purpose of impact evaluation.
• How evaluation is embedded in designing social innovation processes.
• Questions and propositions for understanding impact evaluation.

In traditional evaluation the primary purpose of the evaluation is to assess and report on the efficacy and outcome of a project. In most cases the primary audience for the evaluation is the funding or commissioning party. The emergent nature or the designing social innovation means that the primary purpose of evaluation in that context is to understand what has been done, what has been achieved to date and how best to progress the work. It has a learning, adaptation and an assessment function. The community led (or centred) nature of the work means the evaluation should be based on parameters that make sense to the community. The primary audience for the evaluation is that community and the implementation partners (practitioners, funders and commissioners) working with the community. Sometimes there can be a tension between the needs of the community and the funding or commissioning party. In many cases these tensions reflect how funder or commissioning party has not fully understood the implications of adopting a co-design, participatory or human-centred design approach as opposed to a more traditional transactional project model. The above tensions can be exacerbated when there is not an engaged relationship between the funder and the community, and also due to lack of understanding context of the project by the funder. This leads to misalignment of expectations, implementation, constrains value that can potentially be delivered through projects, and risks impact outcomes for all concerned. Key findings from the report catalysed discussions and questions from the participants that echoed the findings. All participants agreed with the need to shift traditional evaluative approaches to alternative models of impact to enable communities to be central in deciding how, why and what is important, so evaluative thinking is built into the work or project from the very beginning. In the discussion, shifts from a structured, externally led (or defined) practice to an emergent, community led (or centred) practice were identified as having significant implications for evaluation – how it is approached, who and what is defined as the basis for success, how that is assessed, what and who the evaluation is for and the role that the evaluator is expected to play. Participants further identified entrenched (or bureaucratic) organisational practices observed as challenges that inhibited cultural and systemic shifts. In groups, we brainstormed ‘what if’ scenarios as a generative exercise to imagine a way through commonly identified challenges:

• Written reports and spreadsheets that fail to capture meaningful insights, learnings and transformation observed by the beneficiaries > ‘What if’ photos, videos and audio diaries captured on smart-phones were shared instead? Visual language can often be powerful and accessible for many community groups.

• Predefining evaluation and impact criteria > ‘What if’ the ‘Theory of Change’ can be created together among funders, intermediaries and beneficiaries?

• Gaps in power, understanding and relationships > ‘What if’ there are quality conversations between funders and beneficiaries that moves from talking about KPIs to values that are often invisible? Can a ‘neutral party’ broker or facilitate this conversation?

• Agreement in theory but harder to do in practice > ‘What if’ there were exemplary case studies that articulated how organisations shifted from traditional evaluative approaches towards? Could this facilitate new types of leadership to enable cultural shifts in organisations?