How many conferences are you invited to speak at, help fund or attend each year? 10? 100? You could spend your entire workday (followed by your entire evening) reading publications that share lessons learned about how to make international development programming more effective. How do you assess which knowledge management activity offers value to you?
Making Cents International considers the question of how to turn information into knowledge and improved practice from the point of view of an organization that manages a global knowledge management (KM) platform.
Meeting the needs of the global youth population demands evidence-based, scalable, and sustainable initiatives. With those requirements in mind, Making Cents offers a demand-driven KM platform that builds on the capabilities of positive youth development stakeholders worldwide to design, implement, and evaluate high-impact youth economic opportunity programs.
The annual Global Youth Economic Opportunities Summit is the flagship activity for the KM platform. The summit anchors five interconnected and mutually supporting activities that include communities of practice and networks that generate knowledge products, webinars, an online learning hub, and a meeting of funders.
What To Measure? Monitoring and evaluation (M&E) are central to Making Cents’ work and the long-term sustainability and responsiveness of our KM initiatives. We measure the results of our KM platform using the Ripple Model,¹ a four-level framework,² to evaluate each activity.
Level 1: Knowledge Exchange. Like a stone thrown into a pond, the delivery of a knowledge-sharing activity or tool represents the first level of impact. This level documents information about the delivery of a given event and/or the functionality of a tool or resource.
Level 2: Knowledge Capital. A multiplier effect activates a second ripple, increasing the impact and reach of a KM activity or tool through tangible and intangible results (such as documents, skills inspiration, relationships, and procedures). This level records and registers impacts, noting what was produced as a result of participants’ interaction with a given activity or tool/resource. This includes concrete products such as follow-on documents written, web pages viewed, or sessions recorded by participants, as well as results that are harder to quantify such as knowledge gained or empowerment achieved.
Level 3: Changed Practices. As individuals and organizations learn from one another through KM activities, they begin to refine the ways in which they carry out their youth empowerment activities. Level 3 captures these changes – ideally improvements – in participants’ skills and abilities that were acquired as a result of the learning that took place through knowledge-sharing actions.
Level 4: Performance Improvement. Finally, performance of the organization visibly improves and allows it to achieve more as a result of changes implemented by organizational staff, youth leaders and other stakeholders. In the language of our KM platform, this means an increase in the impact, scalability, or sustainability of participant organizations’ programs, policies, or partnerships has occurred.
How Do We Measure Progress? To monitor the first two levels, Knowledge Exchange and Knowledge Capital, Making Cents collects quantitative and qualitative data through surveys and interviews. To assess the higher levels of impact, Changed Practices and Performance Improvement, Making Cents collects both data and anecdotal evidence and uses the information collected earlier in the process to interpret results.
The Ripple Model takes into account a difficulty inherent in KM evaluation: proving “attribution,” the cause-and-effect relationship between a knowledge-sharing activity and its resulting impacts.
Making Cents has been evaluating the effectiveness of its KM platform since the inaugural Global Youth Economic Opportunities Summit in 2007. Over the years, we have seen how greater knowledge exchange and increased knowledge capital result in changed practices and improved performance within the field. This process requires continued commitment on the part of organizations and individuals, as well as adequate time to advance from the initial sharing of knowledge to the demonstrated increase in program impact.
How does having an M&E framework improve performance of the KM platform? This high level of attunement to facilitating the exchange of demand-driven, transferable information has led Making Cents to commit itself to innovation, always be responsive to stakeholder demands, and continuously adopt new and promising technologies.
Investing the time and effort in measuring the effectiveness of our KM initiatives has proven worthwhile. It offers concrete data about the outcomes of each activity, which informs how Making Cents plans future events. Each year we see an increase in the numbers of people who contribute to, and consume information from, our platform. Most importantly, Making Cents - and the stakeholders of our KM activities - see how our investments of time and money in KM lead to more evidence-based, scalable, and sustainable programming for youth around the world.
For more information about the M&E framework, you are invited to read “Making Cents International’s Knowledge Management Platform for Increasing Scale & Sustainability of Youth Economic Opportunity Programs: 2014 Results”.
¹ Joitske Hulesbosch, Mark Turbin, and Sibrenne Wagenaar. Monitoring and Evaluation Knowledge Management Strategies, IKM Background Papers.
² Tara M. Sullivan, Molly Strachan, Barbara K.timmons. Guide to Monitoring and Evaluating Health Information Products and Services, USAID.