In the first months of 2016 we made significant progress in analysing M&E as well survey data and consequently estimating the contribution of CBI and PUM. In addition, we presented our work to various interested parties and conducted a second visit to one of the case study countries (Bolivia).
Two-pagers were written that give an overview of the evidence around the intervention logic of CBI and PUM. The two-pagers – so called contribution stories – were written based on the data available from the 2015 monitoring and evaluation, the online surveys and the literature review. PUM used these stories in its 2015 Annual Report. The analysis of the first survey round showed that both organisations contributed to improved business practices but that the overall contribution was modest. The contributions stories can be found on www.primepartnership.nl.
Between February 26th and March 12th, a mission to Bolivia was implemented for the case study research. This was the second of three visits to Bolivia. Giel Ton and a local researcher interviewed several supported SMEs and key stakeholders involved in the CBI Tourism programme as well as several local representatives of PUM. Based on the findings from this field research a draft case study report is being written in a format that will be applied to the reporting of all six case studies.
PRIME continued to improve and test methods for analysis of M&E data. A measure of PUM and CBI “effectiveness” was created based on the knowledge and practice questions from the online survey. This measure is a function of firms’ perceptions of (i) changes in outcomes and (ii) programme contributions towards these changes. The team also checked the validity of the online survey data by comparing data collected from experts with data collected among firms for self-perceived knowledge and practices. Findings show that experts and firms generally give a similar response: 30% of the replies are exactly the same and around 40% have only minor differences. Both validity checks support the cohort methodology based on M&E and online survey data.
We will further investigate the programme effectiveness and impact pathways by using additional data, which will be collected in 2016 and 2017. In the first quarter of 2016, two online surveys were launched among CBI experts and supported SMEs. The data from these surveys will be used to complement CBI’s M&E data for the yearly monitoring reports.
As part of the PRIME-ITC partnership, 500 surveys were collected in May 2016 from IT/ITES companies in Bangladesh. The sample includes 39 NTF III beneficiaries and two PUM beneficiaries. The survey included questions about knowledge, practices, ultimate outcomes, employment characteristics and subjective expectations of management training. The team expects to receive the final baseline dataset in June 2016.
Linked to the PRIME-ITC survey in Bangladesh, the research team collected data to better understand the decision making process of SME directors and managers on staff training. The results from this study will be combined with results from a lab experiment conducted at Erasmus University to provide a more robust understanding of the mechanisms involved in this so-called “self-other” decision-making. This study will inform policy makers on how to get SME directors and managers to best involve employees in PSD programmes.
On February 8th 2016, the PRIME Programme Board approved the budget for 2016-2017 and the succession of Max Timmermans as representative of CBI instead of Dick de Man. Thijs van Praag indicated that he will retire and be substituted by the newly appointed PUM director Johan van de Gronden, former director at WWF-Netherlands.
PRIME stresses the need to aim for “net effects” where the sphere of influence is large and attribution claims can be made (e.g. on knowledge, practices, immediate outcome). This was one of the topics of a meeting on April 11th at which initial results were presented to stakeholders from CBI, PUM and the Ministry of Foreign Affairs. The meeting again illustrated the sometimes contradictory needs of policy makers and implementers in the search for good impact indicators of PSD instruments.
PRIME was asked to contribute to a roundtable organised by WTO-DFID in Geneva on May 23rd. The roundtable brought together practitioners from development partners to share experiences and discuss new approaches to maximizing the poverty impact of trade-related projects and programmes. Key questions were: do strategies draw links effectively between trade integration and poverty reduction? How to monitor the impact of trade-related projects on poverty reduction? What data exists and how can it be collected more easily? The character of the PRIME consortium (implementing agencies as well as scientific researchers) has given us a very sound and solid basis to contribute to this roundtable discussion.
The PRIME team welcomes Liesbeth Hofs. Liesbeth is the new programme manager Monitoring and Evaluation at CBI. Having worked at programmes such as the Youth Empowerment Alliance and the Connect4Change Consortium, her rich background will certainly add to the PRIME partnership.
The external evaluation of PUM by Erasmus University and Carnegie made extensive use of the data collected by PRIME. They used an adapted measure of effectiveness of the support, similar to the one developed in PRIME, and show positive effects on knowledge and practices. The PUM evaluators conclude that PRIME is a useful approach for impact assessment and expect that the successive annual cohorts and time-series will permit the assessment of quantitative effects on employment, turnover and employment.