Monitoring & Evaluation

Commodity Programmes

Solidaridad’s goal to become a data-driven organization requires solid monitoring and evaluation procedures as well as reliable and valid data. In order to support this journey, we have identified four pillars for our planning, monitoring, evaluation and learning (PMEL) work: Systems for Action, Evidence for Change, Capacities for Measure, and Communication for Resources.

Under Systems for actions, we define a Theory of Change for every project with clear assumptions and measurement plans, define a data model based on the monitoring protocol, collect data at field level (atomic and aggregated data), analyze data, and visualize data. Under Evidence for change we define our monitoring and evaluation methods, collect data collection through ICT and rigorous methodologies, orgainze internal and external  evaluations (internal and external), and develop dashboards that can be utilized to both efficiently track progress for ongoing monitoring, and to provide content for communications. With Capacities for measure we refer to building capacity of project/ programme teams, facilitating and documenting learning sessions, and institutionalizing learning. With Communication for resources we focus on communicating with evidence.

Solidaridad has institutionalized the development of theories of change for all projects, and during 2018 we placed greater emphasis on improving the quality and consistency of data. We continued to embed and refine the network-wide monitoring protocol with the aim of aligning indicators and definitions. At the same time, we laid the foundations for a solid data model to identify key data points across projects, which will allow us to collect atomic data from the field. Being able to track progress on each individual farmer, miner or worker, rather than at aggregated level, will give Solidaridad valuable data which can be used for adaptive management and impact monitoring. But, most importantly, it can be given back to the farmer, miner or worker to monitor their own progress.

The use of technology for collecting, monitoring and analysing data is becoming the norm and the use of open-source data collection tools is rapidly increasingly. Keeping pace with this trend, during 2018 we progressed from conventional paper-data collection to digital methods using mobile phones with integrated open-source tools such as ODK, Taro and KoboCollect. The latest technologies have also been integrated into our monitoring, such as remote sensing technologies (soil scanning), IoT weather stations, and satellite tools to monitor land cover change. This has improved farmers’ insights on their production.

While monitoring and gathering data is key for adjusting interventions during programme implementation, accountability towards our donors and partners is of great importance. Collecting evidence that demonstrates impact for the target groups with whom we work (farmers, miners, workers, women and youth) allows our partners and donors to support result measurement while giving Solidaridad solid proof of concept to scale up. External programme evaluations are an important source for collecting evidence and an important input towards future programming. In order to assure the same principles are applied among all evaluations conducted across Solidaridad, we work with the OEC’s Development Assistance Committee’s (DAC) principles for the evaluation of development assistance. Namely: relevance, efficiency, effectiveness, impact and sustainability.

Below we present the findings of some of the most relevant evaluations conducted over 2018.

Mid-term review of the Advocacy for Change and Practice for Change programmes

The mid-term review looked at Solidaridad’s Practice for Change (PfC) and Advocacy for Change (AfC) programmes following the OECD DAC in terms of relevance, effectiveness, efficiency and sustainability.

Relevance

Findings show that both beneficiaries and stakeholders rate Solidaridad’s work as relevant to their needs, and that Solidaridad has the mechanisms for scaling in place. This includes public private and multi-stakeholder partnerships, using digital solutions and enabling impact investment. Our priority for the coming years is to further embed, test, and prove these scaling mechanisms.

Effectiveness

Particular progress has been made in terms of the performance indicators under the areas of good practices, sustainable landscape innovation and enabling policy environments. Targets for robust infrastructure were not entirely achieved by the end of 2017, and the targets for 2017 were possibly too ambitious. We are however confident that we will be able to catch up during the remaining programme period.

The review concluded that Solidaridad has been effective in turning capacity changes into behavioural changes. It found that Solidaridad is perceived as: “a trusted partner that is technically strong on the commodities included in PfC and AfC, and that builds on previous work by all parties involved in change: business, workers, and government”. The report confirms Solidaridad’s central assumption that connecting work on the ground with dialogue and advocacy leads to better results than focusing exclusively on either one.

Efficiency

The report emphasizes that the inception phase, in combination with the shift to a new organizational model and the novelty of the approaches, caused delays in implementation and budget utilization in 2016-2017. This applies particularly the landscape innovation approach, climate innovation and impact investment, as well as the role of lobby and advocacy in the context of Solidaridad. We are, however, confident that we will be able to act on those areas which require improvement.

Sustainability

The report concludes that Solidaridad has been successful in promoting ownership by supporting and engaging a large variety of stakeholders. It also concludes that ownership of activities and outcomes is recognised as an important element in creating sustainability.

The participatory approach adopted in this mid-term review has provided a valuable opportunity for reflection, and helped identify opportunities to improve our project implementation and management.
 

SUSTAINABLE AGRICULTURE, FOOD SECURITY AND LINKAGES

The endline study for Phase 1 of the sustainable agriculture, food security and linkages programme (SaFaL) revealed important improvements in productivity, food security and gender inclusivity.

The productivity of SaFaL dairy farmers (2.5 litres/cow/day) increased by 90% in comparison with that of non-SaFaL farmers (1.34 litres/cow/day). In the aquaculture sector, SaFaL households produced more fish (across all fish types) per hectare (by weight) than their counterparts in the comparison group. The difference was significant in the case of shrimp where SaFaL households produced nearly 350 kg/hectare as compared to 262 kg/hectare of non-SaFaL households. In the horticulture sector, 2018 figures showed a 45% increase on 2016-17, when SaFaL farmers received BDT 570286.4 per hectare of area cultivated.

In terms of women’s participation in decision making, we found evidence that a higher proportion of women from SaFaL households were taking part in a variety of key business decisions than non-SaFaL households. This includes decisions like which crop (or species of fish or breed of cow) to cultivate, new business ventures, managing finances, and investing and taking loans.

MASO next generation youth programme learning survey

The MASO next generation cocoa youth programme in West Africa assessed how it has influenced the perceptions and attitudes of youth in agriculture. The study sampled and interviewed 2,939 youth, adult cocoa farmers and service providers. Almost all programme beneficiary youth (98%) said they had gained new knowledge and skills in good cocoa farming practices with over 91% admitting to effectiveness of programme training and implementation strategy.

The results also show that over half of young people (59%) in MASO communities now perceive farming as a business that has prospects of transforming their lives rather than travelling to the cities. Another interesting result is more women now appreciate farming as an opportunity for both sexes, and not just job for men as originally perceived in cocoa growing communities.

Substantial gains in sugarcane yield

The assessment of the sugarcane programme in India indicated a substantial increase in yield. It adopted a crop-cutting methodology that includes actual harvest of the crop in specific dimensions of the plot. In comparison with the 2016 baseline, the study revealed that overall there is a 48% improvement in yield of the plant crop and 16% of the ratoon crop. 2018 data also indicated a yield of 789 Qt/hectare for plant crops against 531 qt/hectare and 628 qt/hectare for ratoon crop as against 542 qt/hectare. This was largely due to a change in the variety, adoption of integrated pest management practices, trash mulching and irrigation.

Employee and community satisfaction assessment in mines

An assessment with mine employees and workers was conducted for our programme to improve working conditions in industrial mines. The study revealed that 95% of workers found the programme beneficial as it has helped them to improve safety and working conditions. All workers reported higher level of autonomy in forming or joining associations. The community assessment study indicated that 95% of the villages surrounding the mines were consulted with regard to environmental protection. Sixty percent of the villages reported improvements in infrastructure the local area.