Monitoring & Evaluation

Commodity Programmes

Solidaridad’s goal to become a data-driven organization requires solid monitoring and evaluation procedures as well as reliable and valid data. In order to support our journey, we have identified four pillars for our planning, monitoring, evaluation and learning work: Systems for Action, Evidence for Change, Capacities for Measure, and Communication for Resources.

Four pillars to optimize data use

The first of our pillars is Systems for Action. Here, we define a Theory of Change for every project with clear assumptions and measurement plans, define a data model based on the monitoring protocol, collect data at field level (atomic and aggregated data), analyze and visualize data. 

Evidence for Change is our pillar for defining our monitoring and evaluation methods. We collect data through ICT and rigorous methodologies, and organize internal and external evaluations. We also develop dashboards that can be utilized to both efficiently track progress for ongoing monitoring, and to provide content for communications. 

Capacities for Measure refers to building capacity of project and programme teams, facilitating and documenting learning sessions, and institutionalizing learning.

Last but not least, the Communication for Resources pillar ensures we provide evidence-based communication.

Piloting our new data collection model

In 2019, we developed a standardized data collection model around core prioritized indicators in order to improve the quality and consistency of our data. During the pilot, we collected granular data in three regions: India in Asia, Mexico in Central America, and South Africa, Zambia, and Mozambique in Southern Africa. 

The results showed that simple surveys make it possible for us to collect similar data across all regions and to aggregate and compare it. This puts Solidaridad in a good starting position for the next strategic period (2021-2025). 

Collecting farm data in Mexico for the pilot of our standardized data collection model

Tracking progress beneficiary by beneficiary

Standardized surveys will be deployed across all regions in order to track the progress of each individual farmer, miner or worker, and compare and aggregate data so that we can use it for adaptive management and impact monitoring purposes. Most importantly, it can be shared with the farmer, miner or worker so they can monitor their own progress.

Using technology to collect, monitor and analyze data is becoming the norm and the implementation of open-source data collection tools is growing rapidly. Keeping pace with this trend, we progressed from conventional data collection on paper to digital methods in 2019, using mobile phones with integrated open-source tools such as ODK, Taro and KoboCollect. 

Implementing the latest in monitoring tools

The latest technologies were also integrated into our monitoring work, such as remote sensing technologies (soil scanning), IoT weather stations, and satellite tools to monitor land cover change. This has improved farmers’ insights into their production.

While monitoring and gathering data is key for adjusting interventions during programme implementation, accountability towards our donors and partners is also of great importance. Collecting evidence that demonstrates impact on the target groups with whom we work (farmers, miners, workers, women and youth) allows our partners and donors to support result measurement. It also gives Solidaridad solid proof of concept in order to scale up. 

Gathering data in India

External evaluations 

External programme evaluations are a key source of evidence and an important input towards future programming. In order to assure the same principles are applied during all evaluations conducted across Solidaridad, we work with the OECD's Development Assistance Committee’s principles for the evaluation of development assistance. The principles are: 

  • relevance
  • efficiency
  • effectiveness
  • impact 
  • sustainability.

Internal evaluations

Several internal evaluations were conducted across the regions in 2019. In Kenya, we used the Cool Farm Tool to carry out a Coffee Sector Emissions Estimates and Climate Change Vulnerability Assessment. This showed that the vulnerability of the coffee sector to the impact of climate change varies per region. The vulnerability depends on the nature and intensity of climate hazards, and the ability of the local systems to adapt to or mitigate related impacts.

An internal evaluation in South Africa surveyed 416 farmers from four countries (Malawi, Mozambique, South Africa and Zambia). We wanted to assess which of 11 good practices were being embraced by farmers: crop rotation, water management, fertilizer application, rotational grazing, intercropping, seed selection, use of pesticides, soil testing, weeding, local gap, and mulching. 

The results of the survey showed that on average, 21% of the farmers adopted at least seven of the good practices. Adoption is a gradual process that peaks towards the end of projects. Given that most projects in Southern Africa started in 2018, the 21% average is therefore a fair rate. The highest adoption level was in Soy, at an average of 36%. Farmers practising intercropping as part of our Soy project increased from 48% at the project start to 58% after two years. The use of pesticides grew from 19% to 24%. 

Measuring outcomes in Ghana

We conducted a study in Ghana to document the benefits of mainstreaming gender in the Next Generation Cocoa Youth Programme (MASO). Adopting a mixed-method approach, data was captured from 2,757 respondents across all implementing districts. It revealed that most of the young women benefited from knowledge and skills in good agricultural practices, financial literacy, entrepreneurship, and sexual and reproductive health. 

Unemployment rates among the young women decreased from 28% to 13% and average annual income increased by 250 dollars. The survey indicated that young women can contribute to developing sustainable supply chains given the right knowledge and skill set. The study recommended that Solidaridad integrate women's considerations into its Youth Savings and Loan Association (YSLA) approach in order to enhance their savings culture. 

Abigail Oblie, who opened her own business with support from our MASO youth for cocoa programme in Ghana