Leave No One Behind
AIRFE App is an innovative tool to ensure inclusive evaluation in times where adaptation is needed
⛶ Full screen
Who We Are:"Leave No One Behind"
Our team includes;
Ariella Carolino, Freelancer Monitoring and Evaluation Consultant (Brazil),
Claudia Olavarría, Sociologist, University of Chile (Chile),
Sylvie Margat, Evaluator, French Agency for Development (France) and
Sharon Ndandula, Results Measurement Consultant, International Finance Corporation (Zambia).
What We Are Doing
In the context of the COVID-19 pandemic and other crises, data collection within remote communities and with hard-to-reach end beneficiaries faces challenges due to travel restrictions for international consultants, limited availability of national evaluators due to growing demand, and shifting priorities of field office staff and counterparts due to the COVID-19 response. The limited access to field sites can lead to a heavy reliance on remote data collection methods, which, in combination with convenience sampling, entails intrinsic potential biases against under-represented groups in the selection of respondents.
Our team is working on developing an inclusive- oriented tool, that is adaptable for times of crises, that evaluators can use to get in touch with the most vulnerable and hard-to-reach persons that evaluators can use in order to solve the problem of adequate data collection (and collaborative evaluation management) during moments of crisis, like COVID-19.
Daily Progress and Outcomes
*The team has been meeting daily to brainstorm and has come up with some interesting ideas. We are collaborating via Google Docs, too, to share our ideas and comments.
*The group worked well as a team once it started the brainstorming process through video calls, bringing common ideas/problems during the meetings and developing further from there. There was no need to vote on a better idea because the solution was gradually developed together.
*Due to the hackathon format of delivering a 2-3 minute pitch video, the group decided that an innovative tool would be the best idea to follow, instead of a checklist of standards and procedures to ensure equity and comprehensive representation in data collection in moments where adaptation is needed. That research effort, although extremely necessary, is more easily found across the globe, in diverse evaluation think tanks/institutions. In that case, the innovation itself would be more of a systematization type, which is an ongoing work already, and would demand an immense time that is unavailable during a hackathon.
*The team conducted some interviews with some stakeholders working in crisis areas to find out what tools they are using but also across the data collection community of evaluators. The interviews helped the team understand better the tools for data collection used and how these could be improved.
*The innovative tool idea, on the other hand, would be greatly useful from a hands-on perspective. In face of an immediate crisis (whether it be a climate change-related disaster, a pandemic such as Covid-19, or the outbreak of war), evaluators would benefit from a tool both accessible and easy to tailor to their own needs.
*There was a consensus around the scope of the challenge and the multiple issues/concerns involved, which also led the team to keep a broad frame while thinking about the "ideal persona". Like the team name itself, our goal is to “leave no one behind” during evaluations undertaken in times of crises. This meant trying to develop a tool flexible enough to adapt and to use in various types of scenario and data collection necessities.
Main issues that our challenge should tackle and address
2. Digital inaccessibility (no access to the internet, Wi-Fi or equipment, such as computers, tablets and smartphones)
3. High Vulnerability (marginalization, domestic violence, poverty, drug disorders)
4. Intense spatial mobility during work hours (street vendors, for example)
5. Homelessness or frequent housing mobility
6. Hard-to-reach communities, such as indigenous people in the Amazon region
7. Refugees and recent immigrants without proper documentation=> The issues, above, also describe some of our stakeholder’s persona most challenging social and economic characteristics. They are at the core of challenge 3’s concern, which is how to ensure that evaluation being undertaken in times of crisis (such as Covid-19) can be successfully inclusive and leave no one behind.
The Adaptive and Inclusive Remote Field Evaluation App
Prototyping and Testing (requirements to enhance app’s usability)
- Engagement of grassroot community organizations (GRO);
- Identification and engagement of focal points/facilitators in communities;
- Training GRO and community focal points to contribute with data collection in the app;
- A methodological design that incorporates a set of complementary tools;
- An “evaluation design” feature in the app, making it possible to tailor the process according to program/project, territory, target-population, evaluation goals and limitations (in terms of language, culture, digital inaccessibility, illiteracy, physical disabilities etc);
- A representative (intentioned) sample design;
- Budget to distribute devices to use the app, in case it is needed;
- Provide internet access for grassroot community organization, if needed and feasible.
After a lot of discussions and iterations, we have our prototype!
WE DID IT! Our video has finally been posted. After many hours of discussions, google doc editions and takes, we finally got it right on our 4th take :) This has been a great opportunity for collaboration. We have all learnt so much and built a great network! Thankyou IPDET for the opportunity to participate in the Evaluation Hackathon!