Interest in artificial intelligence (AI) has been growing among the monitoring, evaluation, research and learning (MERL) community over the past decade. Yet AI adoption has been hampered by lack of data science capacities, low trust in AI, ethical concerns, affordability, absence of applicable use cases, and multiple other barriers. The public release of mainstream Generative AI (GenAI) applications, beginning with ChatGPT in late 2022, changed the game. GenAI and natural language processing (NLP) tools are now more accessible. For MERL Professionals, the hope is that they might support program delivery, improve efficiencies, and help us to find meaning in large troves of qualitative data.

At the same time, the untested nature of emerging AI — especially in low resource contexts and fragile environments. The potential harms due to bias, low levels of reliability and validity, lack of accountability, and negative wider societal impacts such as job loss and the environmental costs of data processing make the use of GenAI an ethically fraught exercise. 

The MERL community is varied in its attitudes toward GenAI. Some are rushing to embrace and adopt it, at times uncritically. Others are avoiding GenAI altogether, perhaps out of fear or disinterest, but also because of real concerns about the practical utility and the bigger issues with AI ethics. 

At this talk Linda Raftree elaborates, with real world examples, on how and where GenAI is being used in the MERL space. She also examines how and why the wider AI boom is affecting our societies and our planet to spur us to think critically about AI in our work and in the world and to re-examine the role of Evaluators as users and assessors of AI. 

Links: 

• The MERL Tech Initiative: https://merltech.org/
• Linda's presentation slides

Music: Urbana-Metronica (wooh-yeah mix) (ft. Morusque, Jeris, CSoul, Alex Beroza) by spinningmerkaba http://ccmixter.org/people/jlbrock44; SouljaUnit Remix of DeadRobot Music’s Surfy via https://freesound.org/people/SouljaUnit/sounds/640175/
Photo: Martin Jordan

Thanks to our partners and co-organisers that helped realising the REvaluation Conference 2024: Ministry of Innovation, Mobility and Infrastructure, Ministry of Women, Science and Research, Ministry of Economy, Energy and Tourism, Vienna Science and Technology Fund, COST Action PROFEEDBACK (CA ), Meeting Destination Vienna, FH Campus Vienna, Austrian Science Fund, Austrian Research Promotion Agency, ZSI - Centre for Social Innovation, Fraunhofer ISI, Institut Francilien Recherche, Innovation et Société, Joanneum Research and the whole fteval Community. 

podcast(at)fteval.at - twitter.com/fteval - www.fteval.at