How do we capture what cannot be measured?

By Arundhita Bhanjdeo

My first trip to Bhubhui, Jharkhand, India was three years ago. The first thing I came across was one of the women self-help groups sitting in a circle for their weekly meeting. Some farmers were sharing their experiences of removing weeds from rice paddies, discussing how with a weeding machine they found it so easy compared to the arduous process of doing it by hand. I was working on an agricultural research project funded by ACIAR in collaboration with an NGO in the Eastern India Plateau. It was apparently an unusual sight for someone to enter a village of small-scale farmers in those parts and find a meeting such as this. I later realised it was because of the persistent efforts of the NGO working with women, organising them into collectives and empowering them to drive the change they need.

The project that I was involved in was helping farmers improve their livelihood through implementing innovative cropping systems. It was a collaborative research project with researchers from diverse disciplines such as horticulture, water, soils, and social science, as well as development workers from the field. With women at its core, the project focused on working with farmers to develop crop system research that solved problems identified locally by the farmers. This research was carried out on-farm, by the farmers, with the support of the project team. The farmers “learnt by doing”. The project measured its impacts by assessing food security, farm productivity/income, the number of out-scaled farms/villages and the status of living of the farmers. The project was deemed successful and was applauded by neighbouring villages, district administration, the reviewers, and the funders. Needless to say, this project positively affected the agency of women, but this outcome was subtle and difficult to capture.

Economist, philosopher and author of “Development as Freedom”, Amartya Sen considers empowerment, agency, human capability, and learning important indicators of development. I will not delve into what is development and what are its indicators. The question I wish to delve into through my PhD is, how does one evaluate outcomes from these research for development projects?

• Do we only map the outputs that are structured and described in project documents as goals?
• Or should we also capture outputs that were not? Something that we didn’t anticipate resulting from the project?

Some of these unexpected outcomes may include the contribution to, or creation of knowledge, or learning and transformation of farmers. For instance, the indigenous women in the picture below, to be able to sit confidently in a village meeting with other men and women and plan crops is a transformation. It was a result of being actively involved in project activities and the process of learning they went through. This changed other farmer’s perception towards them (both men and women), the team member’s perception towards them, and most importantly their perception towards themselves. For these women farmers, increase in income and food security does matter, but this change in perception or the learnings received are something that they value equally, if not more so. This is something that will stay with them when they are participating in other agricultural projects and in other aspects of their life, holistically.

 

Picture 1: Babli and Kalpana from Talaburu (village) are seen pointing to the farms in which different crops are grown on a locally made map. Picture credits: Ashok Kumar

Another story; because of increased production on-farm, a woman farmer from a different village now has some savings at her disposal. The farmer is now able to send her children to school, spend money on her children according to whim, or raise her voice for something she believed in or against, something which she thought was wrong in her family. Many other stories like these exist and indicate a change in the perception of people. These changes are intangible but permanent.

Change does not have to be one-way. As part of the same project, I observed the changing perceptions of the stakeholders involved in the project as well. Not limited to farmers, a change occurred in the partners, scientists, development workers and various support staff in the field. Being a collaborative project with people from multiple disciplines, each team member had something to learn from the others. An example; I come from a social science background, philosophy to be precise, and after this project I knew what aquifers were, the importance of groundwater management, and how could we graft two different vegetable stems to avoid certain diseases. The project changed my perception of looking at the natural resources and the farmer communities. It also enhanced my confidence in the farmers and myself. So, the next time I work with a similar community, I will work as a different person, with the learnings from the previous project embedded in me. Every individual has their respective capacity and drivers for learning, but learning is given. In my PhD, I intend to study the learning level and transformational changes of different participants in an agriculture for development project. It will emphasise learning and mapping transformational change as we believe that in the larger development context, in addition to attaining the quantitative indicators for sustainable or millennium development goals, the knowledge, learning, agency or transformation that is achieved in the whole process is also associated with growth, development and poverty reduction.

As romantic as it sounds to me (perhaps not to others), mapping and analysing subtle changes in human behaviour and personality can be difficult.  I have now come back from my first field study after spending two months in three villages of Eastern India collecting about 50 vignettes and narratives from the different stakeholders involved in the project. It was an enriching experience for me to talk to so many people and listen to their stories of learning, or stories of change as I like to call it. The fun yet onerous task for me now is to transcribe all the recorded data and start analysing them. Figuring out how to put them into a framework and how to extrapolate data into change stories will be done in the coming months. I look forward to sharing my ideas, receiving useful insights or feedback and having a candid and purposeful discussion with anyone interested in the topic.

Picture 2: Arundhita(right) trying to get some ‘transformational’ stories out of Babli by cracking silly jokes. Picture credits: Siddharth Patil