Quick, representative and often surprising

We recently interviewed Indonesian smallholders about our ‘Pancer Tani’ service hubs. There are numerous interesting aspects to the methods and results. We asked three closely involved people for their views.    

Pranav Sridhar, Senior Associate, 60 decibels (60dB).  
How would you sum up the findings in one sentence?
The ‘Pancer Tanis’ are providing Indonesian smallholders with high-quality products and services. The relationship is hassle-free and helps farmers to increase their production, income, and resilience to climate shocks.

For 60dB, what was special about this study?
It’s one of our first agriculture studies in Indonesia and our second with the Syngenta Foundation. Hearing farmers’ needs, experiences, and suggestions for improvement have given us a deeper understanding of Indonesia’s agricultural sector. Following our study in Bangladesh, we now also see the Syngenta Foundation’s work in a broader context. 

What do you know now that you wish you’d known before?
We often find that a study produces both valuable insights and new questions!  In this case, we’d been expecting farmers to have used just one Pancer Tani service. On listening to them, we realized that many were accessing two or three. This limited our ability to identify differences between farmers based on the type of service they were using. Next time around, we could ask a larger sample of farmers to list the services they use. That would enable more in-depth analysis and help identify differences in experience and impact based on the services used. 

60dB collects ‘Lean Data’. You say that ensures ‘speed, repeatability, and comparability’. How far do Lean Data allow one to draw conclusions?
Our approach is useful in several contexts but does have certain limitations. In statisticians’ jargon: If we randomly select about 275 people for 5000+ beneficiaries, the interview results will have a 90% confidence level and 5% margin of error. Put more simply: We’re pretty sure our findings reflect the experience of the 5000. Of course, our work is not immune to the limitations of social science research*. But it is designed to support decision-making based on beneficiaries’ own views while bearing in mind the time and budget constraints of non-profits and social enterprises. We recognize the value of large rigorous randomized controlled trials but aim for feedback that our partners can much more rapidly turn into action and impact. 

Looking at the study results in Bangladesh and Indonesia, and from your experience of working with us, what would be your main advice to the Syngenta Foundation?
We’ve been delighted to see your emphasis on listening to beneficiaries and using that information to improve how you serve them. The report lays out our recommendations. However, we recognize that customer feedback, while critical, is just one of the considerations that shape your approach. We believe that the insights in our reports, based on responses from ‘your’ farmers, can enhance your work in both Bangladesh and Indonesia. 

Teddy Tambu, Chairman of YASI, Indonesia
What surprised you about the results?
We appreciate all the findings and are examining how to turn them into improvements. I was surprised to hear farmers mention poor seedling quality. After all, they can bring their own seed to a Pancer Tani and get it nurtured into seedlings. We must look into possible quality issues with the original seed. For example, we may need to give smallholders more advice on storage. 

How might the findings change your team’s work? 
We want to make Pancer Tani as good as we can. The study findings are an extra motivation here. We’re also pleased that our program got a good Net Promoter Score (see page 25 of the summary). That can help us attract new partners and possibly funders.  

32% of farmers say Pancer Tanis has “very much improved” their quality of life. When Indonesian smallholders earn more money, what are the first changes one sees?
They invest in their children’s education and in new assets to support their business and step up as entrepreneurs.

22% of households were badly hit by “shocks”. How can Pancer Tanis help improve their situation?
The biggest such shocks come from pests, disease, and uncertain weather. Tackling these involves training, for example on soil health or seed handling. It also requires offers that reduce weather problems. Instead of planting a seed and not knowing if the rain will be right, farmers can use seedlings from their Pancer Tani. They will soon also be able to get weather insurance there. 

How do you interpret the fairly high disability rate among Pancer Tani customers? What does it mean for your work?
Supporting diversity and inclusion is a very important part of the Foundation’s work. We’re committed to helping smallholders regardless of their gender, ethnicity, religion, or disabilities. As well as customers, there is also a disabled person running a Pancer Tani seedling nursery. That is an excellent entrepreneurial environment for somebody who would face barriers in many other jobs. I expect more people will follow his example in the future. 

Yuan Zhou, Head of Agricultural Policy, Syngenta Foundation
What do you find surprising about the study results?
I’m slightly surprised that 61% of farmers had only used Pancer Tani services once. I expected there would already be more repeat customers. I also thought there would be more than 13% female interviewees. But as we recently presented, women were closely involved as interviewers!  

How do the findings compare with those from Bangladesh?
There are similarities and differences. For example, The 'Net Promoter Score', a gauge of satisfaction, was good in both countries. Smallholders appreciate access to good products and services. With respect to impact, however, more Bangladeshi farmers said their quality of life had improved than their Indonesian colleagues. That may be because Farmers’ Hubs have been operating for longer in Bangladesh.

How much of the study information could help the Foundation’s work on Farmers’ Hubs in Africa? 
One has to be a bit careful because a lot of the insights and recommendations are location- and program-specific. But there are clearly common threads to success. These include filling a gap in addressing farmers’ needs and providing consistently good products and services. That is definitely applicable to African Farmers’ Hub as well.

How have these two studies changed your opinion about Lean Data?
They’ve confirmed my view, rather than changed it. A Lean Data approach provides a fast and effective way to collect farmer perceptions. That’s a great advantage, given the speed of changes happening on the ground. Lean Data should be used more widely for relatively straightforward situations.

What plans does the Foundation have for the next in-depth program research – of this type or others? 
We are working on the impact evaluation plans in connection with the rollout of our new strategy. In the future, I see us deploying several different methods and tools. We’ll probably use Lean Data, online surveys, and field staff surveys, for example. The choice will depend on the specific situation. 

Read the Pancer Tanis’ Impact Insights Indonesia Report