logo mobile

e-Agriculture mobile logo

What is the most effective way to measure the impact of ICT for development (ICT4D) initiatives?

27 posts / 0 new
Last post
shehzaad's picture
Offline
Last seen: 5 years 9 months ago
Joined: 19/03/2009 - 01:24
What is the most effective way to measure the impact of ICT for development (ICT4D) initiatives?

Dear All,

It seems that the forum is warming up very well with great contributions from Surabhi, Shahroz, Arafat, Bulbul and John so far. Now building on Jim's suggestion as above, let me pose the following questions to the forum for further thoughts and opinions for this week -

What is the most effective way to measure the impact of ICT for development (ICT4D) initiatives?

Please let us know what you think is the most effective way and why you think so.

shahid's picture
Offline
Last seen: 2 months 1 day ago
Joined: 06/11/2008 - 10:40
Impact measurement

 

Dear Shehzaad,

Good to see all these contributions and I just like to add couple of points from our working experience in e-Agriculture.

I think there is no short cut way to measure impact o ICT4D initiatives. It is always complex to measure the impact, specially the direct impact (in terms of income or productivity) of ICT usage in development.

There are couple of critical factors in this process, one is considering the social-cultural factors including behavioral pattern of the target groups while setting indicators for measurement of impact, and second is consideration of changes in the eco-system of development dynamics i.e. integration of ICT in the value chain.

What do others think?

Thanks,

Shahid 

 

shehzaad's picture
Offline
Last seen: 5 years 9 months ago
Joined: 19/03/2009 - 01:24
Asad's opinion on universal set of indicators

Dear Shahid,

With reference to Asad's opinion on having universal set of indicators, please share with the forum your thoughts - should we or should we not have a set of indicators which are uniform or treated as globally accepted across any development agency for any ICT4D project?

Or, starting with agriculture only, is it feasible to have a universal set of impact assessment for ICT impact on agriculture?

Thanks, the forum will wait to hear from you.

shahid's picture
Offline
Last seen: 2 months 1 day ago
Joined: 06/11/2008 - 10:40
Universal set of indicators for Impact Assessment

Dear All,

Thanks Asad for bringing this very interesting issue of ''Universal set of indicators'' for impact assessment.

Surely, there are scopes to set some common indicators which may be benficial for different stakeholders like development agencies as well as private sectors. In some cases, we already experienced this kind of common or universal indicators in many initiatives, specially in MDGs.

In ICT initiatives for Agricultural projects, the very basic indicators mostly includes productivity issue which is little complex to identify and measure the magnitude of contribution. Based on last few years limited global experience of e-Agriculture initiatives, I am sure there are few indicators common in different initiatives which can be reviewed and studied further.

Since integration of ICT in agricultural initiative is a new trend, we have to be more innovative and flexible to come up with a set of indicators, particularly to define some universal indicators. 

Any other comments on this from the SMEs ?

Regards,

Shahid

 

gelb's picture
Offline
Last seen: 5 years 5 months ago
Joined: 29/05/2007 - 15:09
Re Since integration of ICT in agricultural initiative is a new

 

Shahid suggests that "integration of ICT in agricultural initiatives is a new trend.." I would like to agree and supplement the comment by noting  that integration of ICT in agricultural initiatives goes back to 1956 - tracing to an inititive to integrate Linear Planning into agricultural management methodology. As I commented yesterday several imsightful indicators regarding the rate of ICT adoption, ICT impact on production and productity and innovation adoption in general are detailed in various areas of agricutural production in the free e-book "ICT in Agriculture: Perspectives of Technological Innovation"  http://departments.agri.huji.ac.il/economics/gelb-main.html                  

It is important to note that the book is NOT an historical review. It is geared to present overtime proven insights for ICT4D project decion makers engaged in evaluating their past activities and planning to undertake new projects.

As before comments are solicited and welcome.

Ehud Gelb gelb@agri.huji.ac.il

asad rahman's picture
Offline
Last seen: 4 years 10 months ago
Joined: 17/12/2010 - 05:38
  Hello   Considering the
 
Hello
 
Considering the present complex scenario of development all over the world the role of technology is becoming more and more prominent. Among all types of technologies the access of the mass ICT has been observed quite prominently. Once again information proved to be the key to many turning points of history starting from the revolutions in different countries like Yemen, Tunisia, Egypt, Lybia to Wikileaks.
 
The impact of ICT in development scenario is increasing exponentially. However the process of capturing impact ICT on development and it’s attribution is still at a very early stage. Considering the ICT4D practitioners and M4P practitioners the universal acceptance and recognition of different tools are needed to be gained. But before that the development of such universally accepted processes are needed, the tools and indicators that can be usable by the development practitioners all over the world for measuring impact.
I hope this forum would lead to that process and I would request participants to share their views regarding a universal set of indicators and tools for measuring ICT impact on development
process.
 
Thanks
 
Asad

 

Michael Riggs's picture
Offline
Last seen: 2 years 5 months ago
Joined: 15/09/2010 - 11:17
what is "M4P"?

Hi. Asad, or someone else, would you please tell us what M4P means? Just to be sure we all have a common understanding of our terminology. Thanks!

shehzaad's picture
Offline
Last seen: 5 years 9 months ago
Joined: 19/03/2009 - 01:24
@asad - Universal set of indicators

Dear Asad,

Interesting when you mention about 'universal set of indicators' - do you mean to say that there should be a certain list of indicators to capture ICT impact on development across the board? Something which should be applied across any project, any country and across any context?

I wonder if you are indicating towards any kind of compatibility or uniformity between indicators used by different agencies for tracking the same kind of sought after impacts.

Would appreciate if you could share your thoughts.

ecouper's picture
Offline
Last seen: 5 years 10 months ago
Joined: 03/04/2011 - 22:41
Defining impact

I see two different categories to consider when trying to measure "impact" of ICT4D initiatives.

I. Impact of Project

The first deals with the initiatives impact on the community, nation, market, etc. that it's trying to serve. In this way, I think you can begin by measuring ICT4D initiatives in the same way that you measure any development project. The same debates, considerations, and challenges apply. 

Within this impact analysis category, ICT4D initiatives often have an unique advantage. If smartly built, a ICT system will routinely capture a lot of important data that will make an impact analysis faster and cheaper.

II. Impact of technology

Secondly, it's important to consider the role that technology is playing in an initiative. ICT is a means, not an end, so it is crucial we ask ourselves if ICT is the correct tool for the job.  Could the same information, service, resource, etc. be delivered in a less ICTish way? Does the information, service, resource, etc. already exist without ICT?

A big part of this is the cost. Does introducing ICT lower the cost? But that isn't the only consideration. Perhaps ICTs speed up the process or make M&E easier or increase transparency. All of these benefits are tangible and should be measured/estimated before a new tool (in this case ICTs) are fully adopted.

shehzaad's picture
Offline
Last seen: 5 years 9 months ago
Joined: 19/03/2009 - 01:24
Focusing on agiculture

Dear Eric,

Great the way you categorise the impact assessment indicators. Do you have any thoughts on these being able to be specified in the field of agriculture only? What I am trying to ask is are indicators for tracking ICT4D impact essentially very different in agriculture or all ICT4D interventions across any sector should have a general common set of indicators? What do you think?

ecouper's picture
Offline
Last seen: 5 years 10 months ago
Joined: 03/04/2011 - 22:41
Re: Focusing on Agriculture

I'm very wary to suggest that all ICT4D projects or even all ICT4Ag projects can embrace a common set of indicators. Simply put, different initiatives have vastly different goals. To throw a blanket set of indicators over a project that connect farmers to markets and another that provides weather alerts risks a poor evaluation of both projects.

Even assuming that all projects are ultimately about improving the quality of life of farmers, good luck trying to demonstrate that your intiative did so.

That said, I do think there is important information that all development projects should publically release. Regardless of what your stated results, they should include a comprehensive look at how much money was spent and how. 

We assess projects for two reasons:

----So that donors and the public can understand better the results of our organizations' efforts.

----Other organizations can learn from our work and adopt similiar approaches if they are found to be sound.

Without financial information, there is no way that another organization can evaluate a projects fully. With finite resources, it is crucuial that we consider not only if a project works but how much it costs to make it work.

 

 

 

shahroz's picture
Offline
Last seen: 5 years 1 month ago
Joined: 10/03/2008 - 08:06
M4P.

Answering to Michael's query, M4P is an acronym for Making markets work for the poor. To elaborate it is an approach whereby market growth or development is facilitated but which leads to pro-poor growth.

Michael Riggs's picture
Offline
Last seen: 2 years 5 months ago
Joined: 15/09/2010 - 11:17
RE: [e-Agriculture] What is the most effective way to measure th
Shahroz, thank you for this clarification. ________________________________________
Nabanita's picture
Offline
Last seen: 5 years 10 months ago
Joined: 21/09/2011 - 13:03
Risk of getting tied to a wishlist of indicators

Dear Shehzad and Dear All,

Interesting to follow the forum. I see the reasoning behind defining some common indicators that can be aggregated, compared, benchmarked across different projects working in ICT4D initiatives; however don’t you think doing so might limit projects to only measure against a predetermined ‘wishlist’ of indicators? I think it is more important for ICT4D initiatives to first map out the logic of their work and then determine what indicators can be used to measure those different levels of changes. Indeed at a higher level, common indicators for market development project often include scale reached, additional jobs and additional income created but I think it’s very risky and pre-mature to have a standard set of indicators that fits all projects. Market development is also about thinking of innovative solutions to engage the private sector, so just as solutions are different, the indicators related to interventions can be different.

Just as an example, I recently did some work for a programme doing some ICT work in Cambodia. One of the projects is about setting a system to pay bills, transfer money using mobile phones. Now one of the many impacts of the project is that traders are using mobile phones to make instant payments to farmers, which saves them the hassle of often travelling long distances to get money. This time can be spent in doing alternate activities. However it would be very difficult to attribute indicators such as increased productivity, increased income as a result of this.

Would be interested in hearing others thoughts on this matter.

Nabanita.
 

asad rahman's picture
Offline
Last seen: 4 years 10 months ago
Joined: 17/12/2010 - 05:38
Re: Universal set of indicators for Impact Assessment

Dear Shehzad & Dear All

Well, when I said a universal set of indicator I meant something like Poverty index or consumer price index or human index. CAPRI (Common Agricultural Policy Regionalised Impact) can be a good example.

Common set of indicators does not mean that measuring the impact should be confined only by those. There are many different types of ICT4D project and other projects using ICT as a tool for access to information, access to market and for similar facilitation of opportunities for the poor. Of course the scope, timeframe and many other factors differ among those projects. But there should be some common set of indicator irrespective of other factors to measure the impacts. Apart from those factors there should also be other indicator to measure the specific intended or unintended impacts of the project based on its objectives. The idea is to create a standard measure against which some of the impacts of all ICT/ICT4D projects can be measured. But then again the merit of the project should not be considered only by those indicators. In this regard I completely agree with Nabanita that there might be a risk of limiting the projects by the common set of indicators. 

And finally indicators can be qualitative also, whenever the word indicator is mentioned generally a set of numbers are considered by most of us. But qualitative indicators are also very important to see the change in the whole system. To tell the complete story the use of the qualitative indicators are no less important than the quantitative ones.

Thank you everyone, have a good day

Asad
 

surabhi75's picture
Offline
Last seen: 4 years 5 months ago
Joined: 23/09/2011 - 07:02
ICT index

Isn't it worth discussing about these indicators and trying to list out both quantitative and qualitative ones to be able to start thinking in developing an ICT index for agriculture. This will build the base the future databasis and comparative studies across states or nations. At this forum we can think and try to do this.

Surabhi 

gelb's picture
Offline
Last seen: 5 years 5 months ago
Joined: 29/05/2007 - 15:09
THE IMPACT OF ICT4D

The comments till now seem to underestimate the comments and evaluation of ICT end users

gelb's picture
Offline
Last seen: 5 years 5 months ago
Joined: 29/05/2007 - 15:09
MEASURING THE IMPACT OF ICT4D

The comment till now seem to underestimate end user comments and evaluation of impact./

Michael Riggs's picture
Offline
Last seen: 2 years 5 months ago
Joined: 15/09/2010 - 11:17
end user comments and evaluation

Hi Ehud. Would you tell us a bit more on what you mean by end user comments, and evaluation? This could add a new dimension to the ongoing dicussion.

Thanks!

tjkf's picture
Offline
Last seen: 5 years 10 months ago
Joined: 08/12/2009 - 09:56
UNCTAD study on measuring impact

In this context, I would like to draw your attention to a recent publication by UNCTAD on measuring the impact of ICT. You can find it at http://www.unctad.org/en/docs/dtlstict2011d1_en.pdf.

Kind regards, Torbjörn

 

 

 

gelb's picture
Offline
Last seen: 5 years 5 months ago
Joined: 29/05/2007 - 15:09
Add comment relating to end user evaulation of ICT4D projects

".....efaminu wrote: � You raised deep and probing questions. There is need to track down how ICT is affecting production and productivity, in relation to farmers' behaviour in terms of adoption rates, and what factors facilitate adoption of a particular technology - cost, availability, accessibility etc. coupled with farmers' access to other information sources. For emerging economies these are very important statistics we need especially as penetration of ICT is low.

My comment:

The free, public domain e--book "ICT Adoption in Agriculture" provides a wealth of information - specifically on ICT Adoption  affecting production and productivity. The link to the book is http://departments.agri.huji.ac.il/economics/gelb-main.html
The authors of each of the chapters review their  long and comprehensive experience in ICT Adoption in their fields of competence with a focus on decision makers faced with ICT Adoption and development. Last year the book was accessed several hundred thousand times. 
Comments will be most welcome.
Ehud Gelb gelb@agri.huji.ac.il

shehzaad's picture
Offline
Last seen: 5 years 9 months ago
Joined: 19/03/2009 - 01:24
Dear Ehud, Many thanks for

Dear Ehud,

Many thanks for sharing this link, I will also create an entry to this document in the 'resource' section of the forum.

Hope to hear more from you in this final week of discussions.

Kantiza's picture
Offline
Last seen: 2 months 3 days ago
Joined: 19/05/2011 - 09:21
ICT not to waste time but for taking right decisions

 Dear all,

 
I look of the image of old women using ICT on the homepage of Gelb's website  (  http://departments.agri.huji.ac.il/economics/gelb-main.html) and I just think that they are wasting time because  ICT could not improve their poor conditions, may be, they could communicate in speed  with ICT more than in  traditional way like sending letters by post.
 
A high person that I met one day said " Do you know why children do not like computers? It is because, they do not find bread or love of their relatives in it" so, children are often more clever than old men, they do not waste time.
 
I think that communication using ICT does not lead in short time to the development, ICT is a new media with his specific rules, anyone must know that when he communicates with ICT, he begins a hard game : he is searching for information or he is sending it and he must have time for it and know how to use such information, it means that only experts persons can know what to do with information collected somewhere with ICT
 
The problems of agriculture are also complex in developing countries,  the current tools of ICT can not do anything for the moment in the prevention of climate change and irruptions of new insects or diseases against plants or animals, the help could take long to arrive due of many things.
 
So, it is needed to get specific tools of ICT for improving the agricultural and pastoral development,  those tools could be used only in agriculture or in livestock situation,  in order to avoid noise in the path of communication using current ICT.
 
I think deeply that public actions would be more useful in agriculture  than words sent by ICT like assisting agriculture's actors by  bringing some necessary inputs and  by advising them in taking right decisions.
 
Prof. Antoine KANTIZA, Master UTICEF,-

 

Muaz Jalil's picture
Offline
Last seen: 5 years 10 months ago
Joined: 03/10/2011 - 18:00
Use of RCT as a measurement tool

Probably a bit late in to the discussion but I was wondering if someone can share their experience (if any) with application of Randomize Control Trial (RCT) in evaluating ICT projects. Basically I was hoping to get answers to the following questions:

1. The time and cost involvement from inception to results.

2.  The sample size that was required (control +treatment group) and the unit of sample (individual, class, household etc)

3. The size of the project (in terms of their geographic scope and number of interventions, other than the one tested, that they were working on)

4. The challenges faced.

The reason for asking this is that from what I have heard and read, it seems RCT has been applied quite successfully in projects which have few interventions, with very precise crisp measurable goals (literacy, health indicators, reduction in absenteeism etc) and significant access to resource (time and money). I am bit apprehensive when it comes to applying such methodologies in  projects with 20-30+ interventions, working on light touch approach, private sector driven and focusing on more complex indicators like competitiveness, sustainability, systemic change etc.

 

shehzaad's picture
Offline
Last seen: 5 years 9 months ago
Joined: 19/03/2009 - 01:24
Thank you for your question

Thank you for your question Muaz. I will hope our SMEs will shed more light on this very specific topic that you have raised.

hshahriar's picture
Offline
Last seen: 5 years 9 months ago
Joined: 12/09/2011 - 04:59
Defining impact - a different view point

Hello everybody.

I would like to add a different angle of defining impact of ICT4D initiatives.

Successful ICT initiatives in agriculture often requires the collaboration and participation of private sector companies. When it comes to the measurement of impact, the private sector entities are mostly focused on issues such as financial feasibility, market scenario, penetration statistics, etc. These information, although required, do not serve the prime purpose of development organizations. They require information more emphasizing on the social and pro-poor impact of the services offered. Thus we can see a gap between the information need of the private sector and the development agencies.

Let me refer back to the rural ICT initiative that my colleagues from Katalyst have already shared. One of our learning was that most of the Community Information Centres (CICs) – those we facilitated to be developed – were more or less commercially viable. Our private partner recognized the business case, and is continuing and expanding. However, in some cases we found CICs flourishing through revenues from mobile phone top-ups, photocopying, internet browsing, etc. (definitely positive impact on pro-poor social development). For these few CICs, provision of agro information remains a sporadic and somewhat philanthropic endeavour.

My query to all participants: how can a project specifically geared towards agricultural development tackle this issue?

gelb's picture
Offline
Last seen: 5 years 5 months ago
Joined: 29/05/2007 - 15:09
Re: [e-Agriculture] What is the most effective way to measure

<html>
<body>
At 06:30 03/10/2011, you wrote:
<blockquote type=cite class=cite cite="">Still missing - comments from
the end users/beneficiaries,</blockquote>
Cheers,<br><br>
Ehud Gelb<br><br>
<blockquote type=cite class=cite cite="">Write above this line to
reply|Escriba sobre esta línea para responder|Pour répondre écrivez
au-dessus de cette ligne<br>
Dear | Estimado(a) | Cher/Chère: gelb<br><br>
 <br>
• New commentÂ&nbsp; <b>|</b> Nuevo comentarioÂ&nbsp; <b>|</b> Nouveau
commentaire: <i>ICT index<br>
</i><br>
• Published on <b>| </b>Publicado el <b>|</b> Publié le: <i>03/10/2011 -
06:25<br>
</i><br>
• <b>surabhi75 </b>wrote <b>| </b>escribió <b>|</b> a écrit:<br><br>
<i>Isn't it worth discussing about these indicators and trying to list
out both quantitative and qualitative ones to be able to start thinking
in developing an ICT index for agriculture. This will build the base the
future databasis and comparative studies across states or nations. At
this forum we can think and try to do this.<br><br>
Surabhi <br><br>
</i>• Read more <b>|</b> Leer más <b>| </b>Lire la suite:
<a href="http://www.e-agriculture.org/forumtopics/what-most-effective-way-measure...
http://www.e-agriculture.org/forumtopics/what-most-effective-way-measure...
<br><br>
<br><br>
<br>
--<br><br>
• To manage your subscriptions log in:
<a href="http://www.e-agriculture.org/en/user/login">
http://www.e-agriculture.org/en/user/login</a> and then click on
'Notifications':
<a href="http://www.e-agriculture.org/user/14663/notifications">
http://www.e-agriculture.org/user/14663/notifications</a><br><br>
• Para manejar sus suscribciones, ingrese con su nombre de usuario:
<a href="http://www.e-agriculture.org/es/user/login">
http://www.e-agriculture.org/es/user/login</a> y haga click en
&quot;Notificaciones&quot;:
<a href="http://www.e-agriculture.org/user/14663/notifications">
http://www.e-agriculture.org/user/14663/notifications</a><br><br>
• Pour gérer vos abonnements, entrez votre nom d'utilisateur:
<a href="http://www.e-agriculture.org/fr/user/login">
http://www.e-agriculture.org/fr/user/login</a> et cliquez sur
«Notifications»:
<a href="http://www.e-agriculture.org/user/14663/notifications">
http://www.e-agriculture.org/user/14663/notifications</a><br>
 <br><br>
<br>
View original post:
<a href="http://www.e-agriculture.org/mailcomment/redirect/">
http://www.e-agriculture.org/mailcomment/redirect/%3C14663.31648.2956.13...
</blockquote>
<x-sigsep><p></x-sigsep>
<font size=4>Ehud Gelb<br>
9 Hagalil St.<br>
Rehovot, 76601, Israel<br>
Tel: 972 8 9411677; <br>
Mobile: 972 52 4459721<br>
Fax: 972 8 9417882<br>
skype: ehud.gelb</font> </body>
</html>

Topic locked