EvalCrisis Blog - 08 - Integrating geospatial methods into evaluations

Author: HUR HASSNAIN

This blog piece is prepared to highlight some of the learning around applications for the geospatial data in evaluation practice from a presentation by the authors titled Geospatial, location, and big data: ‘Where have we been and where can we go?‘ for an online event organized by the European Evaluation Society. The blog also illustrates a few use cases, e.g., the Sierra Leone Rural Renewable Energy Project (RREP) discussed in a previous #EvalCrisis blog post.

Session Summary

Geospatial science is an evolved discipline, whereas the field of evaluation is still in its early stages and often draws on other disciplines for methodological approaches. Geospatial methods and tools have already been proven to be robust and useful for evaluation. Expert views and insights from the experience of applying geospatial methods were shared. The presentation emphasized the role of these innovative methods in evaluation while cautioning against considering geospatial data and approaches as the one-size-fits-all solution for addressing challenges in evaluation practice.

The Session's Key Messages

  • Geospatial methods have grown in popularity, and the opportunities for its application in various fields, including in monitoring and evaluation, have grown exponentially for two main reasons:
    • First, there has been an unprecedented flow of spatial data from multiple sources, including satellite data. Also, moderate and coarse resolution data is free, and high-resolution data is becoming cheaper and widely available.
    • The recent development of data science drives the second opportunity. The infrastructure and tools to deal with enormous geospatial data or the big geodata have grown, and with the availability of application programming interfaces (APIs), cloud-based services, and browser-based development environment have allowed access to geospatial data and analysis without the need for significant computational infrastructure.
  • There are four critical points on how geospatial data and methods can contribute to evaluations.
    • First, it is efficient. We don’t need to visit all the intervention sites. It can save us financial and human resources and can be very useful when working in hard to reach areas, especially in Fragile and Conflict situations or in the current context of the current pandemic.
    • Second, geospatial analyses are scalable, and one can perform analyses at the project site, portfolio level, or at a global level.
    • Third, geospatial methods provide objective evidence and thereby aids transparency.
    • Fourth, geospatial methods and approaches work well in a mixed-methods framework and help deal with common evaluation challenges such as lack of baseline, finding the right counterfactuals, and addressing accessibility issues.
  • Geospatial analysis can be used in different evaluation contexts and for triangulation. E.g., see the GEFIEO’s Land Degradation Focal Area Evaluation that used field surveys to assess the results of restoration initiatives in rural India along with the application of remote sensing data.
  • The main takeaways from experience in applying these approaches are:

     

    • Utilize open data resources and tools​
    • Leverage geospatial methods within mixed methods approaches
    • Plan for variable costs since resources depend on several factors
    • Work with multidisciplinary teams
    • Encourage dynamic learning and upgrading of skills within teams
  • Finally, geospatial data and tools should not be misconstrued as the quick, effective, and easy solution to challenges in evaluation practice. Like any other method, geospatial methods have their limitations. Still, whether applied stand-alone or in combination with other complementary data and processes, these have undoubtedly opened up new avenues for use in evaluation, and they are here to stay.

Speakers' Answers to Participants' Questions

1. It would be interesting to hear what approaches and geo data could be used to measure the potential spatial access to urban facilities (health services for instance).

Answer: Service area analysis is a standard spatial analysis method to estimate the potential access to facilities. This method uses network analysis to estimate the distance and time taken to access facilities.

2. Pre-post datasets do not necessarily allow for making attribution/causal claims. How are you dealing with this?

Answer: The data itself doesn’t lend to attribution. There are two ways we have been dealing with it. One is to consider all the confounding factors that could affect the results. The other way we deal with it is by using other evaluative evidence in a mixed-method framework. For instance, combining the satellite data results with the qualitative methods.

3. Are there examples for collecting qualitative information or analysis, and if so – how?

Answer: Geospatial data could be qualitative as well as quantitative. One example of Qualitative geospatial data can be collected using a smartphone using Open Data Kit. The use of such tools allows integrating interviews with spatial locations.

4. Geospatial data is indeed interesting – but often times the data is hard to find or is private – Are there open data sources that you can recommend?

Answer: The choice of data would depend on the evaluation questions. However, we would like to point out that more and more of the geospatial and remote sensing data are becoming freely available. For GIS, you can check The Humanitarian Data Exchange, and for satellite data, NASA and ESA would be a good start.

5. Do these methods lend themselves well to the work of small NGOs who work on behaviour change at a much smaller scale? For example, personal hygiene, sexual and reproductive health, civic participation?

Answer: Geospatial data and its mapping is hugely powerful for any context but can be expensive. If a small NGO has resources, it can be conducive to use such methods for gathering evidence at the grassroots level, especially in some of the contexts stricken by conflict and fragility, whether the states are affected by an ongoing crisis or not. For example, the social media data (e.g., people complaining about cold) can be taken as an example and develop a geographical framework on it, representing a visualization of personal hygiene vs. the outbreak of diseases such as flu. Relevant stakeholders and donors can then be accessed for designing and funding interventions around vitamin supplements, flu jabs, etc. The same applies to SRH and civic participation. Grassroots organizations, to our knowledge, are far better placed to use such data and suggest remedies of societal or environmental issues because they are closer to reality and can easily enrich geo-location data with real-world examples on the ground.

*This blog post was first published at the European Evaluation Society’s website.