Finding your way in the agricultural spatial data jungle

The recent announcement of a major rethink for the HarvestChoice website sent me on a voyage of discovery. Remember that HarvestChoice, “a partnership between IFPRI and the University of Minnesota, generates knowledge products that help guide strategic investments to improve the well-being of poor people in Sub-Saharan Africa through more productive and profitable farming.” That classically includes maps. And there are definitely a lot of spatial datasets on the site. And in Mappr there is a potentially useful online tool for combining different layers and summarizing the results. No doubt a valuable site.

But let me focus here on a separate issue, and that is whether the resources available globally to develop and present such data, in such sites, are being used, er, optimally. The question occurred to me when, in exploring the maps available at HarvestChoice, I came across a dataset labelled “Cattle population (head) (2005).” The source for the data is helpfully provided:

HarvestChoice/IFPRI 2010/FAO. 2007. Gridded livestock of the world 2007, by G.R.W. Wint and T.P. Robinson. Rome, pp 131.

The funding agency is given as the Bill and Melinda Gates Foundation, and data availability is described as follows:

Data available for download in CSV format. Spatial data layer may be explored using MAPPR and downloaded in geoTIFF and ASCII raster formats.

Now, the 2007 FAO publication referenced as the source of the data is downloadable as a pdf from FAO. But the data are also available in a variety of formats from the FAO Animal Production and Health Division website. Among the formats FAO (and only FAO) offers is a Google Earth file, which is probably going to be the most useful for the average user.

And then there’s ILRI. Map 4 in their monumental Mapping Poverty and Livestock in the Developing World of 2002 does not look too dissimilar to the above, though it may be based on older data. It’s a pdf, of course, but for all I know the data are also available in a more useable format somewhere on the ILRI website. I can’t be sure because their database of GIS datasets is just too clunky to spend any significant time struggling with, frankly. Which is not to say that I didn’t…

So what exactly has HarvestChoice added to the sum total of human happiness by making “Cattle population (head) (2005)” available on its website, above and beyond FAO’s contribution? I’m really hard put to say. The datasets are easier to use in Google Earth from FAO’s website than in any format provided by IFPRI, in my opinion. And the basic analyses available in Mappr will probably be useful to some, sure, but since sharing the map itself is tricky from there, you still have to got to FAO for that.

So the poor user probably has to navigate at least two separate and quite different websites to get all she needs. One wonders whether the donors behind all of these different efforts to provide data on cattle population numbers around the world (if indeed there were more than one, apart from the aforementioned Gates Foundation) considered that before green-lighting the projects. And, of course, that’s just cattle numbers. Life is just way too short for me to plough through all the other datasets at HarvestChoice looking for overlaps like these, but I’d be willing to bet this is not an isolated example. We know that crop distribution data is also out there in all sorts of different places, forms and formats. And, in fact, there’s some evidence of balkanization in other types of livestock data too. I’m not saying it’s hell out there for spatial data in agriculture, on a par with Genebank Database Hell. But it is definitely a bit of a jungle.

Does it matter? I don’t know. Maybe the users of these types of data really like having multiple websites to visit for downloading and visualization. Maybe the funds involved in developing all these different datasets and websites are minimal anyway. I’d really like to hear from the people involved at HarvestChoice, FAO and ILRI (are there other players?). But is there even a forum in which these guys meet to discuss data issues? Because if there were, and I by some miracle were to be invited to it, what I would say is that what I would prefer is a single place to go for cattle population numbers maps and the like (e.g. crop production data), with lots of options for exporting the raw data for use in my own GIS, the ability to import and combine my own datasets online, and some elegant ways of sharing the results. ((Take a look at this, for example. I’m not saying it’s perfect, but I like the sharing options.)) That, for me, would be value for money. And I can’t believe I’m alone in that.