The new QueryPic (or what a difference an API makes)

It seems a bit late to be introducing the newest version of QueryPic. Folks are already using it to explore the contents of digitised newspapers made available through Trove and Papers Past. Some, like the National Library of New Zealand, Andrew S. Bowman and theĀ Carnamah Historical Society are already blogging about it. But I suppose I’d better document a few things…

As I noted in my post about QueryPicNZ (yes I now have a rather confusing proliferation of QueryPics), I was waiting for the Trove API to become public. Last week I noticed a little ‘API’ link pop up in the Trove footer and so I set to work…

"The past" versus "the future" in the new QueryPic

My original version of QueryPic (recently reviewed in the Journal of the Digital Humanities) used a series of Python scripts to harvest and scrape content from the Trove web pages. This meant that you had to download the scripts and be code-confident enough to run them in a terminal. It’s still a useful tool and I’ll be updating it as well, but I wanted to create something quicker and simpler that encouraged people to explore and play.

The latest version of QueryPic (QueryPic+, QueryPic Web, QueryPic 2.0?) simply runs in your browser. It uses JQuery to grab data on the fly from the Trove and DigitalNZ APIs. Like previous versions, it uses the HighCharts library to turn the data into pretty graphs.

What does it do? It’s really pretty basic. QueryPic just displays the number of articles matching your search query over time. By default, these are displayed as a proportion of the total articles available for that year, but a dropdown field lets you switch to view the raw numbers. It’s simple, but it’s also remarkably evocative, suggestive and fun. Just try it!

Why stop at just one query? To compare frequency patterns you can add as many as you like. Just keep entering new words or phrases.

If you notice an interesting peak or trough you can just click on it and another API request will be fired off to retrieve the first 20 matching articles. So it’s also a new way of exploring the newspaper databases themselves.

There are plenty of limitations — not all newspapers are digitised, for example, and the quality of the OCR is patchy. The National Library of New Zealand’s post does a great job summing up a number of issues relating to Papers Past. It’s not magic, it’s not perfect, but is it useful? I think so.

Tasks for the future:

  • Create some sort of backend that makes it easy to save , share and cite your query data. The ‘share’ link just regenerates the graph which, of course, might change as new articles are added to the databases.
  • Make it possible to add more complex queries — I want to keep the interface simple, so I’ll probably create a bookmarklet to take any Trove or Papers Past query and display it using QueryPic.
  • As I mentioned over at the WraggeLabs Emporium, I intend to rewrite my various Trove tools to work with the new API. This will include the classic Python version of QueryPic. I still think it’s useful for harvesting your own data.
The code is on my GitHub site and you can also follow updates at the QueryPic page in the WraggeLabs Emporium.


This work is licensed under a Creative Commons Attribution 4.0 International License.

Tim Sherratt Written by:

I'm a historian and hacker who researches the possibilities and politics of digital cultural collections.

One Comment

  1. […] in the Journal of Digital Humanities. With the release of the Trove API earlier this year I rewrote the whole thing in Javascript and let it loose on the web. People could make graphs without having to download any code or fire […]

Leave a Reply