On Friday 19 October I joined an enthusiastic group of digital humanities explorers at a Deakin University event entitled Dipping a Toe into the Digital Humanities and Creative Arts. @catspyjamasnz has assembled an excellent summary of the day in Storify.

"Every story" – making the data visible by @wragge #dhdeakin #research pic.twitter.com/qmnK1q9s

— Joyce Seitzinger (@catspyjamasnz) October 18, 2012

In the morning I told the story of Invisible Australians. You can view the slides of Too or important not to try and listen to my dodgy audio recording via SoundCloud.

In the afternoon I gave a whirlwind workshop which included a headline roulette smackdown and an introduction to the wonders of Zotero.

I’m deeply in love with the collections of the National Archives of Australia. They move me, they inspire me, they make me want to do something. How do I express my love? I’ve written stories about things like atomic bombs, progress, astronomy and weather forecasting — pursuing lives and events documented in the Archives’ rich holdings. I work on projects like Invisible Australians, hoping to bring the compelling remnants of the White Australia Policy to broader public attention. And I build things. I make tools that help other people explore, understand and use the Archives. I do this because these riches need to be used. They need to be shared. They need to be part of the fabric of our lives.

A few years ago I created a little script for Firefox that put a fresh face on the display of digitised records in the National Archives’ RecordSearch database. It’s publicly available and has been installed more than 500 times. Demonstrating this script at the ‘Doing our bit’ Build-a-thon a few weeks ago made me realise again both how useful it was and how much work it still needed.

One of the most exciting features when I first created the script was the ability to display the records on a ‘3D wall’, courtesy of a Firefox plugin called CoolIris. But CoolIris uses Flash and is no longer being supported. Time for a new approach.

Say hello to the Archives Viewer (naming things isn’t really one of my strengths). Instead of rewriting my existing script I decided to create a completely new web application. Why? Mainly because it gave me a lot more flexibility. I could also make use of a variety of existing tools and frameworks like Django, Bootstrap, Isotope and FancyBox. Standing upon the code of giants, I had the whole thing up and running in a single weekend. The code is available on GitHub.

What does it do? Simply put, just feed the Archives Viewer the barcode of a digitised file in RecordSearch and it grabs the metadata and images and displays them in a variety of useful ways. It’s really pretty simple, both in execution and design.

Yep, there’s a wall. It’s not quite as spacey and zoom-y as the CoolIris version, but perhaps that’s a good thing. It’s just a flat wall of page image thumbnails with a bit of lightbox-style magic thrown in. But when I say just, well… look for yourself. There’s something a bit magical about seeing all the pages of a file at once, taking in their shapes and colours as well as their content. This digital wall provides a strangely powerful reminder of the physical object.

Of course you can also view the file page by page if you want. Printing is a snap — just type in any combination of pages or page ranges and hit the button. The images and metadata are assembled ready to print. No more wondering ‘which file did this print out come from?’.

But perhaps the most important feature is that each page has it’s own unique, persistent url. Basic stuff, but oh, so important. With a good url you can share and cite. Find something exciting? Tell the world about it! I’ve included your typical social media share buttons to help you along.

1891 draft of the Australian Constitution — with doodles. http://t.co/aCulSUhd

— Tim Sherratt (@wragge) August 23, 2012

One disadvantage over the original userscript is that the viewer isn’t directly linked to RecordSearch. You probably don’t want to have to cut and paste the barcode every time you view a file. So I’ve also created a couple of connectors that ummm… connect things up.

The first connector is just a bookmarklet. A bookmarklet is just a little piece of javascript code disguised as a browser bookmark. Just drag this link — Archives Viewer — to your browser’s bookmark toolbar. Then when you’re on the item page of a digitised file in RecordSearch, just click the bookmarklet and you’ll be instantly transported to the wall.

The second connector is a bit smarter. It’s an enhanced version of another userscript I wrote to display the number of pages in a digitised file. It still does that, but now it also rewrites the links to the digitised files so that they automatically open in the Archives Viewer. It’s a bit harder to install. You need Chrome or Firefox and the add-ons Greasemonkey (for Firefox) or Tampermonkey (for Chrome). Then just go to the userscript page and hit the big ‘Install’ button.

You might be wondering about Zotero (at least I hope you are). My Zotero-RecordSearch translator lets you capture page images and metadata direct to your own research database, so what happens when you’re transported across to the Archives Viewer? Never fear, I’ve written a new translator that lets you save pages as you could in RecordSearch. Even better, you get a persistent, context-enriched url, and the ability to capture multiple pages at once. Yippee!

But that’s not quite all. Buried within the pages is some lovely Linked Open Data. To be truthful, it’s not really very ‘linked’ yet, but it does expose the basic metadata in a machine-readable form, borrowing from the vocabularies of projects like Locah and the Archival Ontology. It’s an experiment, as is the Archives Viewer itself. We can learn by doing.

Makin' triples from dhistory Archives Viewer RDFa: file – http://t.co/D0xCehjr & page http://t.co/gSd87cRX

— Tim Sherratt (@wragge) August 26, 2012

I’ve given quite a few talks over recent times encouraging people to take up their tools and start hacking away at the digital collections of our cultural institutions. Yes, I admit it, I’m an impatient historian (and a grumpy one at that). But it’s also because I think it’s important that we recognise that access is never just something you’re given. It’s something that we make through our stories, our projects, and our tools. It’s something that’s grounded in respect and powered by love.

I tell the full story of the newspaper’s campaign in Inigo Jones: The Weather Prophet. but I’ve always wanted to do something more with the letters. Inspiration finally arrived this year when a conference on rural media was organised to mark the The Land‘s centenary. I decided to create a little exhibition using the letters and, in doing so, provide myself with a platform for further research.

Step one: assembling the letters with Zotero

I’d already asked for the file containing the letters to be digitised, so images of all of the letters were available through RecordSearch. Zotero ships with a RecordSearch translator that I wrote some years ago, so 102 clicks later, I’d captured both the metadata and all of the images into my own database.

The Zotero translator saves individual pages with a generic title, so I then had to do some manual editing. I also had partial transcripts of the letters which I’d created during my original research. I simply cut and pasted the transcripts into a note field in Zotero and tidied them up. Pretty soon my collection was complete.

Step two: creating an exhibition with Omeka

I actually started building the exhibition during an Omeka Bootcamp session I ran at THATCamp Melbourne. Starting with nothing but a LAMP server and my Zotero library, I had the basics of site up within the hour.

It was simply a matter of installing Omeka, adding the Zotero import plugin, creating an API key in my Zotero profile, and then pointing the Zotero import plugin to my collection. (The API key is necessary if you want to download files, such as my images.)  With a press of a button the internets started chugging away and pretty soon all the letters were in Omeka. Hey presto — instant exhibition!

With the framework built I could start to play a bit. First I installed Omeka’s geolocation plugin and started mapping where the letters had come from. Google’s geocoder did a pretty good job of finding many of the small country towns, but I also made use of Geoscience Australia’s gazetteer. In the end, I managed to geolocate all but one letter. Hey presto — a map!

I thought the pre-installed ‘Summer’ theme suited my exhibition, but I did make a few minor tweaks here and there. This included adding some RDFa into the list of references (which were also imported from Zotero).

Step three: going deeper with Voyeur Tools

The transcripts of the letters were all available on the site, but I thought it would be interesting to analyse their content a bit more systematically. To do this, I exported the collection from Zotero and wrote a few lines of Python code to save each transcript in a separate text file, named after the letter’s author. I could then zip up the transcripts and feed them to Voyeur Tools.

Most of the letters are pretty short, so there’s a limited amount to be gleaned from them in isolation. Voyeur Tools did, however, make it easy for me to explore further the prevalence of the phrase ‘on the right track‘.

What will be more interesting will be to compare the letters to other bodies of text about Inigo Jones, weather forecasting and rural life. I’m hoping to start mining some of this sort of material from the Trove newspapers database.

Next steps?

Using Zotero , Omeka and Voyeur Tools it was quick and easy to build my own little research hub. It’s still a work-in-progress of course — that’s the point! Now I can continue to assemble and analyse my sources, while giving my work a public face. Magic!

BREAKING NEWS (2.00pm, Monday, 8 December): RecordSearch seems to be back on the old subdomain, so now the userscript fix is not working! To be safe, I’ve updated the userscript again so that it will work on both the old and new subdomains. I’ll do the same with the Zotero translator, though for the time being it should be working. If you updated the userscript in the last few hours, you’d better do it again – sorry…

Just a quick update in regard to the tools I’ve developed for use with the National Archives of Australia’s RecordSearch database — the RecordSearch translator for Zotero, and the RecordSearch Image Tools userscript.

The subdomain under which RecordSearch operates was changed over the weekend to recordsearch.naa.gov.au. As a result of this change neither the Zotero translator nor the userscript will work.

I’ve updated the the RecordSearch Image Tools userscript to fix the problem. Just go to:

http://userscripts.org/scripts/show/33485

and click on the ‘Install’ button. This will overwrite the old version with the new one. As a bonus extra, if you haven’t installed or updated the script recently, you’ll notice that I’ve improved the way it prints out files. Details are on the script page.

I will upload a new version of the Zotero translator to the developers asap. Once they add this to the repository your Zotero will automatically install the fix. However, I’m not sure how long it will take for it to be added. I’ll post an update once it’s been fixed.