The trained guinea pigs in the Wragge Labs bunker have been churning out all sorts of stuff in the last few months, and I’m way behind in my attempts to document their activities. So this is a bit of a catch-up post to try and commit a few pertinent details to the collective memory bank before they are lost forever in the sleep-deprived fog of day-to-day existence.

Identity upgrades

There have been a number of major improvements to Wragge’s Identity Browser. Regular viewers will recall that the Identity Browser is built on top of the People Australia SRU interface. You might not realise, however, that People Australia contains details of many organisations as well as people. We can only be thankful that it wasn’t called Entity Australia.

The first version of my Identity Browser only searched for people, but now all your corporate-entity-identification needs are also met, with only a few minor changes to the interface so-beloved by numerous generations of identity seekers. To be specific, through the wonders of drop-down technology you can choose whether you want to search for a person or an organisation. Or not. You can also just ignore that and search for everything and get back sensible results anyway. It’s your choice. Or not.

Gaze in awe at the power of my dropdown

Ah pattern matching… there are few phrases so redolent of warm summer days, hidden pleasures, and the subtle delights of wildcard characters. The People Australia SRU interface was sadly lacking in the pattern matching department, but this has now been rectified. So now you mix your stems and asterixes with wild abandon. Searching for ‘Curtin, J*’ will now retrieve all those Curtins whose names begin with ‘J’. Amazing isn’t it?

Astonishing too is the fact that the accompanying ‘Identify me!’ bookmarklet continues to function with nary a murmur of protest. There is, however, a little bit of cleverness built-in to enhance your bookmarklet experience. If the text that you highlight has a comma in it, the Identity Browser will conclude that you’re feeding it the name of a person – ie Surname, Firstname – and will treat the Firstname as a stem. So if you highlight ‘Whitlam, G’ and click on the bookmarklet, the Identity Browser will be kick-started into life, searching for everything that matches surname equals ‘Whitlam’ and firstname is like ‘G*’. If there’s no comma – ie firstname secondname – then it heads off to look for either a person whose surname equals ‘secondname’ and whose firstname is like ‘firstname*’, or an organisation whose name includes both ‘firstname’ and ‘secondname’. Got all that?

Basically the idea was to try and provide some sensible defaults so you really don’t have to think about it too much.

I have it in my head to prepare a long and rapturous homage to the wonders of machine tags. With their sly semantic ways and easy-going nature, they offer some exciting possibilities not just for user-generated content, but user-generated meanings and user-generated relationships. But for the full, ripe pleasure of that post you will have to wait another day, for now I shall simply say that as well as RDFa, the Identity Browser provides automagically-generated machine tags.

Where might you use them? Flickr’s a good place to start. Try identifying the subjects and creators of Flickr photos. At the NSW Reference and Information Services Group Seminar the other day I challenged those in attendance to go forth and machine tag. Already more than 100 machine tags have been added to Flickr using my Identity Browser. Expect to hear more about the Great Flickr Machine Tag Challenge soon…

One more thing… try adding ‘.rdf’ on to the end of an identity record – eg http://wraggelabs.com/identities/person/612109.rdf. Just an experiment at the moment…

More machine tag love

One night on Twitter, @lifeasdaddy pointed out that someone had started using fragments of urls from the NLA newspapers site as tags in the Powerhouse Museum’s collection database. In the conversation that ensued with @sebchan and others, I suggested that the PHM could encourage this sort of rich tagging by supporting machine tags, with all their wonderful juicy semantic goodness The guinea pigs got excited as well, and before I knew it, they’d constructed a little Semweb Helper app.

The Semweb Helper comes with its very own custom-tailored bookmarklet. If you find an article on the NLA newspapers site that you’d like to point to, just click on the bookmarklet and marvel as a range of useful machine tags are automagically generated. Then you just pick the appropriate tag, copy and paste et voila – instant semantic gratification.

Try out the Semweb Helper

It’s a very simple little app, and really just a demonstration of how semantic web technologies might be made available to the masses. It was also the first time the guinea pigs had been allowed to play with the Google Apps Engine.

Who am I?

This short catch-up post has become something quite long and rambling. Did I mention that I’m sleep-deprived? Anyway, a recent addition to the Wragge Labs range of lifestyle accessories is ‘Who am I?’ – a simple little game that is something like a cross between hangman and Wheel of Fortune. Choosing a person at random from People Australia and the Australian Dictionary of Biography, ‘Who am I?’ tests your powers of logic, stamina and historical guesstimation.

Your challenge is to figure out the surname of the mystery historical personage. To help you there are a series of clues, such as their birthplace and known associates. With each guess you also see a little bit more of their portrait. But beware! For ten wrong guesses are all that are permitted to any so brave as to enter upon this quest. Not eleven or twelve, but ten and ten only. To ignore this limit is to invite ridicule and disdain – do so at your peril.

Play Who am I?

‘Who am I’ builds upon some work I’ve been doing for the National Museum of Australia – looking at ways of mashing together various types of date-identified data. As part of that project I’ve built a series of APIs and have scraped, pummelled and munged data from a variety of sources.

What’s the point? I wonder this myself sometimes, particularly after I fling such things off into the aethernet and hear naught but a rare retweet. I am, after all, only in it for the glory, oh and the money of course. (Hmmm, I must look again at that business plan.) The point is twofold: first to highlight possibilities for the re-use and remixing of cultural data; second, to play with game-based models for discovery and exploration of cultural resources; and… err… thirdly just to try building something a little different.

Of course, if you like ‘Who am I?’ you will probably also want to try Headline Roulette…

Headline Roulette Reprieve

At the end of our last instalment, the future of Headline Roulette seemed in dire peril. Changes to the National Library of Australia web site threatened its very existence. Did it have a future? Could it survive? And did anybody care?

As we pick up the story oblivion looms. The feared changes are confirmed, but just as all seems lost… is it? Could it be? Yes, an advanced search facility is added to the newspapers site within Trove. Sensing this may be their only opportunity, the guinea pigs leap into action, building a new screen-scraper, saving Headline Roulette from doom, and setting the world upon the path to a safer, happier future.

In short, Headline Roulette will live on… so enjoy.

Handing out some presents

My head is easily turned by flattery and praise. Yes, I really am so shallow and so vain. But this means that if people say nice things to me, I’m inclined to give them presents.

As well as doing exciting things in the web 2.0 realm for the PROV, @asaletourneau leaves nice comments on this blog. So he earned himself a present. It’s not much, but I built a userscript that displays photos from the PROV site in a neat little slideshow (it’s the non-3D javascript version of CoolIris). Install Greasemonkey, get the userscript and try it out (just do a search, then click on the ‘Browse as slideshow’ button’).

PROV transport photos in a pretty slideshow

The State Library of NSW, or more specifically @ellenforsyth, also earned my favour by inviting me to rave on about Linked Data at the afore-mentioned NSW RISG seminar. As a result, I added support for the SLNSW photo collections to my Flickr Context Harvester userscript. Well… it’s the thought that counts, right? Once again – install Greasemonkey, get the userscript and then try it out.

The Flickr Context Harvester in action

And coming up…

Stay tuned for more on the Great Flickr Machine Tag Challenge, screencasts demonstrating my Identity Browser, some playing with relationships, and much much more. But right now the squirming baby on my lap needs a nappy change…

Did I mention that I’m sleep deprived?

I was doing some research using the National Archives of Australia’s RecordSearch database the other day and became frustrated that there is no way of seeing how many pages are in a digitised file without clicking on the ‘Display digital copy’ link. So I fixed it.

As a userscript it’s hardly worthy of a blog post. All it does it find out how many pages are in the file and insert the number in the link text. It’s very simple. But I think it’s also a useful illustration of the changing balance of power between archives and their users.

William E Landis argued that archivists were ‘guilty as a profession of fetishising the outputs of our descriptive systems’. The design of finding aids have often been determined not by the needs of users but by a desire to faithfully represent the underlying archival architecture. But now users don’t have to just take what they’re given.

Technologies such as Greasemonkey are useful for sketching out alternatives. For organisations with IT systems that inhibit experimentation, Greasemonkey (or Mozilla’s Jetpack) provides a way of playing with interfaces without touching any of the underlying code. My rewrite of the way RecordSearch displays digitised files is an example of this.

But no one interface is ever going to meet the needs of all archive users. Fortunately, there are a growing number of ways in which archives can work in partnership with their users to help them create the interfaces they want and need.

Archives are starting to expose their data directly using APIs and linked open data. This gives users the power to create whole new applications. But I still think there’ll be a place for the little tweak – a simple hack that meets some small but specific need. I can imagine communities of interest building and sharing a range of tools, hacks, applications and interfaces specifically tailored to their research habits.

So if you don’t like it, fix it.

Instead of idly waiting for visitors to stumble over their holdings on some lonely information by-way,  archives are starting to push their content out into the bustling metropolis of the social web. They are going where the people are. Photographic collections, in particular, are gaining new lives and new audiences thanks to Flickr.

But that’s only part of the story. Released into the wild, these photos are slowly picking up the habits of the locals. They are making friends, building connections, even speaking with new accents and dialects. Commented, tagged, organised, linked – they are building new contexts for themselves outside of the cloying control of archival descriptive systems.

Unfortunately it seems there is often a chasm between the old lives of the photos, documented in databases and finding aids, and their new post-institutional careers. This is a pity because the new contexts they are gathering can help us both understand and find them. What can we do to overcome this divide? How could finding aids harvest and display the user-generated content that aggregates around collection items living in the outside world?

The good news is that the tools to start doing this already exist – Flickr has a powerful API that makes it easy to extract photo metadata. Time for a bit of experimenting…

The first result is a userscript that displays Flickr comments in a number of collection databases. Just install it and then try it out:

• National Archives of Australia Photosearch – try it!
• State Records NSW Photo Investigator – try it!
• National Archives and Records Administration ARC – try it!

Flickr comments in PhotoSearch

Gory details follow…

So to begin with I thought I’d just harvest comments from Flickr and display them within existing collection interfaces. As before (here and here), Greasemonkey was my tool of choice for hacking finding aids. The plan was to trigger a Greasemonkey script when you arrive at a photo in a collection database, the script would then:

• extract a unique identifier for the photo that could be used to find it in Flickr
• send off a request through the Flickr API to see if the photo was there
• if so, then fire off another request to retrieve any comments
• format the comments and insert them at a suitable point in the DOM of the database page

Easy! Obviously for the script to work there needed to be a way of connecting entries in the database with photos on Flickr. In practice this means that the photos need to be described at item level, and that a unique identifier needs to be used somewhere in the description of the photo both on Flickr and in the collection database.

Any archive that meets these criteria is a candidate for inclusion. Only three pieces of information are necessary:

• the institution’s Flickr id
• an expression to extract the identifier from the database page
• an expression to identify the point on the database page at which the comments should be inserted

The expressions could use XPath or regular expressions – whatever it takes to find the desired elements. I’m using JQuery, so that makes selecting elements a lot easier. For example, NARA ARC includes the item identifier in a div with the class ‘arcID’, so I just select that element using JQuery and then use regex matching to pull out the number:

this.identifier = $('.arcID').text().match(/ARC Identifier (\d+)/i)[1];

To start with I’ve included the databases of three institutions:

• the National Archives of Australia’s PhotoSearch database
• State Records of NSW’s Photo Investigator
• the US National Archives and Records Administration’s Archival Research Catalog

This is the code to save the settings for each institution:

<br />
if (document.location.href.match(/naa.gov.au\/scripts\/PhotoSearchItemDetail.asp/i)) {<br />
this.name = 'NAA';<br />
this.identifier = document.location.href.match(/M=0&#038;B=(\d+)/)[1];<br />
this.flickrId = '24849862@N08';<br />
this.position = 'table:last';<br />
} else if (document.location.href.match(/records.nsw.gov.au\/asp\/photosearch\/photo\.asp\?/i)) {<br />
this.name = 'StateRecordsNSW';<br />
this.identifier = document.location.href.match(/photo\.asp\?([\d\w_]+)/i)[1];<br />
this.flickrId = '27331537@N06';<br />
this.position = 'table:first';<br />
} else if (document.location.href.match(/arcweb.archives.gov\/arc\/action\/ShowFullRecord|arcweb.archives.gov\/arc\/action\/ExternalIdSearch/i)) {<br />
this.name = 'NARA';<br />
this.identifier = $('.arcID').text().match(/ARC Identifier (\d+)/i)[1];<br />
this.flickrId = '35740357@N03';<br />
this.position = '.genPad:first';<br />
}<br />

From there it’s just a matter of building the calls to the API using Greasemonkey’s built-in  GM_xmlhttpRequest method. Once the comments are retrieved, they’re given some basic formatting and inserted at the point in the DOM identified by the siteDetails.position property. Once again, JQuery greatly simplifies all the DOM manipulation. If there are no comments then a suitable message is inserted together with a link to the photo in Flickr. Finally some CSS is added to prettify it all a little bit.

You can view the full code on the Userscripts site.

Of course, it would be good to have this sort of stuff happening on the server side. In fact, with a few small modifications, this script could just be dropped into the code of any of the collection databases I’ve used. But in the meantime, Greasemonkey gives us a chance to play around with some of the possibilities – to start thinking about what finding aids might be like.

So what’s next? I’d like to do some playing around with tags and locations, perhaps using them to suggest related photos. I’ve also just realised that Flickr machine tags allow semantic markup… hmmm…

If you have any suggestions for databases to add to this script – let me know!

Mapping our Anzacs, in case you don’t know, provides a Google map interface to the 375,000+ WWI service records held by the National Archives of Australia. Amongst other other things, you can add scrapbook posts to individual entries and create tributes. It’s meant to encourage exploration, so go on… explore!

If you’ll do, you’ll notice that there are direct links into the National Archives’ database RecordSearch. However, there are currently no links going to other way. Why does this matter? Well perhaps you’d like to use NameSearch to find an individual record, but then add a scrapbook post in Mapping our Anzacs. Up until now you had to find them all over again. But not any more…

Introducing our new range of ‘View in Mapping our Anzacs’ buttons:

• For the discerning Firefox devotee we have a Greasemonkey userscript which adds a button to the RecordSearch item details page.
• For fashion-challenged IE user we have a bookmarklet. Just right click on this link – View in Mapping our Anzacs – and save it as a favourite in your ‘Links’ folder (you may need to enable the ‘Links’ toolbar first by checking Tools > Toolbars > Links.)

Yes, it’s true… you could use the Bookmarklet with Firefox (just drag it to your bookmarks toolbar), but Greasemonkey is so much more chic.

Once you’re fully button-enabled just head into RecordSearch, find an item in series B2455 (the WWI service records) and click! Hurrah! You will be instantly transported to Mapping our Anzacs.

You can test out your new button by heading here:

• B2455, WRAGGE C L E

All dressed up – RecordSearch has a new look

The new version of my Greasemonkey userscript, RecordSearch Image Tools, gives RecordSearch’s digital image pages a rather new look. My previous version had done away with the tired ol ‘lemon-chiffon’ background colour, but I decided it was time to get a bit more adventurous, so I blitzed the old design and rebuilt the page from the beginning.

As you can see from the screenshot, I’ve tried to give the images as much as the screen as possible. I’ve also created a consistent set of navigation buttons, and improved the functionality in various ways.

Archives in 3D – CEDTs from NAA: ST84/1, 1906/21-30

But the most exciting thing is that I’ve worked out how to feed the images to the fabulous CoolIris 3D wall. My previous version used the javascript version of CoolIris, which displayed the images as a flat (but still very nice) slideshow. But now, if you have the CoolIris plugin installed you can zoom, pan, fly through the file, dipping in and out as you so desire. It’s a new way of looking at archives.

You can zoom in and out, even see a complete file on a single screen – NAA: B2455, WRAGGE C L E

To try for yourself you need to have Firefox with the Cooliris plugin installed. Then you need to get the Greasemonkey extension and, finally, install my userscript. Then just dive into RecordSearch, find a digitised file and enjoy!

File links:

• NAA: ST84/1, 1906/21-30
• NAA: B2455, WRAGGE C L E

BREAKING NEWS (2.00pm, Monday, 8 December): RecordSearch seems to be back on the old subdomain, so now the userscript fix is not working! To be safe, I’ve updated the userscript again so that it will work on both the old and new subdomains. I’ll do the same with the Zotero translator, though for the time being it should be working. If you updated the userscript in the last few hours, you’d better do it again – sorry…

Just a quick update in regard to the tools I’ve developed for use with the National Archives of Australia’s RecordSearch database — the RecordSearch translator for Zotero, and the RecordSearch Image Tools userscript.

The subdomain under which RecordSearch operates was changed over the weekend to recordsearch.naa.gov.au. As a result of this change neither the Zotero translator nor the userscript will work.

I’ve updated the the RecordSearch Image Tools userscript to fix the problem. Just go to:

http://userscripts.org/scripts/show/33485

and click on the ‘Install’ button. This will overwrite the old version with the new one. As a bonus extra, if you haven’t installed or updated the script recently, you’ll notice that I’ve improved the way it prints out files. Details are on the script page.

I will upload a new version of the Zotero translator to the developers asap. Once they add this to the repository your Zotero will automatically install the fix. However, I’m not sure how long it will take for it to be added. I’ll post an update once it’s been fixed.