There seems to be a lot of topic modelling going on at the moment. Any why not? Projects like Mining the Dispatch are demonstrating the possibilities. Tools like Mallet are making it easy. And generous DHers like Ted Underwood and Scott Weingart are doing a great job explaining what it is and how it works.

I’ve talked briefly about using topic modelling to explore digitised newspapers, something that the Mapping Texts project has also been investigating. But I’ve also been following with interest Chad Black’s use of algorithmic techniques, including topic modelling, to look for local variations amidst the legal system of the early modern Spanish empire.

As part of the Invisible Australians project, Kate and I are exploring the bureaucracy of the White Australia Policy. In particular, we’re interested in the interaction between policy and practice, between the highly-centralised bureaucracy and the activities of individual port officials. Like Chad, we’re interested in mapping local variations — to try and understand the bureaucracy from the point of view of an individual forced to live within its restrictions.

I recently gave a presentation about the project at Digital Humanities Australasia (post coming soon!), and in preparation I decided to try a few topic modelling experiments. They were very simple, but I was impressed by the possibilities for exploring archival systems.

The problem I started with was this. The workings of the White Australia Policy are well documented by records held by the National Archives of Australia. Some series within the archives are specifically related to the operations of the policy — such as those containing many thousands of CEDTs. But there are also general correspondence series created by the customs offices in each state, as well as the Commonwealth Department of External Affairs which administered the Immigration Restriction Act (responsibility was later taken by the Department of Home and Territories and it’s successors). These general correspondence series are important, because they often include details of difficult or controversial cases — those that required a policy judgment, or prompted a change in existing practices. But how do you find relevant files within series that can contain large numbers of items?

Series A1, for example, is a correspondence series created by the Department of External Affairs. It contains more than 60,000 items. Past research tells us that amongst these 60,000 files are records of important policy discussions relating to White Australia. But these files tend to be labelled with the names of the people involved, so unless you know the names in advance they can be difficult to find.

Mitchell Whitelaw’s A1 Explorer, part of the Visible Archive project, lets you to explore the contents of Series A1 in a easy and engaging way. But while the A1 Explorer provides new opportunities for discovery, it doesn’t offer the fine-grained analysis we need to sift out the files we’re after. And so… topic modelling.

The process was pretty simple. While I can dip into my bag of screen-scrapers to harvest series directly from the NAA’s RecordSearch database, there was already an XML dump of A1 available from data.gov.au. So I extracted the basic file metadata from the XML and wrote the identifiers and titles out to a text file, one item per line. Following the instructions on the website I then loaded this file into Mallet:

/Applications/Mallet/bin/mallet import-file --input ./A1.txt --output A1.mallet --keep-sequence --remove-stopwords

Then it was just a matter of firing up the topic modeller:

/Applications/Mallet/bin/mallet train-topics --input ./A1.mallet --output-state ./A1.gz --output-doc-topics ./A1-topics.txt --output-topic-keys ./A1-keys.txt --num-topics 40

Again, I just followed the examples on the Mallet site.

Once it was finished I opened up A1-keys.txt to browse the ‘topics’ Mallet had found. The results were intriguing. There are a large number of applications for naturalisation in A1, so it’s no surprise that ‘naturalisation’ figures prominently in a number of the topics. What was more interesting was the way Mallet had grouped the naturalisation files. For example:

naturalization christian hans hansen jensen petersen andersen nielsen larsen christensen johannes jens niels pedersen andreas johansen martin jorgensen

and

naturalisation certificate giuseppe salvatore frank la leo samios spina sorbello leonardo fisher natale patane torrisi barbagallo luka rossi ross

Based on the co-occurrence of names within the file titles, Mallet had created groupings that roughly reflected the ethnic origins of applicants. It makes sense when you think about what Mallet is doing, but I still found it pretty amazing.

Mallet also found clusters around the major activities of the department, such as the administration of the territories. But of most interest to us was:

1 0.55539 passport ah student exemption students lee wong chinese young deserter education sing wing chong readmission son hing chin wife

The Chinese names alongside words such as ‘readmission’ and ‘wife’ suggested that this topic revolved around the administration of the White Australia Policy. This was easy to test. In A1-topics.txt was a list of every file in the series and their weightings in relation to each of the topics. I wasn’t sure what was a reasonable cut-off value to use in assessing the weightings, but after a bit of trial and error I fixed on a value of 0.7. I then just extracted the identifiers of every file that had a weighting greater than 0.7 for this topic. I used the identifiers to build a simple web page that Kate and I could browse. I also included links back to RecordSearch so we could explore further.

Browse the full list

It’s a pretty impressive result. Instead of fumbling with the uncertainties of keyword searches, we now have a list of more than 1,300 files that are clearly of relevance to Invisible Australians. There’s a few false positives and there are likely to be other files that we’ll have missed altogether, but now we have a much clearer picture of the types of files that are included and how they are described.

And that was at my first attempt, simply using the default settings. I’m now starting to play around with some of Mallet’s configuration options to see what sort of difference they make. I’m also keen to try out GenSim, a topic modelling package for Python.

I’m really excited about the possibilities of these sort of tools for analysing the contents of archival descriptive systems, something I mentioned in my Digital Humanities Australasia paper. Much more to come on this I suspect…

Earlier this week it was announced that the Mosman Library had been awarded a Library Development Grant for an innovative project that aims to document stories and artefacts relating to the First World War. I’m very excited to be part of it. As well as working with the local community in the creation of a new resource, the project offers an interesting opportunity to explore how we can link in with the ever-increasing volume of WWI material being published as linked data around the world.

But thinking about this new project has also made me reflect again on the creation of Mapping Our Anzacs — a project that still fills me with great pride and immense frustration. I thought I might as well finally post a couple of things I wrote about the project back in 2009. They’re a bit out of date, but I think there’s still a few useful lessons to be gleaned.

The first is a case-study that focuses on the crowdsourcing aspects of Mapping Our Anzacs. The second looks at the project as an example of a mashup. Thanks to Kate Theimer for initiating and publishing both pieces.

Bringing life to records

2009 preprint version of case-study originally published in Kate Theimer (ed), A Different Kind of Web: New Connections between Archives and Our Users, Society of American Archivists, Chicago, 2011. [Order here]

Overview of repository

The National Archives of Australia is responsible for preserving and making accessible the records of the Commonwealth of Australia. It employs more than 400 staff, with offices in Canberra and every state capital. Its holdings include more than 360 shelf km of records – around 69 million items. Through its digitisation program more than 1.6 million items have been fully digitised, making nearly 20 million digital images available online. The National Archives’ website now provides the main point of access for researchers, with more than 2 million images viewed through the online database RecordSearch in the year 2007–8.

Business drivers

Most people now experience the collections of the National Archives of Australia online. With an obligation to provide ‘an accessible, and interpreted, national archival collection’ the Archives is looking to new technologies to enhance access and improve efficiency.

The idea for Mapping our Anzacs arose during planning for a travelling exhibition on the impact of World War I, timed to coincide with the 90^th anniversary of the war’s end. Public interest in commemorating Australia’s war effort was as strong as ever, so a website that encouraged local participation seemed a useful way of extending the exhibition and its accompanying education program.

The major focus of both the exhibition and the website was to be the 376,000 service records documenting the experiences of Australian men and women during World War I held by the National Archives. These records had been fully digitised and described as part of a major project entitled ‘A Gift to the Nation’, but were still somewhat buried within our collection database.

Mapping our Anzacs was intended to highlight these records and open them up to local communities. First a map interface would allow service records to be discovered by place of birth or enlistment. Secondly, users would be able to add tributes – online versions of the war memorials that remain a feature of just about every town, large or small.

While the exhibition and the records themselves provided the main drivers for the project, there was also a growing desire within the institution to explore some of the possibilities of Web 2.0 technologies. This desire was tempered somewhat by a range of familiar concerns centred on issues of authority and control. Would user contributions detract from the reliability of the records? Who would take responsibility for any errors in user-created content? Would the potential for abuse demand vigilant moderation? Mapping our Anzacs gave us a chance to start working through such issues.

Setting the stage

We had an idea, a budget and a launch date, what we needed was a plan. While in theory we had around six months to play with, the project had to be fitted in around the ongoing work of our small web team. On the content side we had one person cleaning up the data. At the technical end we had someone connecting up the various components and making it all work within the Archives’ web environment. In the middle there were two of us trying to marry content and technology and create a usable resource. While we had a range of useful skills, none of us had tackled a project quite like this. We all had to learn on the job.

With few models or examples to work from, we began to experiment – researching available technologies, throwing around possibilities. Our first efforts were largely focused on the map interface and before long we had a working prototype using javascript and Google Maps. But what we also needed was a better understanding of how users might interact with the site.

World War I Honour Roll at the Chiltern Atheneum Museum

We started from the idea of the online memorial – a list of names compiled by users that would be linked through to service records. Our example was a local historical society creating a site to commemorate their community’s war effort. But what if they had more information – photographs or family histories – how could this sort of material be incorporated? Further inspiration came from a visit to the local historical museum in the small Victorian town of Chiltern. On one wall was a typical roll of honour, listing the names of those who had served in the war. But underneath were framed portraits of many of those listed. They were people, not just records. Could we create something like this online?

There were some exciting possibilities emerging, but concerns remained. Would anybody actually want to contribute? Strong interest in family history and a growing community desire to commemorate the experience of World War I offered anecdotal support. We just had to ensure that this interest could be translated into engagement – that the barriers of participation were low enough to encourage visitors to become collaborators.

But what of concerns that such material might detract from the authority of the records, or open our institution up to liability? We needed to make it clear where public contributions began and archival data ended.

Welcoming, but separate; open, but managed – a tricky balancing act was required. The answer, we decided was to create a separate ‘scrapbook’ using the blogging service Tumblr. The ‘scrapbook’ label was intended to be encouraging – this was not a database, or formal register, it was a place to leave your thoughts, comments, information or memorabilia. This was reinforced by our terms of service which simply required contributions to be relevant and respectful.

Scrapbook post

A ‘scrapbook’ was also something quite different to a finding aid. The informality helped to make the boundary clear between record and response. The separation was physical as well as intellectual. While the scrapbook shared many of the design elements of the main site, it was hosted by Tumblr not the National Archives. By using the Tumblr API, however, it was easy to pass information between the two sites. We could also use the API to provide a basic moderation facility.

But this meant that an important part of the site’s functionality would be dependent on an outside service. To make sure we considered fully all the implications of this, we developed a risk analysis and contacted Tumblr staff to inform them of our plans. Our major concern was simply the continuity of the service. While there could be no guarantees, we judged that this risk was manageable. Tumblr staff were interested in the project and offered their assistance if necessary.

Results

Images from scrapbook posts viewed via a media RSS feed in CoolIris.

On 25 April each year, Anzac Day, Australians remember the sacrifices made in war. Over the Anzac Day weekend in 2009, we were astonished to receive more than 200 scrapbook posts. Of course we had expected an increase in use, particularly after the site was featured on the Australian version of the Today Show, but this remarkable response certainly confirmed the site’s success. In the six months since its launch there had been almost 94,000 visitors to Mapping our Anzacs. More than 1,000 scrapbook posts had been contributed and 280 tributes created.

But the greatest success was in the type of posts being contributed rather than their sheer volume. Our ‘scrapbook’ had proved to be just that – as well as photographs of service people and their families, there were pictures of medals, headstones, letters, newspaper clippings, pay books, identity disks, diaries, postcards and certificates. Some people simply commented ‘my grandfather’, while others wrote detailed accounts of family history. Perhaps most moving were those who took the opportunity to leave a message for their loved one: ‘You were the best dad’.

Scrapbook post

Some have taken a systematic approach. Our most frequent contributor is gradually attaching photographs of headstones and memorial plaques that she has gathered from local cemeteries. Others are posting their own contact details in the hope of linking up with family. Perhaps most interesting are the notes that provide links to other people or documents – to family members, for example, or to a later service record. These are helping build a rich web of contextual data. Equally valuable are the corrections and additions that are being offered by eagle-eyed users, pointing out transcription errors or helping us track down elusive locations.

The success of the scrapbook has somewhat overshadowed the tributes, or online memorials, which really provided our starting point. Many tributes have been created and, as we had hoped, schools and other groups are using them to document the impact of war on their local communities. However, some compromises at the implementation stage have meant that it is not as easy to build them as we had hoped. There has also been some confusion by users between the tributes and the scrapbook. This is one area of the site we certainly hope to improve.

Even though the digitised service records had been available online for sometime though our collection database, it’s clear that many people are discovering them for the first time through Mapping our Anzacs. It was ‘a stunning find for me and my siblings’ wrote one grateful user. The scrapbook has aided discovery, providing another way into the records. Indeed, with the addition of a MediaRSS feed for CoolIris, the scrapbook provides two new entry points – one of them a 3D wall of faces and families. By embedding the records in these new contexts and making them easier to find, Mapping our Anzacs has successfully garnered extra value from an existing asset.

The site has also been recognised by others for its successful use of Web 2.0 technologies. We were pleased to be joint winners of the Best Archives on the Web Award, and surprised to be cited by the Federal Minister for Finance in a speech launching a Gov 2.0 taskforce. Recognition such as this has helped strengthen the case for future innovation in the Web 2.0 sphere.

Challenges

Success brings its own problems. One of the main challenges has been simply managing the sheer volume of posts and feedback. This was particularly acute of course after the Anzac Day deluge. As a result we have had to consider ways of streamlining our processes.

The Tumblr API allows us to set the status of a new post as ‘private’. We can then examine the post using the Tumblr dashboard before making it public. This works well enough as a basic form of moderation, however, the dashboard is not really designed for this purpose and it takes several clicks to release each post to the world.

But while moderation takes considerable time, it requires little intellectual effort. Despite concerns about abuse, our contributors have caused us few dilemmas. The only significant questions that have arisen concern the re-use of materials from other sources. This has made consider whether pre-emptive moderation is necessary or appropriate.

While the site includes detailed help information, it’s clear from the feedback that there are certain aspects that continue to cause difficulty. This provides useful data on how the site might be improved, but it has also made us think about how we communicate with our users. At the moment the content we provide is fairly static – there is no way of informing visitors of recent updates, or developing quick guides to common problems. If we took a more active approach to communication we might be able to decrease the number of help requests, while building a greater sense of community.

Similarly, while we have been excited by the number of corrections submitted by users, we can now see ways in which we might have structured the feedback process to capture their corrections more easily and efficiently. For example, a ‘submit a correction’ link on each individual’s page could automatically capture the person’s details, saving both us and our contributors from potential confusion.

We have suffered through the expected number of software glitches, and have a growing list of things we’d like to improve or develop, but overall the experience has been much more rewarding than painful.

Lessons Learned

Perhaps the most valuable lessons revolve around trust. Having entered into the project uncertain of what to expect from public participation, we have found ourselves in an evolving, creative partnership. Our users have defined what the scrapbook is and have taken an active role in improving and developing the resource. Our trust has been repaid many times over, helping us build something that in many ways has exceeded our expectations.

Trust is also necessary in the support of new ideas. Mapping our Anzacs was a very different type of project for the National Archives, challenging ideas both of access and user engagement. By taking the risk we have not only gained valuable publicity and user support, we have opened up the realm of possibilities for future development.

In terms of technology, the project demonstrated the power of the mashup and the efficiencies that can be gained by using existing web services. Tumblr, Google Maps and their associated APIs gave us a kickstart that enabled us to do a lot with a little.

Next Steps

There are so many exciting possibilities! Obviously our first priority is to improve those areas of the site that continue to cause our users grief. There are a number of navigation and usability tweaks that should improve the overall experience. Similarly, we can now see ways in which we might streamline moderation and management processes.

We hope to build on the success of the scrapbook and tributes by enhancing and extending their functionality. Improved editing and creation tools could assist contributors while also enriching the web of connections they build. We might, for example, provide widgets that make it easier to link the records of family members or friends. Over time this could develop into a complex network of relationships, providing new means of finding and visualising the records. Similarly, there are ways in which we might reuse the existing content of the scrapbook posts to develop new modes of discovery.

We could also do more to feature the labours and passions of our contributors. We could give them the option of exposing a public profile that lists all of their scrapbook posts. This would help foster a sense of community while providing yet another means of exploring connections between records.

Recent developments in geospatial technology and mobile devices perhaps offer the most exciting possibilities. Our original aim was to give the World War I service records back to local communities, to imbue the records with a greater sense of context, locality and belonging. Perhaps we will have succeeded when a tourist exploring a small country town can press a button on their mobile phone to retrieve a list of service people born near their current location.

Perhaps they will take a photo of a name on the local war memorial and use it to automatically retrieve that person’s service record or create an online tribute.

Perhaps they will come across a headstone in the local cemetery and immediately upload a geocoded photograph to the Mapping our Anzacs scrapbook.

Instead of merely being markers on a map, the records will start to overlay and inform the very spaces in which we move. The stories they contain will become part of our journeys, the people they document will have found their way home.

Creating a mashup

2009 preprint version of interview originally published in Kate Theimer, Web 2.0 Tools and Strategies for Archives and Local History Collections, Neal-Schuman, New York, 2010. [Order here]

What made you interested in creating a mashup?

It really started with the records. We hold the records of more than 375,000 World War I service people, identified by their places of birth and enlistment. With war memorials in just about every town across Australia, the connection between local communities and the memory of war remains strong. So we wondered how we could we give the service people in our records back to their communities. Having played around with the Google Maps API the answer seemed obvious — find the places, put them on a map and let people explore the connections for themselves.

What information, tools and processes did you need to begin?

The main thing we needed was the confidence to experiment. The process seemed straightforward in principle: first we had to extract the data we needed from the file titles in our collection database, then we had to find the latitude and longitude of each of the place names we extracted, and finally we had to plot these coordinates on a map with links back to details of the service records themselves. Web services, such as those provided by Google Maps, had the potential to do much of the work for us, and we scoured online documentation, user forums and blog posts for hints. But there were many things we could not know until we actually started. How consistent was our data? How many of the places would we be able to find? How would we be able to display thousands of places at once?

Moving from file titles through to coordinates obviously required a lot of data manipulation and we used Perl for much of the grunt work. Because our data set was large and variations in spelling and formatting were often unpredictable (including 13 different spellings of ‘lieutenant’!), we often had to work by trial and error — seeing what results we obtained and then adjusting our processes accordingly.

Once we had a list of place names in a consistent format we could begin to find latitudes and longitudes through a process known as geocoding. Google’s geocoding service was an easy option: it was well documented, reasonably comprehensive and it worked! We fed it our place names through a Perl script and soon we had a list of coordinates. Of course, many places were not found or returned multiple results, but the basic principle was sound. Our places were no longer just names, but points in space — we could begin making maps.

How did you determine what to include?

What we were creating was an archival finding aid, but one which placed the people, their homes and their communities up front rather than the systems that control their records. By browsing from a map a user would be able to find the details of a loved one, read a digitised copy of their service record and then follow a link through to our collection database. These links provide crucial context about the records, but we realised that this project also gave us an opportunity to capture other contexts and meanings. Who were these people? What did they look like? What happened to them after the war? By adding an online ‘scrapbook’ we gave users the chance to enrich the resource by adding notes or photographs about individuals.

This meant we had to deal with three sets of interlinked data: geocoded places, details from our records, and scrapbook posts provided by the public. To bring these all together with limited resources we had to make clever use of what was already out there. Why create our own maps when all you needed to do was write a bit of Javascript to embed a Google Map? Why build our own scrapbook application when the blogging service Tumblr provides free accounts and a simple API to manipulate posts? While a substantial amount of custom scripting was required to glue everything together, much of the core functionality was provided by free web services, available to anyone.

What challenges did you face?

Perhaps the first challenge to overcome was that of imagination. It was difficult for people to understand what the project was until we had a prototype to show them.

The process of handling and cleaning the data at times threatened to overwhelm us. While the geocoding service got us to the point where we could make maps, it also left us with many place names that needed to be manually checked. Often this was the result of misspellings in the original data, or because places either no longer existed or had changed their names. This data cleanup consumed much effort and continues still, though now with the help of our users who regularly point out errors and inconsistencies.

Once we had our coordinates we had to display them on a map without killing anyone’s computer. Showing thousands of markers on a Google Map is a challenge to slower web browsers and can end up hindering navigation. By dividing up our maps, clustering markers and changing the way they were rendered, we managed to greatly improve performance while maintaining the browsing experience. Once again it was trial and error coupled with the advice of the online community that guided us through the roadblocks.

What kinds of positive results have you had? (And any negative ones?)

From the messages we receive it’s clear that Mapping our Anzacs allows people to find records they didn’t know existed. Some have met a great-great-uncle for the first time. Others have learned about the war experience of a much-loved grandparent. Local communities have embraced the project and the scrapbook has developed into a rich and often moving resource. We wanted to give users a new way to explore and interact with our collection, and it seems we have succeeded.

Our users have also become our collaborators, providing corrections and comments that help us improve our data. They have extended the idea of the scrapbook, using it, for example, as a noticeboard for family history research, or as a way of creating crosslinks between related resources.

Success brings problems of its own and the work of moderating the scrapbook and responding to feedback has proved considerable. Issues with performance remain for people on slow connections, and while many are familiar with the Google Maps interface, some find it difficult to navigate. We are planning a number of enhancements based on this feedback, and hope to take advantage of the technology as it evolves to improve and extend the interface.

About how much time did it take?

While the project as a whole stretched over about eight months, much of this time was taken up cleaning and processing the data. The development of the interface was completed in under two months.

What advice would you give an organization wanting to use something similar?

Start experimenting. The technology is developing so rapidly that if you spend 12 months planning a project it’s likely to be out-of-date even before you start. New web services and data sources are becoming available every day. Perhaps you could use Open Calais to extract people’s names from a collection description, or MetaCarta to find the places. You might use the Google Books API to harvest the details of publications that cite your records. Even if you’re not a coder you can use tools like Yahoo Pipes to see what happens when you start to link data and services. Experimentation brings new ideas and possibilities. It’s all about making connections.

This is the full version of a paper I presented at the National Digital Forum, 30 November 2011.

In 1901, one of the first acts of the Commonwealth of Australia was to create a system of exclusion and control designed to keep the newly-formed nation ‘white’. But White Australia was always a myth. As well as the Indigenous population, there were already many thousands of people classified as ‘non-white‘ living in Australia — most were Chinese, but there were also Japanese, Indians, Syrians and Indonesians.

Here are some of them…

The real face of White Australia

The administration of what became known as the White Australia Policy created a huge volume of records, much of which is still preserved within the National Archives of Australia. These photographs are attached to certificates that non-white residents needed to get back into the country if they decided to travel overseas. There are thousands upon thousands of these certificates in the Archives. Thousands of certificates representing thousands of lives — all monitored and controlled.

But is is too easy to see these people as the powerless victims of a repressive system. There were many acts of resistance. Some argued against the need to be identified ‘just like a criminal’. Others exercised control over their representation, submitting formal studio portraits instead of mug shots.

Most commonly and most powerfully, people resisted the policy simply by going ahead and living rich and productive lives.

My partner, Kate Bagnall, is helping to rewrite Australian-Chinese history by overthrowing the stereotype of the culturally isolated Chinese man living a lonely, meagre existence surrounded by gambling and opium dens. By mining the available records, by reading against the grain of contemporary reports and by working with family historians, Kate is documenting their intimate lives — their wives, their lovers, their families and descendants — the sorts of relationships that sent a shudder through the edifice of White Australia. Power can be reclaimed in many subtle and subversive ways.

‘The real face of White Australia’ is an experiment. It uses facial detection to technology to find and extract the photographs from digital copies of the original certificates made available through the National Archives of Australia’s collection database. The photographs you see here come from just one series, ST84/1. There’s no API to the collection so I reverse-engineered the web interface to create a script that would harvest the item metadata and download copies of all the digitised images. There are 2,756 files in this series. On the day I harvested the metadata, 347 of those files had been digitised, comprising 12,502 images. It took a few hours, but I just ran my script and soon I had a copy of all of this in my local database.

Then came the exciting part. Using a facial detection script I found through Google and an open source computer vision library, I started experimenting with ways of extracting the photos. After a few tweaks I had something that worked pretty well, so I pointed my aging laptop at the 12,502 images and watched anxiously as the CPU temperature rose and rose. It took a few emergency cooling measures, but the laptop survived and I had a folder containing 11,170 cropped images. About a third of these weren’t actually faces, but it was easy to manually remove the false positives, leaving 7,247 photos.

These photos. These people.

With my database fully primed and loaded it was just a matter of creating a simple web interface using Django for the backend and Isotope (a jQuery plugin) at the front. Both are open source projects. All together, from idea to interface, it took a bit more than a weekend to create, and most of that was waiting for the harvesting and facial detection scripts to complete. It would be silly to say it was easy, but I would say that it wasn’t hard.

What we ended up with was a new way of seeing and understanding the records — not as the remnants of bureaucratic processes, but as windows onto the lives of people. All the faces are linked to copies of the original certificates and back to the collection database of the National Archives. So this is also a finding aid. A finding aid that brings the people to the front.

According to Margaret Hedstrom the archival interface ‘is a site where power is negotiated and exercised’. Whether in a reading room or online, finding aids or collection databases are ‘neither neutral nor transparent’, but the product of ‘conscious design decisions’. We would like to think that this interface gives some power back to the people within the records. Their photographs challenge us to do something, to think something, to feel something. We cannot escape their discomfiting gaze.

But this interface represents another subtle shift in power. We could create it without any explicit assistance or involvement by the National Archives itself. Simply by putting part of the collection online, they provided us with the opportunity to develop a resource that both extends and critiques the existing collection database. Interfaces to cultural heritage collections are no longer controlled solely by cultural heritage institutions.

It’s these two aspects of the power of interfaces that I want to focus on today.

There are a growing number of examples where the records created by repressive or discriminatory regimes have, in Eric Ketelaar’s words, ‘become instruments of empowerment and liberation, salvation and freedom’. Nazi records of assets confiscated during the Holocaust have been used to inform processes of restitution and reparation. Government records have helped members of Australia’s Stolen Generations trace family members. Descendants of inmates incarcerated by American colonial authorities in what was the world’s largest leprosy colony in the Philippines, have embraced the administrative record as an affirmation of their own heritage and survival. Records can find new meanings. Power can be reclaimed.

Technology can help. Tim Hitchcock has described how something as simple as keyword searching can turn archives on their heads. Recordkeeping systems tend to reflect the structures and power relations of the organisations that create them. The ‘hierarchical and institutional nature of most archives’, Hitchcock argues, ‘contains an ideological component which is sucked in with every dust-filled breath’. But digitisation and keyword searching free us from having to follow the well-worn paths of institutional power. We can find people and follow their lives against the flow of bureaucratic convenience. We can gain a wholly new perspective on the workings of society. ‘What changes’, Hitchcock asks, ‘when we examine the world through the collected fragments of knowledge that we can recover about a single person, reorganised as a biographical narrative, rather than as part of an archival system?’

Projects such as Unknown no longer may help us answer that question.

Unknown no longer

It’s aiming to extract the names and biographical details of slaves from the 8 million manuscript documents held by the Virginia Historical Society. The documents include court records, receipts, wills and inventories. Here is a page from the ‘Inventory of Negroes at Berry Plain Plantation, King George County, Virginia’ for 1855, listing names, occupations and valuations.

Tim Hitchcock is one of the directors of London Lives a project that similarly seeks to find the people in 240,000 manuscript pages documenting the lives of plebeian Londoners in the 17th century.

London Lives

More than three million names have already been extracted from the records of courts, workhouses, hospitals and other institutions. Work is continuing to link these names together, to merge these various shards of identity and piece together the experiences of London’s poorest inhabitants.

Remember me from the US Holocaust Memorial Museum is working with photographs taken by relief agencies in the aftermath of World War Two. The photographs are of displaced children who survived the Holocaust but were separated from families. What happened to them? The project is seeking public help to identify and trace the children.

Remember me

These are all projects about finding people. Finding the oppressed, the vulnerable, the displaced, the marginalized and the poor and giving them their place in history. This is what Kate and I hope to do with Invisible Australians, the broader project of which our faces experiment is part.

Invisible Australians

‘Invisible Australians’ aims to extract more than just photographs. We want to record and aggregate the biographical data contained within the records of the White Australia Policy — to extract the data and rebuild identities.

But we want to do more, we want to link these identities up with with other records, with the research of family and local historians, with cemetery registers and family trees, with newspaper articles and databases we don’t even know about yet. We want to find people, families and communities.

It’s ridiculously ambitious and totally unfunded. But it is possible.

The most exciting part of online technology is the power it gives to people to pursue their passions. As with the faces, we don’t need the help of the National Archives. We need the records to be digitized, but that’s happening anyway and we can afford to be patient. Most of the tools we need already exist, and are free. In the past 12 months, for example, there have been a number of open source tools released for crowd-sourced transcription of manuscript records.

People with passions, people with dreams, people who are just annoyed and impatient, don’t have to wait for cultural institutions to create exactly what they need. They can take what’s on offer and change it.

Interfaces can be modified. It is amazingly easy to write a script that will change the way a web page looks and behaves in your browser. I was frustrated by the standard interface to digitized files in the National Archives of Australia’s Recordsearch database — so I changed it.

Before and after

Not only did make it look a bit nicer, I added new functions. My script lets you print a whole file or a range of pages and display the entire contents of the file on a pretty cool 3d wall.

I’ve shared this script, and a few other Recordsearch enhancements. Anyone can install them with a click and use them.

Wragge Labs Emporium

Interfaces are sites of power and we can claim some of that power for ourselves. Online technologies not only free us from the having to brave the physical intimidation of the reading room, they free us up to engage with the records in new ways. The archivist-on-duty would probably not be pleased if I pulled out some scissors and started snipping photos out of certificates. Or if I pulled a file apart and pasted it’s contents on the wall. But online we are free to experiment.

The power of cultural heritage organisations is perhaps expressed most forcefully in their ability to control the arrangement and description of their collections. ‘Every representation, every model of description, is biased’, note Verne Harris and Wendy Duff, ‘because it reflects a particular world-view and is constructed to meet specific purposes’. Archives, libraries and museums are already starting to share this power, by allowing tagging, or seeking public assistance with description through crowd sourcing projects. But most of the these activities still happen within spaces created and curated by the institutions themselves. Our cathedrals of culture might be opening their doors and inviting the public to participate in their ceremonies, but that doesn’t make them bazaars. The architecture stills speaks of authority.

In any case, people already have a space where they can explore and enrich collections — it’s called the internet.

It would be great to see cultural institutions doing more to watch, understand and support what people are doing with collections in their own spaces — following them as they pursue their passions, rather than thinking of ways to motivate them.

A quick example… You might have heard of Zotero, it’s an open source project that lets you capture, annotate and organize your research materials.

Zotero

One cool thing about Zotero is that you can build and contribute little screen scrapers, called translators, that let Zotero extract structured data from any old collection database. You might not be surprised to learn that I’ve created a translator for Recordsearch. Another cool thing about Zotero is that you can share the stuff that you collect in public groups.

Invisible Australians Zotero group

Put those two cool things together and what do you have? Well to me they spell out user generated finding aids — parallel collection databases created by researchers simply pursuing their own passions.

Linked Open Data greatly increases opportunities for collection description to leak into the wider web. If objects and documents are identified with a unique URL, then anyone can can make and publish statements about them in machine-readable form. These statements can then be aggregated and explored. Initiatives such as the Open Annotation Collaboration will hasten the development of these shared descriptive and interpretative layers around our cultural collections.

And of course all this descriptive and interpretative work can be harvested back to enhance existing collection databases. We could start doing it now — though I will spare you today my rant about the possibilities of mining footnotes.

As well as exploring the possibilities of user-generated content, cultural institutions are starting to open up their collection data for re-use. APIs are great (though Linked Open Data is better), and New Zealand is lucky to have an organisation like DigitalNZ which just gets it. People can and will make cool things with your stuff.

But again, we don’t have to wait for everything to be delivered in a convenient, machine-readable form. If it’s on the web anybody can scrape, harvest and experiment.

You probably all know about the National Library of Australia’s newspaper digitisation project — it’s building a magnificent resource. But I wanted to do more than just find articles. I wanted to explore and analyze their content on a large scale. So I built a screen scraper to extract structured data from search results, and then used the scraper to  power a series of tools. I have a harvester that lets you download an entire results set — hundreds or thousands of articles — with metadata neatly packaged for further analysis.

Harvester

Or what about a script that graphs the occurrence of search terms over time, and allows you to ask questions like When did the Great War become the First World War?.

When did the Great War become the First World War?

In the end I got a bit carried away and built my own public API to the Trove newspaper database.

Unofficial Trove newspapers API

I think it’s important to note that the tools I developed were guided by the types of questions I wanted to ask. While we should welcome APIs and celebrate their possibilities, we should also remain critical. APIs are interfaces, they too embed power relations. Every API has an argument. What questions do they let us ask? What questions do they prevent us from asking?

Even as we move from the age of lumbering, slow-witted data silos into the rapidly-evolving realms of Linked Open Data, we have to constantly question the models we make of the world. Ontologies and vocabularies are culturally determined and historically specific. Yes, they too are interfaces, complete with their own distributions of power and authority. But we can revisit and change them. And we can relate our new models to our old models, capturing complex, long-term shifts in the way we think about the world. That’s incredibly exciting.

All of this hacking, harvesting, questioning, enriching and meaning-making makes me think about the possibilities of grassroots leadership. Online technologies enable people to take cultural institutions into unexpected realms. They can build their own interfaces, ask their own questions, determine their own needs — they can point the way instead of simply waiting to be served.

You might wonder what the National Library of Australia thinks of my various scrapers and harvesters. I can’t speak for them, but I can say that they’ve awarded me a fellowship to explore further the possibilities of text-mining in their newspaper database.

The idea of grassroots leadership brings me back to the title of this talk — ‘It’s all about the stuff’. It seems to me that we tend to model the interactions between cultural institutions and the public as transactions. The public are ‘clients’, ‘patrons’, ‘users’ or ‘visitors’. But the sorts of things I’ve been talking about today give us a chance to put the collections themselves squarely at the centre of our thoughts and actions. Instead of concentrating on the relationship between the institution and the public, we can can focus on the relationship we both have with the collections.

It’s all about the stuff.

It’s all about the respect and responsibility we both have for our collections.

It’s all about the respect and responsibility we both have for people like this.

Entering the web of data

[view the presentation...] [view the triples...]

Keynote delivered at the annual conference of the Australia and New Zealand Society of Indexers, 14 September 2011.

This is me.

Today, Wednesday, 14 September 2011, I’m honoured to be able to join you here in the luxurious surrounds of the Brighton Savoy Hotel for the ‘Indexing See Change‘ conference. This is an event, a moment in history; we can pinpoint ourselves, this gathering, both in time and in space.

If we do that, if we move outside the moment and position ourselves on a timeline or a map, interesting things start to happen. Connections emerge.

Here we are at number 150, The Esplanade, in Brighton. A bit over a kilometre away is the stately villa, Kamesburgh. For many years Kamesburgh was also known as the Anzac Hostel — a refuge for permanently-incapacitated World War One veterans.

The Anzac Hostel opened on 5 July 1919. Here it is draped in its patriotic finery, from the collections of the Australian War Memorial. According to the caption, the Anzac Hostel was ‘a home, not an institute’.

Also amongst the War Memorial’s holdings is a wheeled bed that was used at the hostel. This particular bed was apparently occupied by one man, Albert Ward, for forty-three years.

Death notice for Alexander Kelley. Argus, 29 January 1944.

It was probably in a bed just like this that Alexander Dewar Kelley passed away on 27 January 1944. Alexander Kelley was cremated, and his remains interred amongst the roses at what is now called the Springvale Botanical Cemetery. Not far from my own grandparents.

Alexander Kelley spent close to half his life in the Anzac Hostel. Like many young men, he bravely answered his nation’s call to arms, but returned from war much changed. We can follow Alex’s war through his service record, easily-accessible through the website ‘Mapping Our Anzacs‘.

Alex was a coach painter who enlisted in the AIF in January 1916. Within a year he was in France. In May 1917 he suffered a gunshot wound to the head, but was able to rejoin his unit in August. Less than a month later though, he was wounded again, this time more severely. For Alex the war was over, and he was shipped back to Australia in May 1918.

‘Mapping Our Anzacs’ includes a scrapbook feature through which visitors to the site can attach notes or photographs to a service record. Amongst the the many thousands of postings is a fragment from a diary, found tucked inside the bible of Alexander Kelley’s mother. The diary entry reads simply: ‘Alex arrived from Front. Wet day. Saw him at “Caulfield”.’

Alex had survived and had returned to his family. This was a day to remember. But there was sadness too, for Alex was not the same young man who had left for the battlefields of Europe. In the diary fragment, ‘Caulfield’ is enclosed in inverted commas, indicating perhaps that the reunion took place, not in the suburb, but in the Caulfield rehabilitation hospital. Alexander Kelley was wounded in the face, hands and legs. He was left blind in both eyes and his right leg was amputated. He would live the remainder of his life a little over a kilometre away from here at the Anzac Hostel.

This is just one story. There are over 375,000 World War One service records held by the National Archives of Australia. How can we hope to understand a number like that? How can we hope to imagine the war’s impact on families, on communities?

‘Mapping Our Anzacs’ uses familiar Google maps to display the places of birth and enlistment recorded in many of those service records. But technical limitations make it impossible to display all the places at once. You can, however, take the same data and open it in Google Earth. If you then zoom in on Victoria, you see something like this.

Mapping Our Anzacs data viewed in Google Earth.

Each marker represents a place where a service person was born or enlisted. It’s impossible to read, of course, but that’s the point. There is so little blank space. As you zoom further, more markers appear, more place names resolve. It’s simple, but it’s powerful. They came from everywhere. From the smallest village to the biggest city; nowhere was untouched.

The ‘Mapping Our Anzacs’ scrapbook offers another perspective. It’s possible to extract the images posted to the scrapbook and present them on a 3D wall. Amidst an assortment of memorabilia, there are faces. Not places, or records — this is a wall of people.

Mapping Our Anzacs Scrapbook photos viewed through CoolIris

It’s worth noting too that like the markers on the maps, these faces link back to the actual service records. So they’re not just a new way of seeing the collection, they’re a new way of exploring it.

But the records don’t stand in isolation, they themselves have a context. A couple of years ago, Mitchell Whitelaw from the University of Canberra, undertook a project called ‘The Visible Archive‘ to investigate ways of visualising the holdings of the National Archives of Australia. Have you ever wondered what 360km worth of records looks like?

The collections of the NAA visualised by Mitchell's Series Browser.

This represents the holdings of the National Archives. Files within the archives are organised into series, and each square in this image represents a single series — there are about 60,000 of them. Naturally the size of the square gives an indication of the size of the series itself. It’s a fascinating and strangely beautiful picture.

It’s easy enough to pick out the World War One service records — Series B2455. In the interactive version of Mitchell’s series browser you can click on a box and display links between series, as well as other series created by the same government agency. Again, it’s not just a way of seeing the collection, but a means of exploring and interpreting it. As Mitchell says:

Visualisation enables us to literally show everything, to display large volumes of data in a way that reveals patterns and communicates context, but also provides access to the fine grain of individual elements.

But we can also employ such techniques to ask new kinds of questions. Can you imagine how Alexander Kelley and the other inhabitants of the Anzac Hostel must have felt in 1939? They had lost so much in the Great War, the ‘war to end all wars’, and yet within their own lifetime it was all happening again. More young men were answering the call, more lives were going to be destroyed.

There must have been a dreadful, disheartening moment when Australians realised that the Great War was not an end, but a beginning — the first in a series of devastating global conflicts. At some point the ‘Great War’ became the ‘First World War’, but when?

When did the 'Great War' become the 'First World War'?

What is perhaps more intriguing is the dramatic peak in the occurrence of ‘the great war’ in 1939. It’s no surprise that the looming threat of a new conflict would provoke comment and comparisons, but it does make you wonder about the context of those discussions and how they might have changed as the reality of war edged closer.

To start exploring this I’ve harvested the content of the 6,600 articles from 1939 that included the phrase ‘the great war’. Using an online text analysis service called VoyeurTools I can quickly generate a picture of their contents.

This simple visualisation shows us the relative frequencies of words within the articles. It doesn’t reveal any great mysteries, but it does suggest some possibilities for further prodding. The prevalence of ‘time’ and ‘new’, for example — might these help us understand the shift in perspective from one war to the next? We can follow this up by browsing the different contexts in which the words were used.

But what actually is it that we’re actually searching? We know that Trove includes newspapers from 1803 to 1954, but if we’re really going to analyse shifting words and ideas it’s important to have a clear picture of the sources of those words.

Something like this perhaps. This graph shows the holdings of the Trove newspaper database on 4 August 2011, organised by state. You can see, for example, that if you’re searching on a topic between the 1920s and 1940s you’re probably likely to get more results from Queensland than anywhere else.

So starting from our location here, today, we can make connections across time and space. We can pull back and look at the big picture, or dive in and examine the fabric of a single life. Through the web we can build and explore a rich and complex contextual network.

It’s an exciting time to be a cultural data hacker. We now have a growing range of tools and technologies available for extracting interesting data from a wide variety of sources, both structured and unstructured.

The ‘Visible Archive’ project started with well-structured data, courtesy of Peter Scott, the developer of the Series System — the descriptive framework used by many Australian archives. But we’re rarely so lucky.

Even when the data starts off in nicely-organised fields in a database there’s no guarantee that that’s how it’s going to be delivered to our web browser. In order to extract the data from my Trove graphs, for example, I had to write a little program called a ‘screen scraper‘ to identify and save the important metadata elements from the raw web page itself.

Where there are no subject keywords we can infer them using techniques such as topic modelling. Where there are no access points we can identify people, organisations, places and events using special tools developed for named entity extraction. Where there are no common identifiers across datasets we can employ record linkage technologies to find possible connections.

We can count words, we can identify parts of speech, we can formulate a measure of the similarity of any two pieces of text. Once we have some useful data we can manipulate and enrich it. Place names can be geolocated — you simply send your place name off to a web service and get back its latitude and longitude.

Increasingly these sorts of tools are becoming accessible to anyone. For historians they offer a means of wrestling with rapidly-growing bulk of source material that is becoming available in digital form. How do you make use of 5 million digitised books, 50 million newspaper articles or the complete archive of every public message ever sent on Twitter?

The digital historian Dan Cohen has noted:

These computational methods which allow us to find patterns, determine relationships, categorize documents, and extract information from massive corpuses, will form the basis for new tools for research in the humanities and other disciplines in the coming decade.

Dan is involved in a number of interesting projects investigating the possibilities of these techniques — often grouped together under the heading ‘text mining’. One of these projects, ‘With Criminal Intent‘, is looking to see what patterns can be drawn out of the digitised proceedings of criminal trials held at the Old Bailey from 1645 to 1913. That’s 197,745 trials, in case you were wondering.

Here’s one of their visualisations showing how the length of trials varies over time. Much to the surprise of the research team, this graph suggests a dramatic shift in legal practice around 1825 — defendants started pleading guilty!

A visualisation by the With Criminal Intent project showing changing trial lengths.

Rather than falter under the growing weight of digital sources, these technologies can actually thrive. The more raw material available, the more chance there is to observe and track new patterns. As digitisation continues apace will we ever reach the point when history can simply be read from a graph?

There are some researchers at Harvard who seem to think that’s where we’re heading. Borrowing liberally from the store of scientific metaphors they have staked out the new field of ‘culturomics‘. By mining massive digital resources, like Google’s scanned books, they hope to map the ‘cultural genome’ that would enable us to follow the evolution of language and culture.

But there’s something quite barren in this ambition. I prefer the vision of digital humanist Stephen Ramsay, who commented in regard to the ‘With Criminal Intent’ project:

The Old Bailey, like the Naked City, has eight million stories. Accessing those stories involves understanding trial length, numbers of instances of poisoning, and rates of bigamy. But being stories, they find their more salient expression in the weightier motifs of the human condition: justice, revenge, dishonor, loss, trial. This is what the humanities are about. This is the only reason for an historian to fire up Mathematica or for a student trained in French literature to get into Java.

Ultimately it’s the stories that nourish, anger, inspire and depress us. The closely-packed map of places recorded in World War I service records is so powerful because we know that under each marker are men, women, families, communities — each with their own story. These new technologies offer new perspectives, they raise new questions, and they challenge us with new contexts to explore and understand. But there is still space for stories and perhaps we can use them to give our stories new life and depth.

This is another World War One service record. It belongs to Charlie Allen. Charlie enlisted three times in the AIF and was discharged on medical grounds each time. It seems he had a problem with his ankle.

Charlie’s service record notes a tattoo, proclaiming his love for ‘Maud Gordon’. He married Maud in Sydney in 1917 and had two daughters soon after.

Charlie survived the war without further injury, but was not so lucky in peace. On 11 March 1938, Charlie was crushed to death between two railway cars. The accident happened at the Bunnerong Power Station, only a short distance from his home in Matraville. He was buried nearby in the Botany Cemetery.

We also know quite a bit about Charlie’s early life. Why? Because Charlie’s father was Chinese and he was therefore categorised as a ‘half-caste’, as someone who was not white, and therefore fell under the restrictions imposed by the White Australia Policy.

Charlie was born in Sydney in 1896. His mother was Frances Allen (sometime sweet shop owner and brothel keeper), his father Charlie Gum (a buyer for Wing On company). Charlie was raised by his mother, but in 1909, at the age of 13, he was taken to China by his father.

NAA: ST84/1, 1909/22/41-50

This certificate granted Charlie an exemption to the Dictation Test. Without it, he may not have been allowed back into the country.

Every time one of many thousands of non-Europeans resident in Australia sought to travel overseas and return home again they needed one of these certificates.

Charlie’s father returned to Sydney, leaving him in China. He lived with relatives in the town of Shekki (inland from Hong Kong). Charlie was naturally homesick, but had no means of getting back to Australia. He wrote to his mother in 1910:

Do try and bring me home every minute I think of you and long for a piece of bread and butter this tucker is not doing me well.

His mother wrote to the Prime Minister Billy Hughes in an attempt to enlist government help but to no avail. Charlie finally returned to Australia in 1915.

Despite this experience, Charlie visited China again in 1922 for 7 months. Once again carrying papers to grant him re-entry to the country of his birth.

These fragments of Charlie’s life have been assembled by my partner, Kate Bagnall, a historian of Chinese-Australia. They are remarkable, and yet not so, because there are many thousands of stories like Charlie’s contained within the voluminous records generated by the administration of the White Australia Policy.

We’re all of course familiar with the general outlines of the White Australia Policy, and the way it underpinned conceptions of Australia as a nation in the first half of the 20th century.

But what we sometimes forget is that it was also a massive bureaucratic exercise.

Forms and certificates were printed, issued, used and filed. Regulations were modified, guidelines were distributed and administering officers were managed and advised. Individual cases were reviewed, policy was changed and new forms and certificates were printed, issued, used and filed…

Much of this system is now preserved in the National Archives.

You can get a idea of the range of material available from a case study Kate has prepared focusing on the efforts of Poon Gooey, a successful businessman in Horsham, to keep his wife and family in Australia.

If we look again at Charlie’s certificate from 1909 we can see that it contains a lot of interesting structured data:

• name
• place of birth
• age
• height
• destination
• date of departure
• name of ship

We estimate that there are probably about 50,000 of these forms remaining in the Archives, and then there’s case files and a variety of other government documents.

Wouldn’t it be great if we could extract this structured data. If we could piece together the slivers of identity that remain within the Archives and give people back their lives.

This is the dream of Invisible Australians, a project Kate and I are trying to turn into a reality. Our aim is to build systems that will enable this data to be extracted, aggregated, shared and connected — whether to a family tree, a cemetery record, or another document in another archive.

Imagine being able to navigate the network of lives, families and relationships. To follow their journeys, to share their tragedies, to celebrate their small victories against a repressive system.

Imagine being able to watch them age.

We tend to assume that new technologies require us to change, to adapt. But sometimes they can take advantage of our strengths. Mitchell Whitelaw is interested in finding out what happens when you take large cultural datasets and try to ‘show everything’. Such an approach, he suggests, takes advantage of the raw processing power of computers, while giving us space to do what we’re good at — finding patterns, making connections, crafting meanings.

The History Wall tries to create a similar sort of space. The History Wall brings together material from a range of different sources — newspaper articles from Trove, biographies from the Australian Dictionary of Biography, records from a database of NSW convicts, population statistics, collection items from the National Museum of Australia — you can pretty much plug anything in as long as it has a date attached to it.

Irish History Wall

For a particular year, the Wall retrieves a random sample from the available sources, jumbles everything up and then throws it onto the screen. As a result, no two views of the Wall are ever quite the same. This is not a traditional exhibition. There is no curator controlling the content or designing the structure. It’s ephemeral, it’s serendipitous — instead of relying on an authorial voice to smooth over the gaps and transitions, it leaves open the cracks and allows new contexts to seep in and around each item.

As the pioneering digital historian Edward Ayers noted:

even isolated and inert pieces of evidence — a list, a letter, a map, a picture — can assume new and unimagined meanings when placed in juxtaposition with other fragments.

This is not an absence of narrative, but an opportunity for narration. Edward Ayers suggests that we’re actually quite comfortable filling in blanks and untwisting timelines:

Humans, presented with pieces of information about people, put things into the form of a story. They need not be simple stories, for we know how to deal with unexplained lapses of time, flashbacks, and overlapping narratives. We know how to imagine, infer, things happening at the same time in different places. Film and television train all of us at early ages to weave strands of narrative out of intentional (if carefully constructed) confusion and to take pleasure in that weaving.

And so I can show you a death notice, or a certificate and you will take those fragments, those isolated data points and you will construct a story — you will see the person behind them, you will imagine their life. It’s what we do. We’re good at it.

Computers on the other hand will just see data.

In her ode in praise of humanities data, digital humanist Amanda French wonders whether we always need to crunch our data into abstract, pliable forms:

What I wonder is whether instead we can begin with the data, or with a datum, and simply watch for what it may tell us, even if what it tells us is simply a story.

Yes we can. And we should teach computers how to do it as well. Not because we want them to take over. Not because they can necessarily do it faster or better. But because they can help us share, preserve and connect those stories.

Let’s think again about the array of documents that Kate has assembled to piece together the story of Charles Allen. How can you share this sort of material? Typically you’d ‘write it up’. You’d capture the story behind the data and commit it to words. The documents would then become evidence — points of connection between your text and the historical record.

So in order to share the meanings of these documents we remove them from the context of the person’s life and marshal them as allies to proclaim the authenticity of our rendering. Wouldn’t it be better if we could tell the story, but maintain within our texts the direct connections between sources and subject?

What we need is a data framework that sits beneath the text, identifying people, dates and places, and defining relationships between them and our documentary sources. A framework that computers could understand and interpret, so that if they saw something they knew was a placename they could head off and look for other people associated with that place. Instead of just presenting our research we’d be creating a whole series of points of connection, discovery and aggregation.

Sounds a bit far-fetched? Well it’s not. We have it already — it’s called the Semantic Web.

The Semantic Web exposes the structures that are implicit in our web pages and our texts in ways that computers can understand. The Linked Data movement takes the basic ideas of the Semantic Web and turns them into a collaborative activity. You share vocabularies, so that other people (and computers) know when you’re talking about the same sorts of things. You share identifiers, so that other people (and computers) know that you’re talking about a specific person, place, object or whatever.

Linked Data is Storytelling 101 for computers. It doesn’t have the full richness, complexity and nuance that we invest in our narratives, but it does at least help computers to fit all the bits together in meaningful ways. And if we talk nice to them, then they can apply their newly-acquired interpretative skills to the things that they’re already good at — like searching, aggregating, or generating the sorts of big pictures that enable us to explore the contexts of our stories.

This is why we’ve always imagined Invisible Australians to be something more than an online database. We want to provide points of connection that other people can build into their own stories. But to do that we have to pay attention to things like vocabulary management and authority control, we have to construct web addresses that are not going to break every time we upgrade our software. We have to think about the sorts of things we’re talking about — not just people, but government agencies, legislation, certificates, and correspondence. How do we describe these entities and what sorts of relationships do they have?

And of course we need to expose all these structures so that we can say, these things are people, these are events, these are places and these are documents.

Or perhaps, to introduce Alexander Kelley.

Or remember Charles Allen.

You might be wondering why we don’t just leave it all to the computers themselves. Didn’t I just talk about all the exciting new tools and techniques that enable us to analyse the structures of texts? Perhaps we should just wait for the Culturomics guys to solve all the problems.

But who defines the problems?

Our postmodern sensibilities encourage a suspicion of neutrality. Labels like ‘the new museology’ or Archives 2.0 reflect an awareness that the way we describe and arrange our collections is itself culturally-determined. It’s not just a matter of what our descriptive systems show, but what they hide.

Tim Hitchcock, another member of the ‘With Criminal Intent’ team, has described how online technologies can change the way we access archives. Instead of being forced to navigate the hierarchical structures that archives impose on records, which in turn tend to reflect the workings of the institutions that created the records, we can directly find the people whose lives were regulated, influenced, shaped or controlled by the policies of those institutions.

Instead of merely hearing ‘the institutional voice… in all its stentorian splendour’, he says, we can listen in to ‘the quieter tones uttered by the individual’.

This reminds us that search boxes, along with other digital tools, themselves embody arguments. There are assumptions built into their code about what is relevant, what is significant, what is necessary.

We can build our own tools of course, and we can critique other people’s algorithms. But what if we just want to collect and share stories?

Linked Data gives us a way to present an alternative to Google’s version of the world. We can argue back against the search engines, defining our own criteria for relevance, and building our own discovery networks.

Changing the way we access resources changes the sorts of stories we can tell. Tim Hitchcock asks:

What happens when institutions and archives are ‘decentred’ in favour of the individual? What changes when we examine the world through the collected fragments of knowledge that we can recover about a single person, reorganised as a biographical narrative, rather than as part of an archival system?

Perhaps the invisible become visible.

Presented at the Life of Information Symposium, 24 September 2010.
Slides are available on Slideshare.

This palm print belongs to a 12-year-old boy called Charlie Allen.

Charlie was born in Sydney in 1896.

His mother was Frances Allen (sometime sweet shop owner and brothel keeper), his father Charlie Gum (a buyer for Wing On company).

Charlie was raised by his mother, but in 1909, at the age of 13, he was taken to China by his father.

His father returned to Sydney, leaving Charlie in China. He lived with relatives in the town of Shekki (inland from Hong Kong) for 6 years.

Charlie was homesick, but had no means of getting back to Australia. His mother attempted to enlist government help but to no avail. Charlie finally returned in 1915.

The following year he enlisted in First AIF (well actually he enlisted three times, and was discharged as medically unfit each time).

Charlie married in Sydney in 1917 and had two daughters soon after. He returned to China in 1922 for 7 months.

Charlie Allen died in 1938 as the result of an industrial accident. He was 41.

How do we know all this about Charlie Allen?

We know this because there are fragments of Charlie’s life scattered throughout the holdings of the National Archives of Australia.

The CEDT from 1909 when he left Australia with his father:

NAA: ST84/1, 1909/22/41-50

NAA: A1, 1911/13854

NAA: B2455, ALLEN C A

NAA: SP42/1, C1922/4449

Charlie’s father was Chinese, he was therefore categorised as a ‘half-caste’, as someone who was not white, and fell under the restrictions imposed by the White Australia Policy.

The certificate from 1909 granted Charlie an exemption to the Dictation Test. Without it, he may not have been allowed back into the country.

Every time one of many thousands of non-Europeans resident in Australia sought to travel overseas and return home again they needed one of these certificates.

We’re all of course familiar with the general outlines of the White Australia Policy, and the way it underpinned conceptions of Australia as a nation in the first half of the 20th century.

But what we sometimes forget is that it was also a massive bureaucratic exercise.

Forms and certificates were printed, issued, used and filed. Regulations were modified, guidelines were distributed and administering officers were managed and advised. Individual cases were reviewed, policy was changed and new forms and certificates were printed, issued, used and filed…

For example, between 1901 and 1911, 400 circulars were issued to port officers about immigration restriction. The confidential manual on immigration restriction grew from one page in 1902 to more than 200 in 1912.

Much of this system is now preserved in the National Archives.

For the years between 1902 and 1948 there remain:

• More than 50,000 CEDTs
• 90 shelf metres of records
• 15,000 case files

And within those many thousands of files are the scattered fragments of lives such as Charlie’s — lives that were controlled, monitored and documented in a vain attempt to make Australia ‘white’.

We’ve already seen today some wonderful examples of how these fragments, these slivers of existence, can be found, extracted, aggregated and displayed. But I think it’s worth considering for a moment what happens when we do this.

The historian Tim Hitchcock, behind projects such as the Old Bailey Online and London Lives, has reflected on the impact of digitisation on our access to archives. Archives, he notes, tend to reflect the assumptions and practices of the institutions that created them.

But by providing new ways into these records systems, technology can undermine the power relations that persist within their structures.

‘What changes’, asks Tim Hitchcock, ‘when we examine the world through the collected fragments of knowledge that we can recover about a single person, reorganised as a biographical narrative, rather than as part of an archival system?’

I don’t know, but I think we should find out, don’t you?

**********

I hope you’ve all collected a mini card. These themselves provide a little glimpse at the real face of White Australia and I’d invite you all to head over to the National Archives website, do battle with the monster that is RecordSearch, and look up the file references that are on each card.

The cards are part of a project that Kate Bagnall and I are trying to develop — Invisible Australians.

I should note too that the cards, and most of the examples I’m showing you here today are the product of Kate’s long and detailed research into Chinese-Australian families. In modern project management parlance, Kate is the domain expert, while I am merely the technical resource.

If we look again at one of the CEDTs, we can see that there’s a lot of useful structured data:

• name
• place of birth
• age
• height
• destination
• date of departure
• name of ship

Invisible Australians has the modest aim of extracting this data from the 50,000+ forms in the National Archives. But of course that’s just the start, because each person might have used a number of certificates — so then it’s a matter of matching these identities.

http://invisibleaustralians.org

And then there are a range of other related forms, not to mention case files, alien registration documents, naturalisation applications…

Obviously we can’t do it alone. We’ll be creating a crowdsourcing tool to extract and link the data.

It’s ridiculously ambitious, totally unfunded and is likely to take over our lives.

Is it worth it?

Imagine being able to navigate the network of lives, families and relationships. To follow their journeys, to share their tragedies, to celebrate their small victories against a repressive system.

Imagine being able to watch them age.

Is it worth it? We think so.

**********

For Tim Hitchcock technology opens up the possibility of writing a new history from below, exploring how the poor, the marginalised and the powerless navigated the institutions of the modern state. But it’s not just about search engines and databases. He talks about making ‘best use of the technology of emotions and representation — how you use words and pictures and a story to impact, not just on what people think, but what they see in their mind’s eye’.

In this project, the photos matter. I hope the irony in our project title is obvious.

This is the real face of White Australia.

The photos remind us that the project is not just about shifting data around — these are lives, these are people.

But this brings its own challenge, for if we are seeking to liberate these lives from the fragmentation and obscurity of bureaucratic systems then we should be asking what are we liberating them into?

A database?

This is not just an exercise in data creation and management. We also have to think carefully and creatively about issues of representation, access and discovery.

We have to give these lives back their freedom to associate, to have relationships, to make connections.

We need to embed these lives in a variety of contexts and combinations. To make room for serendipity, celebration, sadness, and yes, even play.

We need to bring these lives into a rich and ongoing conversation with the world.

But how?

**********

I’ve been working on a little experiment for the National Museum of Australia called The History Wall. What the History Wall does is quite simple, it pulls together data on the fly from a variety of sources including People Australia, the Australian Dictionary of Biography, the National Library’s newspapers project, historical population data from the Bureau of Statistics, photos from the Flickr accounts of the PowerHouse Museum and the National Archives, and the collection database of the National Museum itself. It chooses randomly from all this stuff, throws the results up into the air and then displays them however they happen to fall. No two views are ever quite the same.

http://defining.net.au/wall/

It’s something more than a timeline. To me it’s more like a celebration of context and serendipity. There’s a richness to it, a sense of discovery and fun, but there’s also fragility — next time you look it might be gone.

It’s a bit like history itself.

It’s a bit like the world.

How do we create spaces for our data to merge and mingle? How do we encourage the development of new contexts and connections?

I think the first thing we have to do is stop thinking about databases and dictionaries, registers and encyclopaedias. Don’t get me wrong, I’m not being critical of the wonderful projects we’ve seen today. I just think we can use all this work better if we stop thinking about individual resources and start developing on a web scale, on a global scale.

Yes, we have the technology. Time today has spared you from a detailed discourse on the Semantic Web, but I do want to focus on one aspect.

You may have heard of Linked Data, it’s a set of guidelines to help you publish your data to the Semantic Web. There are only four basic principles and I’m only going to talk about one of them. It’s one of those deceptively simple things. You look at it and think, ‘yeah, ok’, but before too long it’s starting to turn your brain inside out.

Use URLs to identify things in the real world.

Yeah, ok…

You know what URLs are, web addresses, the things you type in your browser’s location field.

And hopefully you know what things in the real world are: people, places, objects, events, ideas…

Now you may have detected a problem here, because no matter how many times you click the refresh button, your web browser is not going to be able to use such a URL to magically deliver you the real world thing.

Well, unless you’re on eBay.

Fortunately, the Linked Data guidelines provide for a bit of technical trickery that allow your browser to retrieve not the real world thing, but some information about that thing — perhaps in the form of a web page.

Why bother?

Names are powerful.

We share and use names to talk about things. Computers are the same. If we use URLs to identify things in the real world, then computers can start talking about them.

We can define and explore real-world relationships in an online environment. We can create rich, meaningful linkages across databases, across disciplines, across the world.

We can start building and thinking on a web scale.

**********

Thanks to the People Australia project, I can confidently claim that this is me:

http://nla.gov.au/nla.party-479364#foaf:Person

I keep meaning to get it on a t-shirt.

The most exciting thing about People Australia is not the EAC records or the aggregation of resources — it’s the identifiers, because they enable us to say things about people anywhere on the web that computers can understand and relate back to a specific real world entity — a person.

You can start doing it now with Wragge’s Identity Browser.

http://wraggelabs.com/identities/

This is a little tool I built using the People Australia API. It makes it easy to find identifiers for people and organisations, and it supplies you with some code that you can drop into a blog post or web page that will tell a computer that a name relates to a thing called a ‘person’ , that this person’s name has a certain standard form, and that this person can be uniquely identified by People Australia.

Even if you don’t publish a website or a blog, you can use People Australia identifiers to build semantic linkages. Wragge’s Identity Browser also creates machine tags for you. Machine tags are like normal tags but with built in semantics. When coupled with identifiers they enable you to do some pretty powerful things.

You could for example use machine tags in Flickr to tell computers that a certain photo depicts a person uniquely identified by People Australia. In fact, people have been doing just that.

http://wraggelabs.com/fmtc/

The Flickr Machine Tag Challenge is a sort of scoreboard that I built to encourage people to start adding People Australia enriched machine tags to photos. More than 1200 tags have been added to over 1000 photos. Feel free to join in!

The point is that the technologies already exist to enable us to build web scale biographical resources. Not dictionaries or databases as we know them, but networks capable of constant expansion, elaboration, and cooperation.

What we need are more tools to make it simple, recipes to make it obvious, examples and applications to make it popular, and leadership to make it all seem possible.

**********

Of course most of the lives we hope to liberate through Invisible Australians will not be represented in People Australia.

Not yet.

But Invisible Australians will offer a point of aggregation and disambiguation that will enable our people to find their way from the bureaucratic recesses of the White Australia Policy to a place on the national stage.

And we will encourage others to do likewise. Basil can’t do all the work. The centralised system has to be fed through centres of aggregation and collaboration.

Similarly, there are many great resources already out there relating to Chinese-Australians. There are hordes of family and local historians compiling and publishing biographical data. We want to identify people in these resources and link to them.

We want to publish up to People Australia and link down to a single headstone in a lonely country cemetery.

But to do this we need to help people make their resources linkable. To help them create persistent, re-usable URLs, and expose their data in standard formats. To create Linked Data, even if they have no particular interest in the Semantic Web.

http://invisibleaustralians.org/

Invisible Australians is not just about extracting data from archives. It’s also about working with others to build capacities and demonstrate possibilities.

It’s ridiculously ambitious, totally unfunded and is likely to take over our lives.

Is it worth it?

We think so.

Some of you may have noticed that my Hacking a research project post featured a file from the National Archives of Australia embedded as a Cooliris widget. Huh? To jog your memory, here it is again:

These certificates allowed non-white Australians travelling overseas to re-enter the country. NAA: ST84/1, 1906/21-30

No, it’s not just an image, it’s a little 3D wall. You can pan and zoom to your heart’s content. You can enlarge an image, view fullscreen — you can even share an image via Twitter. Fun for all the family!

Regular viewers will recall my previous encounters with CoolIris — Archives in 3D and CoolIris enabled scrapbook — but these relied on having the CoolIris plugin installed. The embeddable Flash version wouldn’t work when the images were coming from the NAA because it upset Flash’s cross-domain settings.

So how did I get it to work? For various other projects I’ve been playing with simple image proxies using Python and Django, so I just applied the same principles. The image proxy makes it seem as if the images are coming from a local source, thus keeping Flash happy. Hurrah!

I’ve added a few little tweaks, so you can now view any digitised file in the National Archives of Australia in a CoolIris wall. Just go the the file browser page and enter a barcode. Even better you can install a bookmarklet. Just drag this link to your bookmarks bar (or save as a favourite) — View on wall. Then go to an item page in RecordSearch and click on the bookmarklet for 3D magic.

If you want to share a link to a file displayed in the 3D file browser, just use a url of the form:

http://wraggelabs.com/recordsearch/wall/[barcode]

— where [barcode] is fairly obviously the barcode of the file you want to view. For example:

• http://wraggelabs.com/recordsearch/wall/3445411/

If you want to embed one of the mini-walls in your blog post it’s easy. Just go to the CoolIris Express site and create your own wall. When it asks you for content source, click on ‘Media RSS’ and then in the ‘Feed URL’ box put:

http://wraggelabs.com/recordsearch/rss/[barcode]

– where [barcode] is… well, you know…

I think this a pretty interesting way to view, browse and navigate digitised files. Using Flash, rather than a browser plugin makes it more accessible, but I’d still rather have something based on open software and standards. I think it won’t be too long before we see something similar using Canvas and Javascript. That’ll be really exciting.

Amongst the holdings of the National Archives of Australia are some of the most visually arresting documents you’ll see — thousands and thousands of forms from the early decades of the twentieth century, each with a portrait photograph and palm print, each documenting the movements of a non-white resident. Along with many other certificates, regulations, correspondence and case files, these forms are part of the massive bureaucratic legacy of the White Australia Policy.

These certificates allowed non-white Australians travelling overseas to re-enter the country. NAA: ST84/1, 1906/21-30

But these are more than just interesting looking pieces of paper, they are snapshots of people’s lives. The forms capture data about an individual’s place of birth, physical characteristics and more. Over time a person might have submitted several of these forms, so by bringing them together we could trace their history, we could map their journeys — we could even watch them age.

The system which sought to render non-whites invisible has captured and preserved the outlines of their lives. By extracting and linking this data we could build a picture of another Australia, an Australia in which non-white residents lived, loved, struggled and succeeded, despite the impositions of a repressive regime.

I talked about these records at the AAHC conference last year, inspired in part by Tim Hitchcock’s chapter in the Virtual Representation of the Past. Tim Hitchcock argues that technology can allow us to restructure archives, looking beyond institutional hierarchies to the lives of individuals contained within:

What changes when we examine the world through the collected fragments of knowledge that we can recover about a single person, reorganised as a biographical narrative, rather than as part of an archival system?

I don’t know, but I’d like to find out.

During my AAHC talk, Dave Lester suggested that the extraction of data from these forms might make a good crowdsourcing project. It’s a great idea. As you can see, the data is generally well-structured and legible, it should be possible to construct a simple series of forms that would allow volunteers to transcribe the data. The next stage would be to try and match identities across forms. That’s more complicated, but projects such as Tim Hitchcock’s London Lives show how users can construct identities by connecting a range of historical documents.

Then there are connections to resources outside of the archives — photographs, local histories, newspapers, genealogies, cemetery registers and more. By keeping our system open and extensible, and by working with others to help them expose their information in standard ways, it should be possible to develop the framework for an evolving mesh of biographical data.

So, how do we get started? This is the point when you usually have to start thinking about money — how can I fund this? In Australia that generally means a journey into the arcane world of the Australian Research Council. The ARC suffers from all the problems of a peer-reviewed system, but added to this is a rather antiquated notion of what research is.

In the rules covering each of the main schemes it’s clearly stated that the ‘compilation of data’ and the ‘development of research aids or tools’ are not supported. I spend part of my life working for the Australian National Data Service, an organisation that seeks to highlight how the sharing and reuse of data can open up new research possibilities. The ARC, however, seems to think that data has little value beyond its original research context.

Of course you can still mount a case for such activities. Applicants for a ‘Discovery’ grant can argue that data creation is integral to their project and provide details of the ‘specific research questions to be addressed’. But what if you don’t yet know what the questions are? Part of the point of a project such as this is to try and find out what questions we are able to ask. Until we start to compile, link and explore the data, the ‘specific research questions’ will be little more than convenient fictions, dreamt up to satisfy the prodding of peer reviewers.

Tom Scheinfeldt wrote a fantastic blog post recently, responding to concerns about the failure of many digital humanities projects to make arguments or answer questions. Drawing examples from the history of science, Tom argues:

we need to make room for both kinds of digital humanities, the kind that seeks to make arguments and answer questions now and the kind that builds tools and resources with questions in mind, but only in the back of its mind and only for later. We need time to experiment and even… time to play.

The ARC does not fund play.

You might imagine that the ARC’s infrastructure funding scheme would offer more hope for a project such as this. And yes, there are many worthy projects involving databases and online tools that have been supported in this way (and I have benefited from some of them!). But it seems that in the minds of research funders infrastructure is always BIG. Grants start at $150,000, and applications are expected to involve multiple institutional partners. Projects have to be scaled up to fit the ARC’s definition of infrastructure, often resulting in complex, lumbering, long-term projects whose products are out of date by the time of their release.

There is no room in our current infrastructure models for agile, innovative, user-focused digital toolmakers seeking small amounts to experiment with apps, prototypes, datasets or visualisations. I often look with envy upon the US National Endowment for the Humanities Digital Humanities Start-Up Grants.

In any case, neither I nor my partner in this endeavour, Kate Bagnall (@baibi), are currently in academic positions, so our chances of gaining any sort of research funding are next to none. We have the expertise — Kate has spent many years researching Australian-Chinese families and knows the records back-to-front, while I just can’t help playing with biographical data — but is that enough? How can you mount an ongoing research project without institutional support, research funding and the various badges and signifiers of academic authority?

I don’t know that either, but I have some ideas.

Mrs Ah Yin Pak Chong. NAA: ST84/1, 1907/321-330

I didn’t manage to get a contribution together for Dan Cohen and Tom Scheinfeldt’s crowdsourced-in-a-week book, Hacking the Academy, but watching the process from afar I did begin to wonder about how we might hack the way we build and run major research projects. This is what I have in mind:

• To strip down the large, lumbering beasts and design projects that are modular and opportunistic — able to grow quickly when resources allow, to bolt on related projects, to absorb existing tools.
• To follow the data freely across technological and institutional boundaries, developing open networks that invite participation and use.
• To develop a floating pool of collaborators, both inside and outside of academia, who are able to come and go, contributing whatever and whenever they can.
• To make everything public, accessible and standards-compliant, so that even if the project stalls it could be picked up and developed by someone else.

Most of all I just want to be able to do it. I don’t want to second-guess the ARC. I don’t want to spend months negotiating with potential partners or begging for an institutional home. I want to build, experiment and play. I want to make a start.

So that’s what we’re going to do.

We have a topic, plenty of raw materials, some basic principles and the beginnings of a plan. We even have a name — Invisible Australians: Living under the White Australia Policy.

As the project develops, I’ll be blogging here about some of the technical stuff, while Kate will be exploring the content over at the tiger’s mouth. I hope to have a prototype of the transcription tool ready to demo at THATCamp Canberra, while Kate is already at work putting together guides on using the records and developing an Omeka site that follows a number of Chinese-Australian families through the archives.

Can we hack together a major research project? Let’s find out.

All dressed up – RecordSearch has a new look

The new version of my Greasemonkey userscript, RecordSearch Image Tools, gives RecordSearch’s digital image pages a rather new look. My previous version had done away with the tired ol ‘lemon-chiffon’ background colour, but I decided it was time to get a bit more adventurous, so I blitzed the old design and rebuilt the page from the beginning.

As you can see from the screenshot, I’ve tried to give the images as much as the screen as possible. I’ve also created a consistent set of navigation buttons, and improved the functionality in various ways.

Archives in 3D – CEDTs from NAA: ST84/1, 1906/21-30

But the most exciting thing is that I’ve worked out how to feed the images to the fabulous CoolIris 3D wall. My previous version used the javascript version of CoolIris, which displayed the images as a flat (but still very nice) slideshow. But now, if you have the CoolIris plugin installed you can zoom, pan, fly through the file, dipping in and out as you so desire. It’s a new way of looking at archives.

You can zoom in and out, even see a complete file on a single screen – NAA: B2455, WRAGGE C L E

To try for yourself you need to have Firefox with the Cooliris plugin installed. Then you need to get the Greasemonkey extension and, finally, install my userscript. Then just dive into RecordSearch, find a digitised file and enjoy!

File links:

• NAA: ST84/1, 1906/21-30
• NAA: B2455, WRAGGE C L E

Australia is blessed, it seems, with a frightening abundance of treasures. A quick survey of our cultural institutions reveals an escalating ‘treasures race’, as libraries, museums, and archives bombard the public with accounts of their rarest, most beautiful, and most interesting items. The State Library of Victoria, for example, has published a lavish description of its ‘treasures’, and features them prominently on its redesigned website. The National Library of Australia also has an online display of its most treasured holdings, hoping to bring in sponsorship for a permanent ‘treasures gallery’. Meanwhile, the ‘Treasures Gallery’ at the National Archives of Australia is already up and running, while the South Australian Museum guides visitors around a ‘treasures trail’. The Australian Museum recently presented their ‘treasures’ in a special exhibition, and even the University of Melbourne has catalogued the highlights of its collections in a glossy book of ‘treasures’. Celebrating its 150^th birthday, the Museum of Victoria has made an impressive entry into the fray, with a well-designed treasures website, a treasures trail for visitors, and a beautiful volume simply entitled Treasures of the Museum.

All this treasure hunting might be seen a sign of conservative pressure upon our cultural institutions. Instead of seeking to enlarge their roles as places of research, interpretation and debate, it seems safer to fall back on familiar stereotypes of vaults and storehouses, keeping safe the nation’s heritage. Treasures are, by their nature, precious things — often protected by ‘lock and key’, ‘hidden’ from casual scrutiny. It is a label that promotes difference over familiarity, a feeling of reverence and awe over our ability to engage and connect. On the other hand, in an age of interactive exhibits and virtual museums, it is rather reassuring to realise that objects still provide such a potent source of meaning. Institutions trade on their treasures because we crave the experience of authenticity. We want the feeling of wonder, recognition and surprise that only comes from getting up close to the ‘real thing’.

In any case, cultural ‘treasures’ can be identified in a nuanced and reflective way, as Treasures of the Museum well demonstrates. Some of the objects it describes are beautiful, some are perplexing, some are funny, some are horrifying. Seemingly commonplace items are revealed as amongst the most challenging and evocative, such as the red vinyl suitcase with which Cuc Lam fled Vietnam in 1978. That most treasured of treasures, Phar Lap, is included of course. However, ‘the most famous quadruped in Australia’ is introduced by a guest contributor, Phillip Adams, whose recollection of his own childhood fascination, of ‘nose prints on the glass case’, focuses attention not on the object but on our own experiences and memories. The majority of entries, illustrated by a magnificent series of photographs, are by museum staff, and vary in quality and tone. Some are merely descriptive, others offer intriguing fragments of larger stories. There is much pleasure and interest to be gained from repeated dipping and browsing.

Whilst no doubt wishing to claim its own share of treasures, the National Museum of Australia chooses to cast itself not as a repository, but as a ‘storyteller’. Seeking to interpret the ‘national story’ is a brave undertaking, as evidenced by the criticism that has dogged the museum since its opening. Land Nation People: Stories from the National Museum of Australia is a determined restatement of the museum’s commitment ‘to telling the stories of Australia and Australians, and debating the key issues, events and people that have shaped and influenced our nation’. The book provides a condensed version of the museum itself, presenting major themes and selected objects from each of its exhibition areas: ‘First Australians’, ‘Horizons’, ‘Nation’, and ‘Tangled destinies’. With the exhibitions set to change in response to a review foisted on the museum by its critics, the book is an interesting historical document in itself. While the Museum of Victoria celebrates its long and illustrious past, the National Museum of Australia seeks to record the ambitions and achievements of its first few, turbulent years.

By unashamedly drawing attention to the process and practice of storytelling, the National Museum challenges curators, historians, and visitors to face up to the difficulties of narrative. With conservative commentators calling for the reinstitution of grand narratives of Australia’s progress all the way from Cook to cricket, there needs to be greater acceptance that the crafting of engaging and insightful stories from the complexities and contradictions of the past is hard, skilled, and creative work. There are no easy answers.

That said, there is nothing particularly innovative about the storytelling in Land Nation People. The stories are colourful and interesting, though rarely surprising, the themes are important, and like the Treasures of the Museum, the book assembles an intriguing collection of objects and illustrations. Indeed, despite the possible tension between ‘treasures’ and ‘stories’, there is much in the two books that is similar. The organisation of Treasures of the Museum also reflects the institution’s current structure, with the treasures divided into their respective collection areas of ‘Australian Society and Technology’, ‘Indigenous Cultures’, and ‘Sciences’. This is uninspired and unfortunate: Weary Dunlop’s medical instruments, for example, are uncomfortably tacked on to the end of ‘Sciences’, while the anthropological collections of Baldwin Spencer and Donald Thompson, featured in ‘Indigenous cultures’, are separated from their collectors who are locked up in the ‘Sciences’ section. If you are going to take a ‘treasures’ approach, why impose disciplinary boundaries at all? Interestingly, the companion website offers an alternative structure, grouping objects under such headings as ‘Celebrity’, ‘Messages’, ‘Journeys’, and ‘Survivors’.

More importantly, of course, both books are concerned with the relationship between object and story. ‘Museum objects’, remarks the Museum of Victoria’s CEO, ‘are like comets travelling through time and space, trailing streams of meanings’. Both books seek ways of making these meanings visible, and in doing so they reveal connections, contrasts and queries. This process is more explicit in Land Nation People, but Treasures of the Museum traverses much the same thematic territory, relying on assemblage instead of argument to explore the broader significance of its objects. The experience of immigration and arrival feature prominently in both, as does the complexity of indigenous culture. Both also seek to document ways in which we have come to know and understand the continent.

The ‘Tangled Destinies’ section of Land Nation People is most obviously concerned with the interaction of people and environment, but examples of change and adaptation are spread across both volumes. One has the story of William Farrer and his ‘Federation’ wheat, the other counters with the stump-jump plough. The sophistication of indigenous technology, and the ability of indigenous people to adapt to environmental and cultural change are well demonstrated. Both books feature a display of ‘Kimberley points’—spearheads crafted not just from traditional materials, but also from ceramics and glass.

Gesturing towards the supposed inventive streak within the Australian character, Land Nation People introduces two of the best known—and perhaps most overrated—Australian inventions under the banner of ‘Nation’. Yes, where would we be without the Victa mower and the Hills hoist? Treasures of the Museum takes us into less familiar realms with the black box flight recorder and the Shephard micro-ruling engine, which, in the late nineteenth century, pushed the limits of precision measurement. Technologies of measurement and control appear in a variety of guises, reflecting the desire of European settlers to define the limits and boundaries of their new possession. Artefacts from the geodetic survey of Victoria can be contrasted with the Anton Breinl’s hot-air cabinet, used to study the effects of the tropical climate on white workers—both speak to questions of possession and legitimacy. The clock used to maintain standard time throughout Victoria seems to have little in common with the field trowel used by archaeologist John Mulvaney. But both sought to redefine our conception of time: one brought local timekeeping practices within a centralised system, the other helped locate the human occupation of Australia within the immense span of deep time.

The natural sciences, of course, dominate the scientific collections of the Museum of Victoria, reflecting both the nature of the disciplines and the history of the institution. However, featured prominently amongst its ‘treasures’ are not just collections of birds, insects, minerals, and fossils, but the people who assembled them—the collectors themselves. John Gould, Alfred Russel Wallace, William Blandowski, and even Charles Darwin, make an appearance. This is an important inclusion, because it emphasises the process of collecting, the way in which scientific knowledge itself is constructed. Although most of the national science collections went elsewhere, the National Museum effectively uses the stories of Harry Burrell and Colin Mackenzie to similar ends. The lives and works of such individuals offers insight not just into the development of biology, but into the passion for collecting, understanding, and knowing, that motivates science in general.

While it is perhaps the historical and aesthetic dimensions of the scientific collections that make them most appealing, their continuing role in research is vitally important. Treasures of the Museum notes the scientific significance of the many type specimens within its collections, as well as the ongoing work of its staff to develop a cryogenic collection of tissue samples from rare and threatened species. Such a reminder that the collections themselves are living, growing things offers further complexity to the idea of ‘treasures’. Strangely, while the National Museum describes work to conserve and develop the National Historical Collection, there is little mention of its own research activities, particularly in environmental and indigenous history. Surely this too is a story worth telling.

This omission adds to the rather static feeling of Land Nation and People. As a snapshot of the museum, complete with obligatory corporate guff about its cutting edge multimedia technology and innovative architecture, the book seems to be more of a record of a visit­—a reminder or a souvenir—rather than something to be explored and enjoyed for its own sake. Treasures of the Museum, on the other hand, offers the twin pleasures of familiarity and surprise. Museum-tragics like myself, who spent happy days wandering amongst the old Swanston Street exhibitions, will discover many favourites amidst the ‘treasures’. One of the goldfield models is included, as well as the wax fruits and the working models case. At the same time, you have the sense that you are peeking behind the scenes, gaining access to wonders rarely seen in public.

The National Museum is committed to telling a diverse range of stories, but this worthy aim does not seem well-served by Land Nation People. The attempt to downsize the exhibitions for book consumption has taken away any feeling of exploration or uncertainty—it all seems a little too controlled. It is precisely this feeling of exploration that makes Treasures of the Museum so much fun to dip into. There’s more space here to imagine, wonder, and connect. There is much left unsaid, many questions unanswered, and the entries are frustratingly brief. But you are left with the feeling that there is much more to know, many more stories to tell, many more treasures to be revealed.

HISTORY OF SCIENCE in Australia is a field intimidated by its subject. Historians have been too slow to examine the local context of knowledge production and use, deferring to scientists and their uncritical catalogues of the past. Historical analysis has given way, too often, to the antiquarian plod or the celebratory frolic.

In nineteenth century Australia, ‘progress’ was a popular theme whenever the scientifically-inclined paused to reflect on their discipline. By the latter half of the century, science was developing an institutional base with the establishment of universities, societies and government posts. History helped locate this fledgling enterprise within a grand progressive tradition. Catalogues of scientific achievement provided a preface to current endeavours. Obituaries of local workers portrayed them, stuffed and mounted, within a gallery of revered pioneers; there were no Darwins or Newtons amongst them, but their lives were still dedicated to the same glorious ideals. The collecting of history parallelled the collecting of specimens – the main scientific activity within the colonies. Science progressed by the steady accumulation of plants, platypi and people.

By the end of the century, science was changing. Experimental science was beginning to displace natural history; professional scientists were edging amateurs out of the vanguard of progress. With more scientists, more societies, more publications and a never-ending parade of anniversaries to be commemorated, disciplinary and institutional studies became commonplace in the twentieth century – predominantly in scientific journals. As science fractured into increasingly narrow specialisations, history seeped in to fill the cracks, smoothing the narrative flow of scientific progress. Many of these historical efforts were of the ‘amateur-antiquarian’ variety identified by Michael Hoare (‘The History of Australian Science: Prospect and Retrospect’, Newsletter of the Australasian Association for the History and Philosophy of Science, no. 5, 1974), but some disciplines have been better-served. T.G. Vallance and David Branagan in geology, and R.W.Home in physics, have helped us to understand how disciplinary communities coalesced and changed (e.g. Vallance and Branagan, ‘The Earth Sciences: Searching for Geological Order’, The Commonwealth of Science: ANZAAS and the Scientific Enterprise in Australia 1888-1988, ed. Roy MacLeod, 1988; Home, ‘The Beginnings of an Australian Physics Community’, Scientific Colonialism: A Cross-Cultural Comparison, eds Nathan Reingold and Marc Rothenberg, 1987). Astronomy has been remarkably well-endowed, with detailed institutional histories, Beyond Southern Skies: Radio Astronomy and the Parkes Telescope (1992) and The Creation of the Anglo-Australian Observatory (1990), as well as a recent general work, Explorers of the Southern Sky: A History of Australian Astronomy (1996).

Significantly, much of this work has been undertaken by scientists. Despite the oft-heard protest that scientists look forward, not backward, scientific institutions have been major supporters of historical research in Australia. In 1962, the Australian Academy of Science established the Basser Library as a ‘centre for the study of the history of Australian science’. Ann Moyal and Michael Hoare used research positions within the Library to effectively pioneer the field, creating bibliographical resources and outlining many of the broad historiographical questions. The Academy continues to publish the only specialist journal in the field, Historical Records of Australian Science. Within this journal, however, scholarly historical articles are juxtaposed with biographical memoirs of deceased Academicians, written almost exclusively by Fellows. The memoirs are far-advanced beyond the taxidermic tributes of the nineteenth century, but the relationship between the articles and the memoirs is uneasy. Such biographies, as with many disciplinary histories, do not just comment on science past, they help to define science present. Scientists writing history are helping to establish, both within the scientific and broader communities, what makes a scientist and what counts as knowledge. In a powerful sense, the history of science is science. This nexus has been largely ignored, robbing the field of much analytical insight.

A lack of access to sources has sometimes been blamed for this historical inertia. But no more. The Australian Science Archives Project (ASAP) provides details of archival and published sources through its online database, Bright Sparcs (http://www.asap.unimelb.edu.au/bsparcs/). Few other fields of Australian history are so well served. In addition, some naturalists’ journals have been published, and collections of correspondence are appearing. A major project is under way to collect and publish the voluminous correspondence of eminent nineteenth century botanist, Ferdinand Von Mueller. Over the past few decades, tools such as these have been put in place to support an active research community, but where is the activity?

Biographies have appeared for major scientific figures, though there are notable omissions and inadequacies. The ADB has paid increasing attention to scientists over the years, and its articles remain the authoritative sources for many significant workers. Scientist-cum-explorers have fared rather better, as in E.M. Webster’s biography of Leichardt, Whirlwinds in the Plain (1980). The belated impact of women’s history has encouraged efforts to recover the names and deeds of women whose contributions to Australian science has been heretofore overlooked. However, there have been few attempts to move beyond this cataloguing process, to analyse the structures and assumptions that circumscribe participation in the scientific community, though some of the contributors to On the Edge of Discovery (1993) do raise such issues. Recent years have brought a mixed bag of scientific biographies with subjects including Macfarlane Burnet, Crosbie Morrison and Ian Clunies Ross, but none really manage to blend the scientist with the person. The dilemma is revealed most clearly in Lyndsay Gardiner’s biography, E.V. Keogh: Soldier, Scientist and Administrator (1990) – Keogh’s scientific work is dealt with separately in an appendix by a scientist. Scientists will remain divided figures as long as biographers are intimidated by science.

Science has been quarantined within Australian history – unfamiliar and forbidding territory for historians – best avoided. W.K. Hancock, however, thought differently when he emphasised the economic significance of William Farrer’s wheat breeding heroics (Australia, 1930). Ernest Scott likewise showed no qualms when he took it upon himself to lecture the Australian scientific community on their own history (‘The History of Australian Science’, Report of ANZAAS, 1939). In the 1950s and 1960s science seemed to provide healthy fodder for historians exploring the boundaries of their discipline. Geoffrey Blainey recognised its importance within economic history, Geoffrey Serle mapped its development alongside other cultural markers, while George Nadel and Michael Roe began to explore the meanings and uses of science within Australian cultural history. An exciting program of research was unfolding, and yet faltered. Why?

The 1960s and ’70s brought a loss of faith in the benevolent bounty of science. Perhaps a growing sense of suspicion served to alienate rather than inspire historians, reinforcing not revealing the mythical ‘two cultures’ divide. Significantly too, the same period saw the rapid growth of the history of science as a separate discipline. The interests of such historians generally lay in the grand themes of science, such as the Scientific Revolution. Their models and mentors were international, not local. When attention was finally turned towards the history of Australian science, it was within such an international framework; Australia became a case-study in the diffusion of scientific knowledge. International conferences in 1981 (Scientific Colonialism: A Cross-Cultural Comparison, 1987) and 1988 (International Science and National Scientific Identity, 1991) explored, in a cross-cultural context, the development of the colonial scientific community and its interaction with Europe. Analyses of ‘colonial science’ extensively modified the crude diffusionist models. Roy MacLeod, for example, in his ‘Visiting the Moving Metropolis’ (Historical Records of Australian Science, 5:3, 1982), offered an alternate taxonomy, that recognised the complex trajectories of science and imperialism. Nonetheless, conceptions of Australian science have largely remained bound by the ‘top-down’ perspective assumed by the diffusionist program. Australia was a receiver of knowledge, not a creator of culture. As a consequence too much has been assumed and too little explained. The impact of isolation on the Australian scientific community, for example, has been much commented upon. Blainey’s clichéd ‘tyranny of distance’ is wielded as a causal mechanism without reflection upon the actual meaning of isolation within an Australian setting. Wade Chambers provides some antidote in ‘Does Distance Tyrannize Science?’ (International Science and National Scientific Identity, 1991), exhorting historians to challenge the ‘metaphorical power’ of ‘the myth of “tyrannical distance”‘. But his call-to-arms has brought forth few eager combatants.

Important contributions to our understanding of science in Australia have come from outside the mainstream discipline. Bernard Smith’s European Vision and the South Pacific: 1768-1850 (1960) drew attention to the way the land and its inhabitants were perceived by European scientists. Tom Griffiths’ Hunters and Collectors (1996) revealed how amateur scientists and collectors were involved in the construction of Australia’s past. Environmental history has similarly explored how science is involved in the complex relationship between people and environment. ‘Australia’ is revealed as a participant in this process, not merely a receptacle for transplanted institutions. Greater awareness of indigenous knowledge systems has also highlighted the limitations of diffusionist models. Aboriginal knowledge systems, where they have been examined at all, have tended to suffer from ‘first chapter syndrome’. Used as an introduction to the history of Australian science, Aboriginal knowledge is paternalistically portrayed as containing the seeds of our ‘modern’ understanding. By implication or design, Aboriginal knowledge is embraced as familiar and then discarded as incomplete – as proto-science. Non-indigenous scholars now recognise the power and sophistication of such knowledge systems. Works such as Singing the Land, Signing the Land (1989), are beginning to demonstrate how indigenous knowledge stands alongside, not prior to, the western mode of science.

In 1988, Roy MacLeod noted that ‘we await works of synthesis… a new Hancock, who will convey and interpret for us the scientific enterprise, from colony to Commonwealth in the making’ (‘Introduction’, The Commonwealth of Science, 1988). And beyond. Only Ann Moyal, in a Bright and Savage Land (1986), has dared to move towards such a broadly integrative project. But hers is the work of a pioneer – a sketch rather than a detailed analysis. The 1988 Bicentennial prompted a surge in history of science publications. However, the main works, The Commonwealth of Science (1988) and Australian Science in the Making (ed. R.W. Home, 1988), were collections of articles – hors d’oeuvres only at the history of science banquet. The pickings are sparser still in relation to twentieth century science. R.W. Home has rightly directed attention towards the impact of World War II on the scientific community (‘Science on Service’, Australian Science in the Making, 1988), but D.P. Mellor’s volume of the Official History, The Role of Science and Industry (1958), remains the most comprehensive account. Boris Schedvin’s history of the CSIR, Shaping Science and Industry (1987), stands out as a the story of a scientific institution within its political and economic context, and also as a history of Australian science in the early twentieth century. However, the companion volume on the CSIRO has never eventuated.

While historians may have successfully occupied territory on the other side of World War II, scientists largely remain in control of the recent past. Academic interest in modern Australian science has more typically been within the realm of science policy or sociology of science, although with some interesting results. Life among the scientists (1989), for example, describes a quasi-anthropological study of the Walter and Eliza Hall Institute for Medical Research. Such studies should give historians cause for reflection. Methodologies and insights drawn from the social studies of science can, and should, inform the practice of historians. In the same way, historians of Australian science need to explore the cultural context of their studies, moving beyond an examination of the culture of science, towards an understanding of science as culture. Our knowledge of science, as well as of Australian history, will greatly benefit.