The functions of government

What are the functions of government? You’ll find one possible answer in the Australian Governments’ Interactive Functions Thesaurus (AGIFT) maintained by the National Archives of Australia. AGIFT sets out a standard series of terms to describe common business activities across the different levels of government.

Functions are important to the National Archives — they aid discovery, inform appraisal, and guide government agencies in understanding what records they should keep keep. Functions also offer researchers an alternative way of exploring the Archives’ rich collections — instead of blindly searching for keywords, you can track the activities of government agencies through time.

RecordSearch, the National Archives’ online database, provides a simple way of browsing by function, though it’s not very obvious. Click on the ‘Advanced Search’ tab in RecordSearch and you’ll see an interactive diagram representing the way that records are described and arranged using the Commonwealth Records Series (CRS) system. I made this little widget a very long time ago, and there’s an alternative version, with links to the CRS manual, over at WraggeLabs.

You’ll notice that functions are associated with agencies — they’re a way of describing what government departments do. Click on the ‘Functions’ button to open up a very basic browse interface that lists preferred and alternative names for functions. There seem to be a few differences between AGIFT and the terms used in RecordSearch, I’m not sure why. The CRS Manual includes a summary of the thesaurus used in RecordSearch, and for convenience I’ve extracted a flat list of all the functions.

Just click on any function in RecordSearch to find agencies associated with it. It’s basic, but it does at least provide an alternative to the tyranny of the search box, so well described by Mitchell Whitelaw. If you’re starting a research project and don’t know what keywords are relevant, perhaps following functions will help you map the territory.

Functions also provide a way of exploring how the organisation of government changes to meet social and political needs. How are the government’s priorities in areas such as ‘migration’, ‘education’ or ‘security’ reflected in the sorts of bureaucratic structures they create? To answer this type of question you really need to extract all the data related to a function for further analysis and manipulation, but that’s not something you can easily do through RecordSearch’s web interface. Time for some hacking…

Harvesting data from RecordSearch has been something of an obsession over the past five or six years. Over that time I’ve developed a set of tools that pull structured data out of the web interface. I’ve used these tools to expose records relating to the White Australia Policy, to grab a copy of every publicly available ASIO file, and to analyse records that remain closed to public access. But to harvest functions I needed to extend my toolkit — improving the handling of agency data, and adding a simple search module to find all agencies associated with a function. Once this was done I could build a harvester — a script that just makes a series of requests to RecordSearch until all of the desired records have been retrieved, processed and stored.

All the code for my toolkit and harvester are freely available on GitHub — please download, use and improve! I’ve tried to provide fairly detailed instructions with the harvester, but you will need to fiddle about on the command line.

I haven’t done anything very exciting yet — I just wanted to make sure it was all possible. So far I’ve harvested data about agencies related to ‘MIGRATION’ and its associated functions ‘CITIZENSHIP’, ‘DEPORTATION’, ‘MIGRANT SERVICES’, ‘MULTICULTURALISM’, ‘PASSENGER ENTRY CONTROL’, and ‘REFUGEES’. You can view CSV-formatted summaries for each function on GitHub.

The CSV file for MIGRATION, for example, lists 191 associated agencies. These range from departments of state, such as the current Department of Immigration and Border Protection, Central Office, to smaller bodies like the Committee on Overseas Professional Qualifications. For each agency I’ve listed it’s status, location, life dates, and the dates during it was associated with the migration function. This is only a summary of the data I have for each agency, which includes relationships with other agencies, as well as details of any other related functions. It would be interesting to use this to draw out possible connections between functions.

As a preliminary experiment I thought it’d be interesting to create a simple timeline displaying the different government departments associated with immigration since 1900. I was trying to avoid too much coding, so I ended up hacking Plot.ly’s basic line chart. As a visualisation it’s not very exciting, but as an initial poke at the data it’s quite encouraging.

Hover over the lines to see the agency names and date ranges. Click on one of the agency identifiers (the CA numbers) to open its page in RecordSearch.

The code to create the CSV summaries as well as simple charts like this is all included with the harvester in my RecordSearch Functions repository. I’ll add tools and examples as work proceeds or time allows. Hope you find it useful.

This work is licensed under a Creative Commons Attribution 4.0 International License.