Ian Johnson and Andrew Wilson

Archaeological Computing Laboratory
University of Sydney NSW  2006, Australia

+61 2 9351 3142 Tel  +61 2 9351 6392 Fax
johnson@acl.archaeology.usyd.edu.au
wilson@acl.archaeology.usyd.edu.au
http://www.timemap.net

Introduction

GIS is revolutionizing fieldwork practice in archaeology, but the analytical potential of GIS is far from being realized. This reflects the technology-driven nature of much GIS implementation and the quest for an appropriate body of theory. What interests us particularly is the lack of a systematic approach to handling the temporal dimension, so fundamental to archaeology and other historical disciplines.

In pursuing the TimeMap project (http://www.timemap.net/) we have sought to develop explicit methods of handling temporal data within a GIS framework. Our project grew out of the use of GIS in archaeology, but has a much wider application to cultural data in general; the term cultural data is intentionally broad and can include anything from archaeological site locations to historical census records; from the trajectory of expeditions to the distribution of religions; from details of a city’s history to the rise and fall of empires.

The initial work of the TimeMap project focused on three issues (Johnson 1998):

1. a methodology for recording time-based cultural features;
2. an interface for displaying time-based maps; and
3. the generation of map-based animations to display time depth.

The main products of this research were a non-prescriptive standard for structuring TimeMap-compatible datasets, a map-based interface (TMView) to TimeMap-compatible datasets and a toolkit for generating map animations from them.

Over the last two years, the project has expanded to encompass access to distributed GIS data indexed in a central metadata clearinghouse. The clearinghouse can be searched and updated through a web interface, and we have also developed tools for map registration, metadata preparation, data upload and data serving. We have developed these aspects of the project for the Electronic Cultural Atlas Initiative (ECAI), an international group dedicated to developing Internet-based access to substantial cultural datasets. ECAI aims initially to encourage exchange of scholarly research data, but also plan to make data available to the wider public through specific educational products.

From January 2000, we have funding through an Australian Research Council SPIRT (Strategic Partnership with Industry for Research and Training) grant over three years, in partnership with the Museum of Sydney and ESRI Australia. The aim of this venture is to develop an educational interface to TimeMap data – specifically information on the history of Sydney – that will lead to a new exhibition space at the Museum of Sydney, and to CD-ROM educational products.

Methodology

As the first step in the TimeMap project, we have developed a methodology for recording time-dependant features, based on vector GIS, which we call the snapshot-transition model (Johnson 1998). This methodology reflects the nature of our knowledge of historical phenomena: in general we have specific knowledge – snapshots – at irregularly spaced intervals, together with generalised knowledge about trends and changes – transitions – occurring between snapshots. Our methodology allows for explicit recording of snapshots as vector GIS objects, and of transitions as attribute data attached to them.

Although we have a workable simple model, we fully recognize that we need to do far more work to adequately capture the richness and uncertainty of historical information. Much additional work is also needed to develop analytical methods appropriate to time-based data. For practical reasons our work has so far necessarily concentrated on data recording and display through time-based queries and animation.

TimeMap data viewer

The TimeMap data viewer, TMView, is one solution – time filtered mapping – to the issues of displaying spatio-temporal data in a GIS context. TMView provides a map interface (Figure 1) to compatible time-based datasets. Datasets can be made compatible with TimeMap by creating a table of metadata which describes the structure, format and naming conventions of the data – our idea is that most time-stamped datasets can, at some level, be mapped to the TimeMap model and used without restructuring. A metadata editing tool and metadata creation wizard facilitate the creation of the required metadata table.

Figure 1: TimeMap data viewer (TMView)

TimeMap-compatible datasets – whether on a CD, hard disk, local network or remote server – are not displayed directly. Instead they are displayed as map layers created from SQL queries which include selection criteria to subset the dataset (e.g. selecting particular categories of site) along with spatial and temporal query components. Queries are updated and reprocessed as the time limits are changed in the time bar (top of screen). The data viewer is built around ESRI MapObjects and provides all the normal GIS map tools, such as zooming, selection, legend control, querying, printing and so forth, along with display of metadata and documentation relating to each layer.

The TMView program is easily adapted to provide alternative interfaces for specific applications, such as a museum visitor kiosk (see Sydney TimeMap, below) or CD-ROM interface. By judicious creation of a number of TimeMap project files[1] which retrieve and overlay different views of the data to illustrate different aspects of the resource, one can create a rich environment for exploring a substantial body of information without restricting the user to a narrow straightjacket of predefined images. The shell from which these projects are called can be as simple as a web page, or can provide greater functionality through a customized application written in, for example, Macromedia Director.

One of the essential functions of TMView is the linking of objects on a time-filtered map to local or web-based resources – such as web pages, photographs, multimedia or other maps. Map-object-to-resource linking is the key to spatial and temporal browsing of large datasets, such as heritage inventories, and a key component of our strategy in developing museum-based visitor access to historical data. Unlike ArcView or ArcExplorer, these links need not be hard-coded as URLs in the data attached to each map object, but can be generated by combining data fields with a URL mask to create links to database-generated web pages for individual objects (such as those found in many site register or museum catalogue web sites).

TMView can also work with distributed datasets stored on remote servers. Connection to remote datasets is established through a map-based metadata clearinghouse browser (Figure 2) which allows, among other things: filtering of the clearinghouse based on spatial and temporal limits and metadata searches; display of bounding boxes for datasets on a map of the world; display of metadata for selected datasets, including extended HTML-based documentation; display of thumbnail and detail views of datasets. Remote datasets are filtered on the server, the subset is downloaded and cached locally, allowing for offline access and the responsiveness of a local application rather than Internet data transfers.

Figure 2: TimeMap clearinghouse browser

Map animation

Our second area of research on the display of spatio-temporal data is map animation (animation within a geographically referenced coordinate space). Map animation is an obvious solution to the problems of displaying an extra map dimension (time) on a 2D computer screen. While there are numerous examples of map animation on the web and in CD products – for example, the shifting pattern of Iroquois sites (Snow 1997) or changes in postglacial vegetation zones (Spikins 1997) – many map animations are time-lapse animated GIFs, using a small number of base images and simple blending to create a sense of continuity.

More sophisticated animations of contemporary phenomena (e.g. weather patterns) are based on distinct data for each frame, either resulting from repetitive data collection or from modelling of the phenomena. The former is obviously beyond reach for historical phenomena, while the latter requires a level of understanding of the phenomena which is often unavailable.

We have therefore taken a simple shape-based approach to map animation of cultural data, based on tweening of vector shapes recorded as ‘snapshots’ in a TimeMap compatible GIS format, in order to generate intermediate forms for each frame of the animation. This results in a smoother animation, although it does not necessarily respect the actual system mapped. We are now working on methods of incorporating additional information to control the progress of the tweening between snapshots, and hope in the future to include a modeling capability to generate intermediate conditions (effectively keyframes) so that the animation conforms to our understanding of the system.

Our first map animation experiments were done with Houdini running on an SGI workstation, but this is a costly, one-off approach to the problem which was not based on GIS coordinates and could not be generalised into a toolkit. Since then we have developed a toolkit approach which generates intermediate frames for an AVI movie by uncontrolled tweening of objects in a MapInfo MIF file. While this makes GIS-based map animation possible, even simple, it gives rise to peculiar map animation artifacts, such as coastal features which move across bays, and the resulting AVI files are generally too large for web delivery.

We are now developing a shapefile editor, based on MapObjects, which allows the tweening to be controlled by specifying equivalent points in each successive instance of a map feature. It will eventually provide additional means of controlling the path of the animation using expert knowledge or anecdotal information. The animation is generated, direct from the shapefile, as a Shockwave Flash format (SWF) file, which can be played with a Flash player. The resulting files are much smaller to download than previous AVIs and provide smooth animation.

Sydney TimeMap

As a practical application of the TimeMap methodology, TMView and map animation toolkit, we started a project in 1995 to collect information on the history of Sydney. Known as Sydney TimeMap, this project has recently received grant funding for three years.

Sydney TimeMap will be a substantial historical information resource delivered through the TimeMap interface in a museum context. The information we are collecting includes a series of historical maps which have been scanned, registered and warped to fit the modern map base; information on the growth of the urban area (which is presented as a map animation); and historical images (drawings, engravings, etchings, paintings and photographs) which are spatially located and linked from the map interface.

A pilot TimeMap workstation was installed at the Museum of Sydney in November 2000. The pilot provides proof-of-concept for one element of the final museum gallery – a TimeMap ‘kiosk’ allowing visitors to interrogate the database, draw maps and create map animations. Through monitoring of the pilot workstation we expect to obtain valuable feedback on visitor interaction with the data and map interface. The pilot project is also a valuable opportunity to consolidate the rather piecemeal work we have carried out on Sydney historical data over the last few years and to complement it with a substantial amount of new visual resources.

Figure 3: Sydney TimeMap dual-screen kiosk interface

The pilot comprises a series of TimeMap projects based on time periods, historical themes or specific historical maps which are loaded from a visitor front-end written in Macromedia Director. TMView runs as a window within this front end on one screen of a dual monitor display, and links to resources displayed on the other monitor. Selecting a project loads a group of map layers displayed on an appropriate geo-registered historical map of Sydney. These give access to temporally and thematically selected resources drawn from a database using SQL queries and displayed in html files generated on the fly. As well as historical images the resources include sound and video files. In addition some layers provide live access to remote on-line datasets such as the NSW State Heritage Inventory.

Although the TimeMap projects deliver pre-selected datasets, visitors are free to explore the database. They can add and overlay data from different themes and periods – such as displaying the modern street grid over a nineteenth century map.

The ECAI Metadata Clearinghouse

Each TimeMap compatible dataset requires a file of metadata (implemented as a dBase file locally or a table on an SQL server). This file or table contains two types of metadata: connection metadata – which gives the TimeMap software information ondata structuring and naming conventions – and descriptive metadata for resource description, search and discovery.

In the latter part of 1998, an ad hoc group of Australian ECAI members[2] put together a descriptive metadata standard based on Dublin Core, with extensions and completion of poorly defined elements, allowing dataset to be described and catalogued in a searchable clearinghouse. The standard developed is fully documented on the ECAI web site (http://www.ecai.org) and implemented through the ECAI metadata clearinghouse (Figure 2) running on the Sydney University Archaeological Computing Laboratory Internet server.

A key feature of the ECAI metadata clearinghouse is the recording of the definitions of the metadata elements in a central, Internet-accessible server database, rather than as hardcoded values built into the software. This means that new elements, schemes and allowable values can be added simply by editing the database, and this extends to providing alternative versions of the metadata elements in different languages. Such flexibility can only be achieved by the use of an arbitrary set of element and scheme identifiers (in our case sequential numeric codes), not by the use of conventional metadata stored as a pre-defined set of textual tags. Flexibility was essential for an evolving system such as ECAI, with many international participants.

Drawing on the definitions stored in the database, our web-based software generates data entry forms with pulldown lists wherever possible, automatic entry of common default values, online help and succinct examples. Metadata are verified for consistency before they are stored in the clearinghouse.

Datasets registered with the clearinghouse are controlled by the user who registered them (anyone can sign up as a user through the web). Various mechanisms have been built to provide email notification to the user when data is modified and to an editorial team who vet datasets before they become visible to searches (to allow filtering of incomplete metadata, and of inconsequential, spurious or inappropriate datasets). The editors also apply a simple ‘rating’ of datasets – protocols for rating are still being developed.

Figure 4 summarises the system we have built for ECAI:

1.             Members of ECAI place their data on an Internet accessible server and create TimeMap connection metadata and ECAI descriptive metadata using a web-based interface or a Windows program (TMT) available for free download from the TimeMap web site;

2.             The dataset is registered with the ECAI clearinghouse, which stores a copy of the metadata to allow searching (and a useful long-term backup);

3.             ECAI users, whether other researchers or members of the general public, connect to the clearinghouse either through a web browser search or through the TimeMap viewer, TMView, and search for datasets of interest to them by spatial extent, time period or keyword searches;

4.             Information retrieved from the clearinghouse allows TMView to connect directly to the data server and place SQL queries to download relevant map data, which is cached locally and displayed as superimposed map layers on the user’s screen;

5.             Hyperlinks from map layers retrieve web resources from the clearinghouse, the data server or from third-party web servers. Hyperlinks can be static (fixed URLs) or can generate query strings to retrieve specific pages from database-driven web sites. Resources may be web pages, images, video, databases, spreadsheets or complete multimedia presentations, depending on bandwidth.

Figure 4. Distributed data architecture developed for the Electronic Cultural Atlas Initiative

Windows metadata tools

In 1999 we developed a number of Windows GUI tools to manage the metadata clearinghouse and assist with data preparation. The main user-oriented tools have now been combined into a single data preparation toolkit program, TMT, which includes the following components:

1.     Map Registration Tool
Common sources of map data for cultural projects include paper maps and aerial photographs which require scanning, registration to geographic coordinates and warping to conform to present-day coordinate systems. To streamline the process of collecting registration points on difficult source documents, we have developed a tool which allows either simple positioning and scaling or multiple control points with attached metadata, takes advantage of dual-screens when available and generates MapInfo TAB files, ESRI World files and ERMapper GCP files;

2.     Metadata creation ‘wizard’
The ‘wizard’ reads a variety of file formats (ESRI Shapefiles, MapInfo MIF files, Access, Paradox, dBase, delimited ASCII or any ODBC-compatible data source) and, with a small number of user decisions, creates appropriate TimeMap connection metadata, at the same time inserting default descriptive metadata;.

3.     Metadata editor
Metadata editing occurs under full control of the definitions downloaded from the metadata clearinghouse (which are cached locally to allow off-line work). The editor includes selection of values from pulldown lists; range checking; mandatory, repeating and single value elements. Elements can be added to the data from a collapsible tree view of the available elements;.

4.     Data pump
The ‘data pump’ reads GIS and database data from a number of formats and uploads them to an SQL data server identified in the metadata. The data pump provides selection of data fields for upload, map preview, thinning and compression of map data to reduce download volumes and speed-up map generation, automatic generation of bounding rectangles and storage of GIS data as objects in conventional (non-spatial) SQL database servers such as MySQL or Interbase;.

5.     Clearinghouse access
The toolkit communicates with the ECAI metadata clearinghouse to register new datasets, upload metadata to the clearinghouse and download metadata for local editing. The clearinghouse maintains connection information which allows the TimeMap interface software to connect to datasets on servers anywhere on the Internet.

Directions

While the current model for TMView works well for local datasets such as the disk-based application we are developing for the Museum of Sydney pilot, we run into some problems for web-based distribution, not least of which is the problem of program download size. Use of a thick-client approach – with all database access direct from the client – results in increasing download size and a multiplication of maintenance problems as we start to support more different server products.

We are therefore moving database access onto a middle tier server using CORBA protocols, so that the client software does not need to know how to access many different server products. This also gives us improvements in system security. This is the only viable solution as TMView goes beyond a small test group of users. We have a pilot three-tier version of the clearinghouse browser and dataset access working, and hope to combine this with the existing TimeMap software in October/November 2000 for initial beta testing by the end of the year. The use of a three-tier Corba approach also allows the development of clients in other languages – particularly Java for web-based access – while continuing the use of our preferred development language (Delphi) for faster, Windows-specific clients (including special implementations such as museum kiosks).

Bibliography

Johnson 1999 ‘Mapping the fourth dimension: the TimeMap project’ In CAA97: Computer Applications in Archaeology 1997. BAR International Series 750. 21pp [on CD-ROM].

Snow 1997 ‘GIS and Northern Iroquoian Demography’ In Johnson, I. & North, M. Archaeological Applications of GIS. Sydney University Archaeological Methods Series #5. Sydney: Prehistoric & Historical Archaeology, University of Sydney. CD-ROM.

Spikins 1997 ‘GIS Modelling of Holocene Vegetation Dynamics in Northern England’ In Johnson, I. & North, M. Archaeological Applications of GIS. Sydney University Archaeological Methods Series #5. Sydney: Prehistoric & Historical Archaeology, University of Sydney. CD-ROM.