Contributors:
Peter Lindsay and Simon Connelly,
ITEE;
Andrew Neal, Mike Humphreys and Shayne Loft (Key Centre for Human Factors and
Applied Cognitive Psychology)
Past contributors: Andrew Hussey, David Leadbetter,
Antonio Cernone
Description:
Poor Human Computer Interface (HCI) design has contributed to many mishaps
involving aircraft, medical equipment, power stations and process plants.
Aspects of a system's HCI that contribute to mishaps include its screen
displays, alerts, and operator controls, as well as the procedures the operator
follows in using the system.
The SafeHCI project is developing a
rigorous methodology for safety assurance of HCI design, based on formal models
of operators' cognitive processes and interactions with the system, and
analysis of the causes and consequences of operator error and their
contribution to system risk. The aim is to enable HCI design options to be
assessed with respect to operator errors and system risk. The methodology is
being developed on a case study from Air Traffic Control (ATC). Psychological
experiments will be used to validate the cognitive models and to calibrate the
risk models.
Funding:
The project is being funded by an ARC large grant, and is a collaboration
between ITEE and the Human Factors Key Centre. It represents a combined
research effort within the fields of cognitive psychology, human-computer
interaction, and system safety engineering. From 2004 it has been rolled into
the ARC Centre for Complex Systems.
Sections of this site:
Within Air Traffic Control (en-route), there is a large reliance upon the human operator. The operator is required to ensure that all aircraft within their sector of control do not come within a set distance of each other (referred to as minimum separation distance). An Air Traffic Controller has failed in their task if aircraft violate this separation distance. Some ATC systems have recently changed from the traditional paper strip based system of aircraft control and moved to a wholly software based hand-over control scheme. This has many possible ramifications on the usability of a system, and it is still not known whether it is more or less safe than the old system.
In most cases some usability has been taken into consideration when designing the user interfaces for a control system, it has been traditionally hard to assess the effectiveness of HCI designs. This difficulty is partly due to the complexity of performing initial task analysis for a task as interleaved as ATC. None of the existing usability task analysis techniques translate well to a medium where the task cannot be broken into discreet segments.
Breaking the ATC task down into smaller segments is possible, it is however, impossible to analyse any given task segment without consideration of the previous task segments. For example it the probability that an operator will classify a given problem as a possible conflict situation is dependent on whether they have previously made any judgements about that particular problem. It is for these reasons that we are designing a new method of analysing operator error within complex interleaving tasks. The goal is to be able to objectively assess the effectiveness and safety of a given interface when compared to another interface.
The project began with a pilot study in 2000 on a simplified Air-Traffic Control task, with a focus on memory-related errors. A range of interrelated models were developed, to describe and draw together different aspects of the case study, from cognitive processes to user-interface design features and error propagation. The models provide a basis for evaluating the potential of operator errors and for developing safety cases for interactive systems. ARC Large grant funding was awarded from 2001.
The models that were the result of the initial investigations were a detailed cognitive model of the flow of control through the air traffic control task and a preliminary model of operator memory. Both of these models have since been updated, and the current state of these can be found in SVRC TR00-33.
A paper outlining the goals of the project was presented at the 2001 Australasian User Interface Conference. The cognitive model from the small grant was further developed for the ATC task and formalised using Statecharts (SVRC TR01-31). From the cognitive model was derived a formal CSP model of key operator decisions and actions, including how mistakes arise and how errors propagate through the operator task.
This lead us to propose a new approach to Human Error Identification which is for tasks involving highly interleaved, concurrent, ongoing activities (such as ATC). The new approach models operator failure types as behaviours – formally, sets of CSP traces – rather than as events. The failure types are formalised using temporal logic. A paper on the results was presented at the 2002 Australasian User Interface Conference (citation).
On the experimental side, the studies in 2001 focussed on testing our model of the processes responsible for conflict recognition in the ATC task. Four studies have been run (approx 120 subjects) assessing participants' ability to recognise aircraft conflicts that vary in their similarity to each other. Two further studies (200 subjects) have been run in order to identify the factors that predict individual differences in performance on the task. These studies have produced an important database for the development of the formal models.
The main focus of our research at the moment is to outline methods for using the defined formal models to aid in task and design analysis. We are examining the possibility of breaking the methodology down so that it will be generic enough to be applied to a broad class of human-computer interactions, not just those within the ATC field.
To support the methodology we are developing a prototype tool that will perform an exhaustive analysis of a given HCI design, and return relative safety when compared to either a baseline model, or another design. This will allow system designers to analyse objectively the changes that they may have made to a design, to predict the possible effects these may have on a human operator, and to make qualitative statements about whether these changes would make the system safer to use.
In 2004 the project was rolled into the new ARC Centre for Complex Systems in support of the research program on Free Flight Air Traffic Control.
Related projects: