NATS Research and Development (R&D) team and the Assuring Autonomy International Programme, a partnership between the University of York and Lloyd’s Register Foundation, have teamed up on a project to research the assurance for safety of automated decision-making tools in the complexity of an air traffic control environment.
Named Project WIZARD, the collaboration has experimented with ‘Wizard of Oz’ prototyping, which can enable faster and more cost-effective design and development for future tool concepts to explore the safety assurance of human-machine collaboration in airspace management. Existing methods to develop, test, validate and assure the safety of such tools are often resource and time intensive to set up and deliver. By using these rapid prototyping techniques, it has been possible to draw insights through iterations of different fidelities for an early design concept for a future air traffic controller tool.
The rapid prototyping approach involved a human ‘wizard’ simulating the automated decision-making capabilities of a future tool by controlling how the interface responds to user input, allowing for the research team to gather rich and useful feedback about the prototype, human-machine interactions, and how to evolve future versions.
Through this collaboration, diverse minds and expertise were brought together to shape new concepts through inclusive ways of working. The participation of NATS allows the prototyping and use cases considered in the project to have more tangible ‘real-world’ findings – in this case, the world of air traffic control.
During the development phase of the project, workshops were held with different stakeholders to explore the types of future scenarios a user of such a tool would encounter and how they might respond. This allowed the team to gain a deeper understanding of what this tool should look like, what it could be capable of doing, and how human-AI teaming could be supported safely.
The findings of the project are significant, as UK airspace is expected to become more complex over the next decade with new airspace users taking to the skies. Managing this complex environment safely requires data, planning and the ability to spot any potential problems early on, including the identification of possible hazards.
As a result of the WIZARD project, rapid prototyping techniques can be developed further and shared with the wider industry as a new way of working, developing concepts and gathering feedback from stakeholders. This also makes it possible to engage with stakeholders and gain useful insight without having to invest significant resources to develop a fully functional product or service. In turn, such approaches will also provide insights into potential hazards and safety concerns which can be gained early in the development process.
“This way of working inspires new types of collaborations and new types of projects which NATS can do with its current and future collaborators within our industry and others. We have been creating an initial playbook from the WIZARD project to support sharing of our learnings and to facilitate for others to easily adapt and use the techniques in their contexts.” – Preetam Heeramun, NATS project technical lead.
Dr Jo Iacovides, project lead from the University of York adds: “Our work on WIZARD has opened up potential avenues and opportunities to usher in safe human-machine collaboration in airspace, with real potential to impact a wide range of domains. Wizard of Oz techniques not only allow for innovation in exploring new automation concepts more rapidly, but can also support safety assurance through providing vital data on human-AI interaction much earlier on in the development process.”
