Scenario-based Safefty Assurance for AI-based Aviation Systems
European Union Aviation Safety Agency (EASA) defines a drone as "an aircraft without a human pilot on board, whose flight is controlled either autonomously or under the pilot's remote control on the ground or in another vehicle". Nowadays, drones are actively used in military and civilian domains, which renders several use cases like surveillance, aerial photography or video recording, traffic monitoring, and package delivery. More and more applications require autonomy features, especially when the remote pilot does not have good situational awareness or the drone is flying beyond the visual line of sight scenario.
As autonomy increases, human involvement decreases. Autonomous systems employ many complex AI-based systems. A level of safety needs to be ensured that these systems do not cause errors and endanger humans or other vehicles. A new method called scenario-based testing is being actively used in the Autonomous Vehicles domain and is slowly gaining momentum in the aviation domain.
Scenario-based Testing (SBT) is an alternative to typical real-life testing (generally distance-based) for autonomous systems. SBT for an autonomous system allows a systematic approach through designing test scenarios for its verification, which are typically followed by execution in simulation as a cost-effective method. The goal is to reduce the test cases to specific scenarios and set up a collection of validated and relevant scenarios, depending on the test subject. This is done by removing the negligible sections without an action or event (e.g. monotonous straight driving) from the validation procedure. Scenarios are generally defined by an expert using plain language and, during the scenario development process, get translated into machine-ready instructions which can be executed in the simulation environment.
Our research group has been working on a simple scenario-based approach for aviation in compliance with the European Aviation Safety Agency(EASA), illustrated in the figure below.
To realize the approach, our group has also developed a tool which allows the user to model scenarios and generate executable specifications using a user-friendly graphical environment called Operational Domain Modelling Environment.
Drone Test Bench
Our group has applied parts of the scenario modelling process in multiple projects. More information about these projects can be found in the following link.
In CPS4EU, we showcased the benefits of using scenario-based approaches to manage the verification process in simulation. Various use cases ranging from Hybrid electric vehicles to a factory floor were modelled in our tool and executed in different simulation environments.
In Musthaf, we plan to use the scenario modelling tool to test a multiflap wing system in HiL testing setup. Since the project also entails the formation of a digital twin, the SBT approach will not only be used for the physical test bench but also its virtual counterpart.
