The Black-Box Simplex Architecture for Runtime Assurance of Autonomous CPS (2102.12981v3)

Abstract: The Simplex Architecture is a runtime assurance framework where control authority may switch from an unverified and potentially unsafe advanced controller to a backup baseline controller in order to maintain the safety of an autonomous cyber-physical system. In this work, we show that runtime checks can replace the requirement to statically verify safety of the baseline controller. This is important as there are many powerful control techniques, such as model-predictive control and neural network controllers, that work well in practice but are difficult to statically verify. Since the method does not use internal information about the advanced or baseline controller, we call the approach the Black-Box Simplex Architecture. We prove the architecture is safe and present two case studies where (i) model-predictive control provides safe multi-robot coordination, and (ii) neural networks provably prevent collisions in groups of F-16 aircraft, despite the controllers occasionally outputting unsafe commands.

• The paper presents an innovative runtime assurance method that uses a decision module to validate commands from unverified controllers in real time.
• It employs runtime simulations and reachability analyses to dynamically ensure safety when integrating advanced control techniques like MPC and neural networks.
• Numerical case studies in multi-robot coordination and mid-air collision avoidance demonstrate the architecture’s effectiveness in maintaining safety and performance.

An Evaluation of the Black-Box Simplex Architecture for Runtime Assurance of Autonomous Cyber-Physical Systems

The paper "The Black-Box Simplex Architecture for Runtime Assurance of Autonomous CPS" introduces an innovative architectural approach aimed at enhancing the runtime safety assurance of autonomous cyber-physical systems (CPS). The authors propose the Black-Box Simplex Architecture (BSA), which provides runtime assurance by replacing the traditional requirement of statically verified baseline controllers with runtime verifications, thereby admitting the integration of powerful but unverified control techniques such as model-predictive control (MPC) and neural network controllers.

Key Concepts and Methodology

The BSA departs from the classical Simplex Architecture by allowing both the advanced controller (AC) and the baseline controller (BC) to be treated as black-box components, removing the necessity for static verification of the BC. Instead, the framework relies on a decision module (DM) that conducts runtime checks to ascertain the safety of commands generated by the controllers. The DM either accepts or rejects command sequences based on runtime simulations or reachability analyses, utilizing stored command sequences to maintain safety when necessary. This approach leverages previous computations where the DM validated safe command sequences.

The BSA has been theoretically validated, with the authors proving two critical theorems: one assures that system safety is preserved under the architecture, and the second outlines conditions under which the system maintains transparency, allowing the AC to control the system whenever feasible.

Numerical Results and Case Studies

The practicality and effectiveness of BSA are demonstrated through two substantial case studies. The first case paper involves a multi-robot coordination system where robots must avoid collisions while pursuing targets. In this scenario, a model-predictive control based BC generates command sequences, ensuring safety through a lookahead strategy combined with potential-field techniques for collision avoidance. The system successfully manages scenarios where the controllers generate unsafe commands, illustrating the DM’s ability to recover and maintain safety using stored command sequences.

In the second case paper, a mid-air collision avoidance system for groups of F-16 aircraft is investigated. Here, neural networks serve as BCs to prevent collisions in a dynamically complex environment. Despite being unverified, these neural network controllers, when integrated within the BSA framework, are demonstrated to safely navigate scenarios previously prone to collisions, achieving safety distances above specified thresholds.

Implications and Future Directions

The BSA marks a significant advancement in runtime assurance methodologies by facilitating the inclusion of sophisticated and potent, yet unverifiable controllers in safety-critical autonomous CPS. Its implications can be far-reaching, particularly in fields where static verification of complex systems is impracticable. The architecture not only enhances safety but also expands the design space for employing learning-based controllers in real-world applications.

Moving forward, the research direction could involve optimizing the runtime verification process to ensure faster verifications, which would enhance the architecture's applicability to real-time systems with strict latency constraints. Another trajectory could involve extending BSA to heterogeneous systems involving mixed controllers and exploring hybrid assurance techniques that blend offline computations with runtime checks more effectively.

In conclusion, this research provides valuable insights into a feasible methodology for ensuring safety in autonomous CPS while leveraging the capabilities of advanced but unverified controllers. The Black-Box Simplex Architecture represents a promising avenue for addressing the intricate balance between system performance and safety assurance in ever-evolving autonomous system landscapes.