Assuring City Scale Infrastructure Systems

Principal Investigators: Yair Amir, JHU Computer Science and Tamim Sookoor, JHU Applied Physics Lab

Contributors

• JHU Computer Science, DSN Lab
  • Brian Wheatman
  • Sahiti Bommareddy
  • Jerry Chen
  
  
• JHU Applied Physics Lab
  • Sebastian Zanlongo
  • Brad Potteiger
  • Tim Krentz
  • Christina Selby
  • Paul Wood
  • Nick Sarfaraz

Overview

We describe RADICS: Runtime Assurance of Distributed Intelligent Control Systems, which combines a Simplexbased, black-box monitor with a white-box monitor to ensure correct behavior and good performance of AI systems. The blackbox monitor allows the system to detect when the AI controller is on a failing trajectory and use a provably safe, but less performant algorithm, to right the system. The white-box monitor predicts when the AI controller will be put on such a trajectory before it happens and helps maximize the performance of the overall system.

Proposed Approach

Designing a monitoring architecture that can take over from the AI with a safe controller when the AI system is at risk of breaching the system invariants. We intend to combine an invariant-based Black-Box Monitor with a White-Box Monitor that evaluates the confidence of the machine learning algorithm

Publications

• DSML Workshop, June 2021: slides, paper, talk

Presentations

• Review Meeting, June 2020: slides
• Kickoff Meeting, March 2020: slides

Traffic Simulator Testbed Videos

Switching Mechanism

• Black-Box Monitoring

• Switch to Safe Controller when the average speed of past 1000 steps is below 4 (m/s).

• Switch to AI Controller when the average speed of past 1000 steps is above 5 (m/s).

• RADICS

• White-Box Monitor: Run a test simulation every 100 steps. It uses the average inflow rates of the past 100 steps to simulate the next 1000 steps and outputs the average speed.

• Switch to Safe Controller when the average speed of past 1000 steps is below 4 (m/s) or the white-box monitor predicts that the average speed will drop below 4 (m/s).

• Switch to AI Controller when the average speed of past 1000 steps is above 5 (m/s) and the white box monitor predicts that the average speed will stay above 5 (m/s).

Each of the following simulations has 25,000 steps (2500 seconds).
Between 1-10,000 steps (Segment 1), the inflow is 500 vehicles per hour on each outside edge.
Between 10,001-13,000 steps (Segment 2), the inflow is 500 vehicles per hour on one edge, and 100 vehicles per hour on other edges.
Between 13,001-25,000 steps (Segment 3), the inflow becomes 500 vehicles per hour on each outside edge again.

Safe Controller

The following simulation uses only a safe controller.

Average speed (m/s) of all vehicles is 5.65

AI Controller

The following simulation uses only an AI controller.

Average speed (m/s) of all vehicles is 5.53

Black-Box Monitoring

The following simulation uses black-box monitoring approach. The light on the top-left corner becomes green if the AI controller is running, and it becomes blue if the safe controller is running.

Average speed (m/s) of all vehicles is 5.76

RADICS

The following simulation uses the RADICS system, as described above. The light on the top-left corner becomes green if the AI controller is running, and it becomes blue if the safe controller is running.

Average speed (m/s) of all vehicles is 5.94

Here's the a version with test simulations. The yellow light on the top-right corner becomes visible if a test simulation is currently running in the background. In reality, test simulations won't cause any delay since it can be run much faster in real time given enough computing resources.

Evaluation