TEASER: Simulation-based CAN Bus Regression Testing for Self-driving Cars Software (2307.03279v1)
Abstract: Software systems for safety-critical systems like self-driving cars (SDCs) need to be tested rigorously. Especially electronic control units (ECUs) of SDCs should be tested with realistic input data. In this context, a communication protocol called Controller Area Network (CAN) is typically used to transfer sensor data to the SDC control units. A challenge for SDC maintainers and testers is the need to manually define the CAN inputs that realistically represent the state of the SDC in the real world. To address this challenge, we developed TEASER, which is a tool that generates realistic CAN signals for SDCs obtained from sensors from state-of-the-art car simulators. We evaluated TEASER based on its integration capability into a DevOps pipeline of aicas GmbH, a company in the automotive sector. Concretely, we integrated TEASER in a Continous Integration (CI) pipeline configured with Jenkins. The pipeline executes the test cases in simulation environments and sends the sensor data over the CAN bus to a physical CAN device, which is the test subject. Our evaluation shows the ability of TEASER to generate and execute CI test cases that expose simulation-based faults (using regression strategies); the tool produces CAN inputs that realistically represent the state of the SDC in the real world. This result is of critical importance for increasing automation and effectiveness of simulation-based CAN bus regression testing for SDC software. Tool: https://doi.org/10.5281/zenodo.7964890 GitHub: https://github.com/christianbirchler-org/sdc-scissor/releases/tag/v2.2.0-rc.1 Documentation: https://sdc-scissor.readthedocs.io
- N. H. T. S. Administration, “Summary report: Standing general order on crash reporting for level 2 advanced driver assistance systems,” National Highway Traffic Safety Administration, 1200 New Jersey Avenue, SE Washington, D.C. 20590, Tech. Rep. DOT HS 813 325, June 2022.
- NPR, “Nearly 400 car crashes in 11 months involved automated tech, companies tell regulators,” npr.org. [Online]. Available: https://www.npr.org/2022/06/15/1105252793/nearly-400-car-crashes-in-11-months-involved-automated-tech-companies-tell-regul
- 9News Staff, “Warning for drivers using autopilot after melbourne crash that critically injured woman,” 9news.com.au. [Online]. Available: https://www.9news.com.au/national/tesla-autopilot-driving-warning-following-melbourne-armadale-car-crash/e2fd7193-d8e7-490d-a225-f568a1dc4223
- S. Khatiri, S. Panichella, and P. Tonella, “Simulation-based test case generation for unmanned aerial vehicles in the neighborhood of real flights,” in Conference on Software Testing, Verification and Validation. IEEE, 2023, pp. 281–292. [Online]. Available: https://doi.org/10.1109/ICST57152.2023.00034
- A. Di Sorbo, F. Zampetti, A. Visaggio, M. Di Penta, and S. Panichella, “Automated identification and qualitative characterization of safety concerns reported in uav software platforms,” vol. 32, no. 3, 2023. [Online]. Available: https://doi.org/10.1145/3564821
- CAN in Automation (CiA), “History of can technology,” can-cia.org. [Online]. Available: https://www.can-cia.org/can-knowledge/can/can-history/
- T. Huang, J. Zhou, and A. Bytes, “ATG: an attack traffic generation tool for security testing of in-vehicle CAN bus,” in International Conference on Availability, Reliability and Security. ACM, 2018, pp. 32:1–32:6. [Online]. Available: https://doi.org/10.1145/3230833.3230843
- S. Yang, D. Tang, and X. Shi, “Testing system for CAN bus-oriented embedded software,” in International Conference on Computer and Information Science. IEEE Computer Society, 2014, pp. 379–384. [Online]. Available: https://doi.org/10.1109/ICIS.2014.6912162
- O. Cros, A. Thiroux, and G. Chênevert, “Cacao, a can-bus simulation platform for secured vehicular communication,” in Ad Hoc Networks - International Conference, ADHOCNETS, ser. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol. 345. Springer, 2020, pp. 213–224. [Online]. Available: https://doi.org/10.1007/978-3-030-67369-7_16
- M. Bozdal, M. Samie, and I. Jennions, “A survey on can bus protocol: Attacks, challenges, and potential solutions,” in International Conference on Computing, Electronics & Communications Engineering, 2018, pp. 201–205. [Online]. Available: https://doi.org/10.1109/iCCECOME.2018.8658720
- J. Zhang and T. Li, “Based on the queuing model of CAN bus simulation and application,” in International Conference on Bio-Inspired Computing: Theories and Applications, ser. Advances in Intelligent Systems and Computing, vol. 212. Springer, 2013, pp. 631–639. [Online]. Available: https://doi.org/10.1007/978-3-642-37502-6_76
- Y. Vershinin, B. Nnadiekwe, and S. Schulz, “Simulation of signal transmission in motion simulator using controller area network (can-bus),” in International Conference on Intelligent Transportation Systems. IEEE, 2015, pp. 2688–2693. [Online]. Available: https://doi.org/10.1109/ITSC.2015.432
- R. Louali, S. Bouaziz, A. Elouardi, and M. Abouzahir, “Platform simulation based unmanned aircraft systems design,” in Second World Conference on Complex Systems, 2014, pp. 736–742. [Online]. Available: https://doi.org/10.1109/ICoCS.2014.7061002
- C. Liu and F. Luo, “A co-simulation-and-test method for CAN bus system,” J. Commun., vol. 8, no. 10, pp. 681–689, 2013. [Online]. Available: https://doi.org/10.12720/jcm.8.10.681-689
- C. Birchler, N. Ganz, S. Khatiri, A. Gambi, and S. Panichella, “Cost-effective simulation-based test selection in self-driving cars software with sdc-scissor,” in International Conference on Software Analysis, Evolution and Reengineering. IEEE, 2022, pp. 164–168. [Online]. Available: https://doi.org/10.1109/SANER53432.2022.00030
- International Workshop on Search-Based Software Testing. IEEE, 2022. [Online]. Available: https://ieeexplore.ieee.org/xpl/conhome/9810653/proceeding
- International Workshop on Search-Based Software Testing. IEEE, 2021. [Online]. Available: https://doi.org/10.1109/SBST52555.2021
- C. Birchler, S. Khatiri, B. Bosshard, A. Gambi, and S. Panichella, “Machine learning-based test selection for simulation-based testing of self-driving cars software,” Empirical Software Engineering, vol. 28, no. 3, p. 71, 2023. [Online]. Available: https://doi.org/10.1007/s10664-023-10286-y
- C. Birchler, S. Khatiri, P. Derakhshanfar, S. Panichella, and A. Panichella, “Single and multi-objective test cases prioritization for self-driving cars in virtual environments,” ACM Trans. Softw. Eng. Methodol., vol. 32, no. 2, apr 2023. [Online]. Available: https://doi.org/10.1145/3533818
- C. Berger, “Accelerating regression testing for scaled self-driving cars with lightweight virtualization – a case study,” in International Workshop on Software Engineering for Smart Cyber-Physical Systems, 2015, pp. 2–7. [Online]. Available: https://doi.org/10.1109/SEsCPS.2015.9