CevTes - Project overview
The Project
|
The FIT-IT funded research project CevTes - CounterExample Validation and Test Case Generation Framework for Verifying Embedded Software - is dedicated to a novel approach to test case generation and test case execution for embedded systems software based on binary code analysis.
|
![]() |
News
- July 2012
- Our paper Real-Time Runtime Verification on Chip (co-authored with Matthias Függer and Jörg Brauer) has been accepted to RV'12. In this paper, we present an algorithmic framework that allows on-line motioning of past-time MTL specifications into efficient observer hardware blocks, which take advantage of the highly-parallel nature of hardware designs. For the time-bounded Since operator we get arrive at a time complexity that is double logarithmic in the time it is executed at and the given time bounds of the Since operator.
- June 2012
- Our paper A Runtime Verification Unit for Microcontrollers that we co-authored with Martin Horauer and Andreas Steininger, has been accepted to S4D'12. In this paper, we show an application of our runtime verification approach as watchdog replacement in microcontroller designs.
- Andreas Hagmann (Vienna University of Technology) and Phillip Kadletz (University of Applied Sciences Technikum Wien) joined our team to work on their Master Theses. Welcome guys!
- April 2012
- Our paper Parallel Runtime Verification of Temporal Properties for Embedded Software that we co-authored with Johannes Geist, Patrick Moosbrugger, Martin Horauer and Andreas Steininger, has been accepted to MESA'12. In this paper, we extend our runtime verification framework to a highly parallel setting, that yields a close to linear speedup compared to a single core implementation.
- August 2011
- We have received the best paper award at FMICS'11 for our paper "Past Time LTL Runtime Verification for Microcontroller Binary Code".
- October 2011
- Jörg Brauer from the Embedded Software Laboratory at the RWTH Aachen University will visit our group in the first week of November. He will give a talk "Automatic Abstraction for Bit-Vector Relations" at the RISE Seminar on Nov. 3rd.
- August 2011
- We have received the best paper award at FMICS'11 for our paper "Past Time LTL Runtime Verification for Microcontroller Binary Code".
- Our paper Automated Test-Trace Inspection for Microcontroller Binary Code that I co-authored with Jörg Brauer, Daniel Schachinger, Andreas Steininger and Stefan Kowalewski, has been accepted to RV'11. In this paper, we describe a non-intrusive framework for runtime verification of executable microcontroller code. The framework includes a custom-designed, resource friendly, microcontroller that evaluates ptLTL specification in parallel to program execution.
- July 2011
- Our paper Precise Control Flow Reconstruction Using Boolean Logic, co-authored with Jörg Brauer, our colleague from RWTH Aachen, has been accepted to EMSOFT'11. In this paper, we combine several techniques such as bit-blasting, Jörg's CAV'11 elimination algorithm and interleaved forward/backward analysis to compute fairly precise approximations of CFGs from binary code.
- June 2011
- Our workshop paper "Testing Microcontroller Software Simulators" was accepted for WS4C'11.
- May 2011
- Our paper "Past Time LTL Runtime Verification for Microcontroller Binary Code" was accepted for FMICS'11. In this paper, we show how to perform runtime verification for binary code without code instrumentation.
- Our paper "Hardware Support for Efficient Testing of Embedded Software" was accepted for MESA'11. The aim of this work is a hardware unit that is (a) easily attachable to an off-the-shelf microcontroller IP-core and (b) allows to validate whether a property violation found by abstract interpretation of binary code also exhibits in a concrete run of the system.
- November 2010
- We have received a best paper award at MEMICS'10 for our paper "Test-Case Generation for Embedded Binary Code Using Abstract Interpretation".
- September 2010
- First results of the CevTes project will be presented at
MEMCIS'10 Workshop in the paper "Test-Case Generation for Embedded
Binary Code Using Abstract Interpretation".
- August 2010
- Tobias Müller, a student majoring Electronics at the University of Applied Sciences Technikum Wien, joined the project as a student member. He will be working on hardware implementations and concepts, crucial for ruling out infeasible test traces at the actual target hardware.
- Thomas Reinbacher visited the Embedded Software Laboratory @ RWTH Aachen for a month, to collaborate on abstract interpretation techniques.
- The problem to (manually) create a model of the system beforehand
- The resulting large state spaces
- Occurrence of false-negatives originating from the abstraction techniques applied
- Automatic validation of counterexamples
- Abstraction techniques supporting binary code analysis
- Automated test case generation
- Automated execution of these test cases within the target environment
- System architecture for microcontroller IP cores to support test case generation and test case / counterexample execution within the target platform
- A test case generation approach for embedded systems software based on binary code analysis through static analysis, abstract interpretation and model checking
Introduction
Verification and validation of embedded systems software is crucial for providing flawless functionality of nowadays intelligent computer systems found in automobiles, elevators, aircrafts, medical devices, robots, etc. The common approach most widely used in industry relies on testing of defined corner cases. Although everyone is aware of the fact that only a very limited set of the test space can be covered in this way, no other more complete approaches have been adopted so far. Formal verification methods such as model checking complemented with various techniques to reduce state spaces have recently gained some momentum in this regard. Nevertheless, formal verification of embedded systems still plays a minor role. Practical restrictions of this approach are:
The novel approach presented in the CevTes project focuses on the development of a hardware framework to assist assembly code model checking and abstract interpretation. The innovation targets:
Herein, CevTes focuses on ruling out false-negatives by cross-checking error traces on the real target platform (i.e., a customized microcontroller IP-core), reducing the state explosion problem for assembly code model checking in particular, and address the problem of creating test cases for tests such as acceptance tests of embedded software.
Project Aims
The fundamental milestones of the project are:
Publications
Reviewed conference papers
| RFB12 | T. Reinbacher, M. Fuegger, and J. Brauer, Real-Time Runtime Verification on chip, in Proceedings of the 3rd International Conference on Runtime Verification (RV'12), September 25 – September 28, 2012, Istanbul, Turkey, accepted. [ conference ] |
| RHS12 | T. Reinbacher, M. Horauer, and A. Steininger, A Runtime Verification Unit for Microcontrolles, in Proceedings of the 4th System, Software, SoC and Silicon Debug Conference (S4D'12), Vienna, Austria, September 19-20, 2012, accepted. [ conference ] |
| RGM+12 | T. Reinbacher, J. Geist, P. Moosbrugger, M. Horauer, and A. Steininger, Parallel runtime verification of temporal properties for embedded software, in Proceedings of the 8th IEEE/ASME International Conference on Mechatronic and Embedded Systems and Applications (MESA'12), Suzhou, China, July 8-10, 2012, pp. 224-231. [ conference ] |
| RBS+11 | T. Reinbacher, J. Brauer, D. Schachinger, A. Steininger, and S. Kowalewski, Automated Test-Trace Inspection for Microcontroller Binary Code, in Proceedings of the 2nd International Conference on Runtime Verification(RV'11), San Francisco, California, US, September 27 - September 30, 2011, pp. 239-244. [ conference ] |
| RBS+11 | T. Reinbacher, J. Brauer, D. Schachinger, A. Steininger, and S. Kowalewski, Automated Test-Trace Inspection for Microcontroller Binary Code, in Proceedings of the 2nd International Conference on Runtime Verification(RV'11), San Francisco, California, US, September 27 - September 30, 2011, accepted. [ conference ] |
| RB11 | T. Reinbacher and J. Brauer, Precise Control Flow Reconstruction Using Boolean Logic, in Proceedings of the 11th International Conference on Embedded Software(EMSOFT'11), Taipei, Taiwan, October 09-14 2011, accepted. [ conference ] |
| RGKH11 | T. Reinbacher, D.Gueckel, M. Horauer, and S. Kowalewski, Testing Microcontroller Software Simulators, in Proceedings of the Workshop on SLE for CPS at INFORMATIK 2011 (WS4C'11), Berlin, Germany, October 04-07 2011, accepted. [ conference ] |
| RBH+11 | T. Reinbacher, J. Brauer, M. Horauer, A. Steininger, and S. Kowalewski, Past Time LTL Runtime Verification for Microcontroller Binary Code, in Proceedings of the 16th International Workshop on Formal Methods for Industrial Critical Systems (FMICS'11), Trento, Italy, August 29-30 2011, accepted. [ conference ] |
| RSM+10 | T. Reinbacher, A. Steininger, T. Mueller, M. Horauer, J. Brauer, and S. Kowalewski, Hardware Support for Efficient Testing of Embedded Software, in Proceedings of the 7th International ASME/IEEE Conference on Mechatronics & Embedded Systems & Applications (MESA'11), Washington, DC, USA, August 28-31 2011, accepted. [ conference ] |
| RBH+10 | T. Reinbacher, J. Brauer, M. Horauer, A. Steininger, and S. Kowalewski, Test-case generation for embedded binary code using abstract interpretation, in Proceedings of the Annual Doctoral Workshop on Mathematical and Engineering Methods in Computer Science (MEMICS'10), Mikulov, Czech Republic, October 22- 24 2010, pp. 151-158, Best Paper Award. [ conference, pdf ] |
Participating Organizations & People
![]() |
![]() |
![]() |
| Vienna University of Technology | University of Applied Sciences Technikum Wien | RWTH Aachen University |
| Institute of Computer Engineering, Embedded Computing Systems Group http://ti.tuwien.ac.at/ecs |
Department of Embedded Systems http://embsys.technikum-wien.at |
Department of Computer Science XI http://embedded.rwth-aachen.de |
| contact: Mag. Thomas Reinbacher, MSc Ao.Univ.Prof. DI. Dr. Andreas Steininger |
contact: FH-Prof. Dr. Martin Horauer |
contact: Dipl.-Inf. Jörg Brauer |
| role: thesis supervision, scientific concepts, synergies in formal methods and testing | role: system-on-chip, CPU simulator know-how, implementation and expertise on electronics | role: formal verification techniques, abstract interpretation, static analysis, model checking |
(Student) Team Members
![]() |
Phillip Kadletz University of Applied Sciences Technikum Wien Masters degree program Embedded Systems July 2012 - present Topics: Monitor Compilation |
![]() |
Andreas Hagmann Vienna University of Technology Masters degree program Computer Engineering June 2012 - present Topics: Monitor Optimizations, Monitoring of Future Time Properties |
![]() |
Patrick Moosbrugger University of Applied Sciences Technikum Wien Bachelor's degree program Electronic Engineering October 2011 - present Bachelor Thesis: Runtime Verification of properties involving timing constraints |
![]() |
Johannes Geist University of Applied Sciences Technikum Wien Bachelor's degree program Electronic Engineering October 2011 - present Bachelor Thesis II: Runtime Reasoning for Embedded Systems Bachelor Thesis I: Parallel Runtime Verification |
![]() |
Daniel Schachinger Vienna University of Technology Bachelor's degree program Computer Engineering March 2011 - October 2011 Bachelor Thesis: Combining real-time monitoring and WCET analysis towards timed Runtime Verification |
![]() |
Tobias Müller University of Applied Sciences Technikum Wien Bachelor's degree program Electronic Engineering August 2010 - February 2011 Bachelor Thesis: A Hardware Monitoring Unit to assist testing of Embedded Systems binary code |
Acknowledgements
![]() |
The project CevTes is funded by the initiative FIT-IT of the Austrian Federal Ministry of Transport, Innovation and Technology (BMVIT) in co-operation with the Austrian Research Promotion Agency (FFG) under grant 825891.
More information on FIT-IT can be found on http://www.ffg.at
|
![]() |











