ICES Seminar: Hardware in the loop testing; Technology/research trends and Industrial experiences!
32 people from industry and 11 academic researchers attended the ICES seminar on Hardware in the Loop Testing with an overview on Research Trends and Industrial Experiences, which took place at KTH on Tuesday 16th March 2010.
HIL has a background in the development of early advanced systems such as spacecraft and aircraft. With the advent of embedded systems, more advanced and complex systems are being built and this also encompasses other types of applications. This emphasizes the need for more powerful tools and methods for testing. For example, accidents such as the Ariane V flight failure, provide evidence that testing a computer system in a simulated real-time environment prior to system deployment is one important approach towards system verification and validation.
HIL is typically used for functional testing and algorithm tuning, either at the subsystem or system level, where a computer control system is in a set-up that includes a real-time simulation of the controlled plant (and in general the environment that the computer control system needs to operate). HIL is primarily relevant for systems that require closed-loop testing (many modern control systems will simply not work or even start if the expected environment is not there!). Clearly, also, HIL is relevant for safety and cost critical applications. A main benefit of HIL is that abnormal and dangerous "real" scenarios can be tested prior to deployment.
There are in practice many settings of hardware in the loop simulation; in the simplest, there is only the computer control systems, interfaces and the real-time simulation of the environment. Basic challenges in making such a set up to work is to provide environment models, capable of real-time simulation, and in generating realistic "fake" inputs to the control system. Other HIL setups can also include external hardware, for example actuators, that may be difficult or impractical to simulate.
A term which is related to HIL is that of model-based testing. The seminar revealed that this term seems to be used in two ways, referring to
• the environment model that is simulated in real-time as part of a HIL setup,
• or a model of the (black-box) system under test, typically used to generate test cases.
The HIL seminar provided two sessions; the first with an overview of research and technology trends, the other with experiences from HIL use in industry. These sessions provided insight many into many interesting topics, including the following ones as examples:
• test case generation; the state space of advanced control systems is huge and there is therefore a strong desired to find new methods and tools that can generate high quality test cases.
• new research techniques for black box testing, useful for synthesizing a model of the black box and for constructing test cases.
• new challenging applications including anriroll control for car with integrated motion sensors requiring that the control system is actually moved as part of the HIL setup, as well as driver in the loop and cooperative systems (for example active safety) requiring new approaches for HIL.
• new approaches for developing real-time models, and advanced models such as thermodynamic sumilation of a combustion engine in real-time
• new hardware platforms including many core systems and HIL architectures; from centralized to decentralized with different communication standards.
• an outlook on HIL-test process standardization
• the fact that SW interaction problems were reported to be found when moving from model-based simulation to HIL
• the importance of proper organizational adoption and resources for HIL, as well as the fact that HIL can play the role to improve the development process.
The seminar was concluded by a panel discussion including the seminar speakers. The need to explain the return of investment for motivating HIL investments was discussed. Testing acts as a kind of insurance. Testing resources can also be budgeted and allocated over production entities. But testing is also closely associated with company tradition. The panelists agreed that HIL is not sufficient for finding all bugs. An overall emphasis on product quality is required to deal with complex systems development. To accomplish this, testing and testers need to involved early on in system development. After all, problems in systems integration point to the earlier development stages, and problems in the system architecture, specification, design and implementation. Also, testing for increasingly sophisticated systems requires requirements formalization to at least some degree as a basis for understanding what to test and for automating the generation of test cases. Limitations in technical education were also pointed out. It appears that students in engineering are seldom exposed to requirements engineering, and are not provided with an understanding of the needs and techniques for systematic testing.
It was agreed that follow-up seminars are relevant to arrange and that these should focus on "the other side" of the V-cycle, thus relating back to the above summary and the needs to deal properly with requirements and system design.
