Non-regression testing (NRT) is used in testing of complex engine control units. It is not defined in software testing literature or publications.
Video Non-regression testing
Background
Throughout the years engine control unit (ECU) software requirements have become more complex and difficult to reach. This is due to the increasingly stringent emission norms and the ambitious performance in terms of fuel consumption and power request. This, in turn, increases the demand and complexity of in-vehicle driving tests and diagnostic functionalities. As a consequence, along engine control systems development, each new software release results from a sequence of many others, each one introducing new functions seeking to satisfy, time after time, the demands.
NRT is applied during each software release testing phase, at the final stage during integration testing, right before the execution of system testing, and after the module testing (or unit testing) phase. In the module testing phase, single software modules are evaluated individually, which allows the identification of elementary errors like overflow, underflow, round-off, as well as discrepancies between algorithm model simulation results and the signals coming from the engine management system (EMS). The integration testing phase, performed afterwards, aims to verify whether the tested module is correctly integrated in the overall software system. Finally, functional testing (also called validation testing) is applied to validate the algorithms concerning functional requirements. This stage is usually performed after the calibration phase and characterizes an overall system testing, concluding the new software testing phase, and allowing, therefore, its release.
In automotive applications, non-regression testing is performed as follows:
- Selection of test manoeuvers and definition of engine parameters to be monitored;
- Execution of the selected manoeuvers on benchmark software and the software under test;
- Post-processing and analysis of data acquired during these tests.
The selected test manoeuvers must be able to stimulate as many algorithms implemented in the software as possible. Cold start, overshoot of rounds per minute, and ECE cycle (a standard manoeuver used to calibrate on-board diagnosis) are relevant examples. In addition, the engine parameters selected to be monitored must represent the engine global operating state along the manoeuvers executed, such as accelerator pedal deflection, engine speed, vehicle speed, engine temperature, and throttle body opening percentage. It is also necessary to monitor the mean variables of the control chain of air and torque estimation. All these diagnostic variables must be kept under control during the execution of the manoeuvers.
The tests are performed in simulation environments such as hardware-in-the-loop (HIL) simulators or Micro HIL (feed-forward systems that work as HIL downsized simulators), which support the drawing and execution of complex maneuvers usually very difficult to perform on a real engine or car (mainly because of time, cost and equipment restrictions).
Afterwards, a post-processing tool is required to process the acquired data, offering graphic analysis and statistical data, generally addressed to skilled personal able to identify possible regression on the software. This kind of tool can also be endowed with an automatic report generator, which gathers in a single document all the analysis results and conclusions from the comparison between the two software releases during the NRT.
Maps Non-regression testing
References
Source of the article : Wikipedia
0 Comments