Context
The certification of safety-critical systems requires the exhibition of concrete evidence that a proper industrial process has been put in place during the development of the system itself. Such evidence is usually in the form of:- The development, verification, quality assurance and configuration management plans to be followed.
- A set of certification artifacts, such as requirements, test cases, source code and verification reports – possibly in different formalism (UML models, Simulink models, formal properties).
- information which links related artifacts, for example requirements to test cases.
Main Objectives
The goal of the QM (the Qualifying Machine) is to provide a tool allowing a "Continuous Qualification or Certification" process for High Integrity software as found in Civil Aviation processes (DO-178B/C). We believe that such process could drastically decrease the cost of safety-critical system certification by:- Increasing the efficiency of the certification process by automating several certification-oriented activities related to the maintenance and evolution of artifacts and traceability data.
- Promoting the reuse and evolution of already certified or qualified components supporting incremental and delta certification.
- Management of heterogeneous artifacts like requirements, test cases and verification report.
- Automated maintenance of end-to-end traceability information following the evolution of artifacts and of their mutual relations.
- Validation of artifacts against their formalized life cycle by comparing activities performed on artifacts agains those described in the plans formalized in a process modeling language.
- Support for change impact analysis by calculating:
- which artifacts may require to be re-certified if an artifact they depend on is modified
- which activities need to be (re)performed if an artifact is modified.
- Support of incremental and delta certification by identifying, at each instant in time, the minimal and ordered set of activities to be performed on each artifact to (re)achieve a "certifiable" state for the entire system.
- Support for continuous certification by automatically triggering certain certification-oriented activities each time an artifact is modified.
Technical vision
The QM is an intelligent repository specialized in managing the lifecycle and evolution of certification-oriented artifacts. We may thus call it a Certification or Qualification Artifact Management System. In short, the Certify Machine permits to:- Import artifacts in a centralized and shared repository
- Decorate artifacts with traceability and dependency links
- Associate a precise life cycle model to each artifact type
- Monitor the life cycle of artifact, acknowledging when an artifact is changed.
- Support delta and incremental certification by identifying which artifact shall be re-validated after each change.
- Support continuous certification by triggering automated activities each time an artifact is modified.
- Produce verification reports to gain certification credit.
Certification artifacts are stored inside the certifying machine after importing them. The import process permits to extract all (meta)-data relevant for the QM. Typical metadata may be the physical location of the artifact and the heuristic to understand whether an artifact has been modified. For example, an importer for UML models may extract all class methods and treat them as atomic artifacts as each of them may reasonably represent a low-level requirement; in this case, the heuristic required to evaluate whether an artifact has changed may involve comparing elements with the same xmi:id in two versions of the same UML model. A conceptually similar mechanism may be imagined for artifacts of different nature such as formal properties, Word documents containing requirements and so on. The QM will support an open interface to implement such importers as plug-ins, possibly permitting the import and management of heterogeneous artifacts.
Once imported, artifacts may be linked via traceability and dependency links Such links may be calculated starting from some user-specific rules. For example, the user may state that all test cases derived from the same required must have a name which starts with the name of the originating requirement2: in this case, the Certifying Machine will be able to calculate the traceability matrix on the fly. Alternatively, the user can provide a explicit traceability/dependency matrix and the Certifying Machine will keep track of it.
Imported artifacts are also linked to a precise life cycle model which tells which activities shall be performed on them. Such life cycle model may be expressed in a process modeling language such as BPMN.
Imported artifacts are monitored Each time an artifact changes, the QM will analyze its dependency and traceability links and produce a minimal set of activities necessary to re-achieve certifiability. Such activities may be related to both the artifact itself and its related artifacts. If, accordingly to the life cycle model, some of these activities may be performed automatically, then the Certifying Machine executes them immediately or during the next building cycle (continuous certification). The execution of such automated activities of course requires plug-ins to be developed to invoke specific tools. For example, when a test changes, the QM may be instructed to re-execute the test to re-calculate the structural coverage report.
When approaching certification, the QM can produce verification report which verifies that the modeled lifecycle was followed, all required traceability information is present and all artifacts are in a consistent state. Such report may well be part of certification evidence itself and be useful to gain certification credit. For example, the QM may produce evidence for all traceability-related objectives of DO-178 (for example A-3.6, A-4.6, A-5.5, A-7.3 and A-7.4).