Toward a cooperative and open framework for the development of certifiable software

The problems encountered within the AirBus 400M program highlight the importance of deploying an effective infrastructure when developing high-integrity systems. The core of DO-178 is indeed really about:

• the quality of artifacts (how good is a requirement/algorithm/test/etc.?)
• the quality of relations between artifacts (can I justify the existence of an artifact by tracing it to other artifacts?)
• the evidence a well-defined process has been followed (was I faithful to my plan?)

The major issue within DO-178 is thus to provide evidence of the points above at a reasonable cost. The DO-178 standard enforces a requirement-driven process  with a focus on verification activities: the connection with Test-Driven Development is thus evident, as explained in the Open-DO Concepts and Ideas. 

So far, I’ve seen two major experiments to support lean and agile DO-178 certification/qualification. The first is through the use of complete tools such as OSEE. OSEE is able to track each user activity along with the artifacts it involves: it is “basically” an Application Lifecycle Management System integrated with an Action Tracking System, an Automated Testing Framework, a Requirement Management System and advanced Version Control System. Evidence of the quality of  artifacts  and of their mutual relations is provided by checking that appropriate verification activities have been performed; on the other side, evidence that a given plan has been followed is provided by analyzing the flow of tracked actions against a user-defined workflow. OSEE has been extensively used at Boeing for the Apache Program.

An alternative approach can be applied for more lightweight processes, for example the qualification of a verification tool. We have been using an hacked version of FitNesse (a web-based tool for acceptance testing) to support:

• requirements, test cases and (unit) tests management;
• tests execution;
• editing of qualification documentation (Tool Qualification Plan, Quality Assurance Plan, etc.);
• tracking of verification and quality assurance activities when needed.

We are able to track verification and quality assurance activities for each atomic artifact and we use our Version Control System to check that artifacts are modified following a precise order (a verification activity for a given artifact shall take place after the editing of the same artifact). This lightweight approach is effective, but so far we have applied it just for the qualification of verification tools.

Applying lean and agile methodologies to DO-178 certification/qualification requires investing on tools – but the reward is well worth the cost. What is your experience with this? Which tools do you use? Comments are welcome!