Verification quality improvement: Verification plan

Root of most the functional verification quality problems lies in the quality of verification plan and its management. So one of the key component of quality overhaul of verification requires quality overhaul verification plans.

Verification plan is a seed. A bad seed grows into tree with the bitter fruits. Poor verification quality is result of poor quality in the planning phase.

Verification plan consists of three plans : test plan, checks plan and coverage plan. In a coverage driven constrained random approach test plan and checks plan tends to get ignored. This is dangerous. Test plan and Checks plan are equally important. In fact more important than functional coverage plan because they talk about what needs to be achieved and how it needs to be achieved.

Quality overhaul of verification plan is divided into four steps. First three steps are divided into separate blogs due to their own importance as a stand alone topic. They are linked into respective steps. It’s recommended to read those as well to gain the complete handle on the quality overhaul of the verification plan.

Quality rehaul of the verification plan is primarily made up of following steps:

1. Identify state of current verification plan

First challenge is presence of the verification plan. If present, identifying its current status. Verification plans are created initially in many cases but they are not kept up-to-date.

Three possible scenarios are described here as first challenge of the verification plan review.

All of the good, bad and ugly verification plan scenarios have to be translated to traceable verification plan. This is key step in the quality overhaul. Yes, it’s going to take effort and resources. Verification plan is backbone of the functional verification project. Unless it’s fixed the functional verification will not be able stand up to quality challenge. This should be first step of the quality overhaul.

2. Make it a traceable verification plan

A traceable verification plan primarily is one in which test plan is a single point reference where test plan, tests, test variants and their regression status stays in the common plan. This ensures that if a test or test variants is present in the verification plan it’s also getting executed.

This should be done even before verification plan review because it enables clear picture of what exists. Only when we know where we are we can go where we want.

3. Traceable verification plan review

Collect and analyze the pattern of the bugs discovered, functional coverage holes and features not verified.

Create the high level independent verification plan for review reference.

Follow the verification plan review guidelines specified for the review and identify the updates to be done.

4. Traceable verification plan update

Using these various techniques provided identify the holes, identify any updates required based on the bugs found and holes in the functional coverage, and any recent specification changes or the changes not included in the verification plan.

Repeat the principles of the planning phase to do the updates to the verification plan with the new tests, new functional coverage and new checks to be implemented. Make list of the tasks to implement the updates to the verification plan.