Verification project management- Planning phase

Verification project management gave an overview of the six essential parameters and three phases of the verification project. This article will focus on applying the six essential parameters to planning phase.

Planning is very important phase of the verification project. Verification plan and test bench architecture are created during this phase. It sets the foundation for the development phase and regression phase. Entering execution with weak plan is like taking a route of slippery slope. Once taken there isn’t much chance of recovery and it takes you down for deep fall.

Key objectives of this phase:

• Get the verification plan in place
• Get the test bench architecture in place
• Get the detailed tasks list for the test bench development
• Milestones lists and division of the tasks per milestone

Next let’s look at what each of the key management principles identified in the verification project management translate to in the planning phase.

Planning phase: Clarity

Following four elements should be clear during planning phase. Overall work scope clarity needs to be established during this phase.

• Scope understanding: Scope of the verification is based on the specifications. Key prerequisite for planning phase is specifications. Specifications mean both the standard requirement specifications and implementation specifications. Most of the today’s designs deal with multiple specifications. It’s very important to identify the list of all the specifications applicable.
• Specifications understanding: Clear understanding of the specifications both the requirement specifications and micro-architecture specifications
• Verification strategy understanding: A clear verification strategy needs to be established based on the specification scope and understanding. This forms the basis for the verification plan and test bench architecture creation
• Team understanding:  Clear list of the contact persons in architecture, design and verification team at one place

Planning phase: Metrics

Metrics during planning phase are meant to provide the idea about the completeness of the verification plan, test bench architecture and development task lists creation.

• Total number of features
• Verification plan completeness percentage
  • Number of tests identified
  • Number of checks identified
  • Number of the functional coverpoints identified
• Test bench architecture completeness percentage
  • Overall architecture completeness
  • Major blocks of test bench
  • Total number of major blocks identified
  • Total number of major blocks for which interface and functionality is defined
  • Abstract classes
    • Total number of classes identified
    • Total number of abstract classes coded and compiling
  • Development tasks lists
    • Total number of blocks for which task list is identified
    • Total development tasks

Planning phase: People

• Best of your experienced engineers should be working on verification plan and test bench architecture development. There have been misconceptions to get this done by the relatively inexperienced engineers with perception that all we are producing is a document finally. This is totally a wrong perception to follow
• Technical leader of the project should be involved in this activity very actively. He should be one of the key contributor to verification plan and test bench architecture
• Engineers with natural attraction for the engineering depth are well suited for this activity. Both the verification plan and test bench architecture requires someone to put his heart into activity and go deep in the details. Figure out as many things as possible
• Engineers working on plan should be allowed to think theoretically enumerating all the verification scenarios without worrying about the resource and schedule content. Think theoretical and execute practical approach should be followed

Planning phase: Tracking

• Planning phase involves one of the highest levels of creative work. Out of box thought process may be required to set the verification strategy. Good test bench architecture requires fairly deep and creative ideas. Make sure it’s not rushed through. It’s easy to rush and create half-baked plans as it’s difficult to judge if it’s complete in early stages of the project
• Put your best team in action and trust them to use their best judgment.  Agree we do not want paralysis by analysis situation but at the same time high pressure to close can result in premature closure. Premature closing of planning phase will result in costly reworks later. Its pay now or later with huge interest rates
• Put your team working on planning and architecture in an island. Talk to them only when needed. Allow the team members to get into flow state for best work to emerge. Instead of weekly consider biweekly or monthly status tracking of the metrics identified.

Planning phase: Review

• Verification strategy should be carefully reviewed. It sets the basis for the verification plan and test bench architecture
• Verification plan structure review has to be conducted periodically. This ensures no big ticket items are missed
• Review of the verification strategy, verification plan and test bench architecture should be conducted with the architecture and design teams
• Verification plan should be executable. It should not be limited to a very high level plan. A junior verification engineer should be able to pick up the test plan and should be able to write a test, implement a check from checks plan and write cover points from functional coverage plan with very little interaction with the verification plan owner
• Verification managers should pick a feature and question it for its consistency in thought process across all three test plan, checks plan and coverage plan
• Architecture should be questioned for flow from normal operation, error injection, control event generation and processing, various checks implementation and ease of functional coverage collection point of view
• Alignment of the verification plan and test bench architecture to the verification strategy should be reviewed
• Review actions should be tracked and closed. It’s possible that certain areas are going to remain open for a while till the further understanding is developed. This is fine as long as they are clearly documented and tracked

Planning phase: Closure

• All the specifications applicable are identified and made accessible in a global shared location for verification team
• Code repositories for development, email aliases for the communication and discussions, Task tracking system for project and bug database are setup
• All three plans test plan, checks plan and coverage plan constituting the verification plan in executable form should be checked-in under code repository
• Detailed test bench architecture in place and checked-in under code repository
• Reviews of the verification strategy, verification plans and test bench architecture completed.  Major review items addressed and open items documented and put in a task tracking system for closure
• Tasks list for all the development activities captured in the task management system. Tasks grouped under the functional milestones
• Schedule forecasts made created based on tasks lists. Based on the forecasted schedule, deadlines and current team availability hiring or external contractors required for the next phases should be identified
• Decisions regarding the exploration and evaluations of third party Bus functional model(BFM )s, verification solutions or tools should be short listed
• Simulator licenses and compute resource requirements should be checked for sufficiency based on the forecasted verification plan’s requirements. If additional upgrades are needed the procurement process should be started
• It’s possible verification plan and test bench architecture is not 100% complete. It’s hard to achieve that unless its repetition. Focus during planning phase should be on completeness of major sections in verification plan and major blocks in test bench. Second level of sub-sections in verification plan and details of blocks in test bench architecture can be complete any where between the 60-80 %. This provides good visibility and rest can be figured out as part of development phase. This achieves balance between waiting too long vs. starting too early