Verification project management – Development phase

Verification project management gave an overview of the six essential parameters and three phases of the verification project. This article will focus on applying the six essential parameters to development phase which follows planning phase.

Development is a steady phase of the verification project. Development schedule will remain mostly in control as long as the good job is done in planning. It should be done. A good task management system is central for the development activity. Verification team expands during this phase. Process should be put in place for productive absorption of engineers in to team. Development phase well executed will make verification team to look forward for regression phase.  Development and regression phase repeat for every milestone.

Key objectives of this phase:

• Get the internal BFMs development done
• Get the test bench development and DUT integration done
• Get the most of the test writing completed
• Get the sanity tests passing for all major features

Development phase: Clarity

• Verification team size starts growing. New members come in. Make sure the team is aware of the relevant parts of the specifications, verification plans and test bench architecture
• Clarity about the verification strategy by every team member in verification team is extremely important
• Specification experts, technical leads should present about the specifications, verification plans and test bench architecture to accelerate the ramp up
• Test bench architect should have one-to-one sessions with the major block implementers to explain the abstract classes. Interfaces and functionality should be covered. The expectations should be clearly called out.
• Technical lead or verification manager should then discuss the milestones and tasks lined for the milestones. The tasks should be assigned out engineers. Schedule buy-in should be obtained from engineers at ballpark level
• Discussions about the abstract classes which architect has implemented for guiding the implementation and milestones should be series of discussion throughout this phase to maintain the clarity between architect, engineers and manager
• Various processes, simulators, proprietary tools, pointers to documentations, tasks tracking system, bug modules, code repositories, recorded trainings, FAQs should be introduced and pointers should be passed to development verification team coming on board.
• More importantly any internal methodology wrappers, reusable code libraries, code reuse from other projects should be clearly called out to the verification team
• Our engineers are smart they will figure everything out. Sure, it will waste their finite precious energy on stupid things and then don’t complain about the delayed poor results where it matters.  Don’t complain about the rules broken and stupid mistakes.  Make it easy wherever possible so that engineers can stretch where it matters. Please do
• High level verification language and verification methodologies are projected as big challenges in implementation, They are not as big challenges as establishing clarity on above points

Development phase: Metrics

• Full development
  • Total milestones
  • Completed milestones
  • Percentage completion in current milestone
  • Total development man days pending
• Tasks
  • Total tasks
  • Completed tasks
  • Pending tasks
• Current milestone
  • Total tasks of this milestone
  • Completed tasks of this milestone
  • Total tasks per engineer this milestone
  • Total completed tasks per engineer this milestone
  • New tasks added to milestone

Development phase: People

• Test bench development has some areas where you want to your expert engineers handling it.  They are, in stimulus generation constrained random control event generation, in checks implementation scoreboard and bus functional models if they are being developed in house
• An enthusiastic engineer should be encouraged to ensure productive development environment for everyone. This role can be played by the technical lead as well partly along with junior engineer. It involves hooking everything and getting simulations up and running, keeping check-in regressions healthy, setting up flows for compile, simulation, full regression ownership and any productivity enhancements. This is a key role and tough one. It should be well supported and appreciated by everyone in team
• When team grows beyond four members create smaller teams with the enlarged scope of tasks under a theme. For example a layer ownership in protocol based BFM. Smaller teams or would call them tribes work very well. This also creates the support system by creating the knowledge pipelines. It helps in covering one another during vacations or attritions
• Group the related tasks and help engineers build the expertise. They love to meaningfully develop themselves. Help them to develop mastery and become better
• Verification lead should be available any time when needed by verification team for discussions, brain-storming sessions, to help engineers when they get stuck with some problems, clearing day-today issues or working knowledge related questions. This will ensure development engine keeps running at full steam
• Load balance based on the tasks per engineer to avoid burn out of the engineers

Development phase: Tracking

• Development is also creative work. Some of the blocks need time to build them well. To make the code reusable. Use the medium level of follow up for best results
• A good task tracking system is key to development management. Task management system should be able to generate most of the metrics listed automatically
• It’s not just sufficient to set up the tasks. Periodic scrub of the tasks is equally essential. Scrub to discover if tasks are blocked for information or dependencies. Keep clearing obstacles to ensure continued progress
• Tasks lists are mainly owned by the verification manager or verification lead. Encourage team to add new tasks discovered to task management list. Task lists are also assets. Task lists provides good understanding of work to be completed and load on engineers. Tasks worked without being on tasks management system can mislead the total project work and individual engineer’s workload
• Weekly meetings should be utilized as opportunity to assess how the closure on the current milestone is coming up. This should be used as opportunity to bring up the surprises and to identify the areas that need more attention
• Not every development task or engineer requires equal attention. Identify critical tasks and complex tasks that need attention. Verification lead should pay more attention to those tasks
• Verification leads should invest more time with the engineers who need it rather than disturbing star engineers. They will take it to closure to faster if they are left undisturbed and they will appreciate the autonomy
• Blocks that are slipping schedules or getting many new tasks than average across other blocks are also candidates for the special attention. This is the area for architects to review as to what is missed in the initial assessment. It may be possible major chunk missed in architecture and engineers are discovering it in small pieces at time. Architects should analyze and get to root of it

Development phase: Review

• Code reviews for every code check-in is great but may be an overkill. Of course nuclear weapon can also kill mosquito if you have that luxury. Code reviews at certain milestone points are very productive. Invest in the code when it has proven to be useful by meeting functionality of the milestones. Premature code reviews are waste of precious energy
• Code reviews are best done as a code walkthroughs between developer and reviewers. This provides opportunity to developer to provide the context behind the functionality of the code. This ensures the time is optimally utilized for everyone and best from the code reviews can be extracted
• Code reviews are also opportunities for providing short customized feedback to the developers on the areas for the improvements and learning’s  
• Code review actions should not be left floating in email or code review system. Code review actions which require more than couple of hours of time for fixing should go in the task tracking system
• Alignment with the architectural intent, functional correctness, adherence to standard coding practices and ease of understanding and maintainability should be accessed as part of review
• One of the ignored parts of verification planning is verification of checks itself. This should be questioned as part of the review

Development phase: Closure

• All the developed code should pass the sanity tests. It’s understandable that the detailed verification will be carried out in regression phase. All code developed should be compiling and sanitized with the basic tests.
• All critical code review actions should be addressed. If its not completed the same should be added to the task tracking system
• Compile warnings and run time warnings should be periodically reviewed and should be fixed unless its an approved as exception. These lead to failures to be painfully debugged as programming errors
• Any code limitations or parts of the features planned but not implemented should be documented. Tasks to fix these later should be added to the task tracking system
• Some times even when milestone approaches not all the tasks planned may be completed. Make an evaluation if the milestone needs to be extended or remaining tasks be moved to next milestone. Some times, it may make sense to move remaining tasks to next milestone and adjust the schedule accordingly rather than squeezing it and doing a poor quality job in rush. This provides the closure and provides sense of progress