Testbench is primary vehicle for verifying the functional verification requirements. Quality of testbench affects the quality of verification. Yet, it’s often ignored due to various myths.

This blog will look into 5 such myths about testbench quality and share different perspective to look at them and how it affects the projects.

1. Verification coverage is the only quality metric

Often verification quality is perceived by number of the RTL bugs discovered pre-silicon and no showstopper bugs discovered in post silicon validation or in field. While without doubt, this is important and primary but this is not end.

Unless you are ending the design IP right after the tapeout, the process of verification continues for the next set of enhancements and bug fixes in the design IP. While your first tapeout may be high quality tapeout but that does not guarantee your subsequent will also be high quality. Sustainable quality requires looking beyond bugs discovered and verification coverage.

Functional verification is made up of three phases planning, development and regression phase. Results of all three phases contribute to the quality.

Quality metric often ignored is that of maintainability. Is the testbench and test suite easy to maintain? Can it cope with the variations of the next generation design changes? Can someone new come in help with verification task at critical point quickly? What is effort for doing simple enhancements? What is efficiency of the verification process? Are you ignoring these questions? Please rethink !

2. We (wor)ship RTL. We don’t ship test bench

It’s true. Semiconductor companies ship the RTL design IP in the form of ASIC not the testbench. Yes, its also true testbench is not seen or directly experienced by customers. Does that give license to abuse testbench code to get things done?

Does it not matter at all? Yes, it does matter. If you see the latest semiconductor studies, the growth in number of verification engineers has crossed number of design engineers. Why go elsewhere count them in your own team? Does math match studies? If not rethink are you staffing it right?

Wilson studies 2016 indicates verification engineers have become a major project cost management concern, and is one indicator of growing verification effort.

Typically there is no difference in the compensation of design vs. verification engineers. Color of money spent is same in producing design and testbench. If that is case then why is the quality of the testbench code compromisable?

Don’t just look at what you ship but also look at where you spend. There have been instances where the money spent on verification is exceeding mask costs.

3. Just taped out

After design is taped out, why do I need to care about testbench quality? It’s job is done.

If your testbench is like a disposable syringe, which is, discarded right after the design tapes out, then no need worry about the quality of the testbench. If you are going to retain the same testbench for next 3-5 years then you need to worry about testbench quality.

Why? Because maintenance cost is going to come right out of your pocket. Plenty of duplication of the code, lack of theme in architecture and code abuse, it all leads to additional engineering resources to maintain it. It raises the cost of the design IP is maintained and it grows as the time progresses as it continues degrade further.

4. Our testbench does the job

Our testbench does the job and it does it well. We are not missing any showstopper issues why do I need to care about the quality of testbench?

Well, sure if you are in business of making one chip and shutting down the shop then don’t worry. But if you are in it for a long haul, start thinking about it. Poor quality testbench will affect the quality of the engineers working with it. Don’t you need high quality verification engineers on your next project?

Of-course you do. Don’t you want to invest in building them? A junior verification engineer, who starts off on suboptimal testbench gets used to it. After working several years on it, when you ask him to build testbench for your next critical design don’t expect it to be state of art.

High quality engineers are built around high quality code. A high quality testbench is the best training school for your engineers. A high quality testbench is the real training your engineers need as to how to do high quality verification work.

5. My verification team will retire here

When the same verification team continues to work, project after project, it will certainly ease and reduce the maintenance effort. Even if the testbench code has quality problems the team knows its weak points and will take care of it. They have understood it very well by working with it over years and so they will manage it.

Question is can you hold on to same team? Let’s accept reality its bit challenging. Teams are going to keep changing. Every time testbench changes hands the high quality testbench lends itself well for documentation and quick understanding. Whereas sub-optimal testbenches will be difficult to document due to lack of theme and also make the understanding to new team difficult.

To summarize: Quality of testbench is important. Some of the myths that we need to be overcome and pay attention to it are:

• Verification coverage is the only quality metric: Although verification coverage is primary metric but don’t completely ignore maintainability as well
• We (wor)ship RTL. We don’t ship test bench: While its true, consider not only what you ship but also where you spend
• Just tapedout : Design tapeout may not be end of testbench lifecycle
• Our testbench does the job: While this is important in short term but long term remember high quality verification engineers are created around high quality testbench
• My verification team will retire here: In reality, teams keep changing. High quality testbench lends itself for faster ramp up to new team