Testbench logging is one of the under focused areas. It does not receive the level of attention it deserves. What testbench architects fail to realize is poor logging has direct impact on the debug efficiency. Logging also should be designed with same intensity as testbench architecture. Most of what is discussed below is all obvious things but still not widely adopted. One of the most pathetic and most easy to fix problem of the functional verification is logging.

Logging refers to all the messages displayed by the testbench during test execution. Logging includes the error messages and debug messages. Error messages are printed when the check or assertion fails. The debug messages are printed by the different verification components of the testbench during their operation.

Logging is a commentary of test execution. Purpose of logging is to provide insights into what has happened during the course of test execution. This may not be of much value when test passes but when test fails this is highly valuable information to debug and root cause the failure. It’s worth adding few extra lines of code to track some additional state or put some additional debug information to help trace the issues faster. Any investment early on to make debugs easier pays off very well in the regression phase.

Debug starts with exploring, reading and analyzing the test execution log. We all know through our experience that debugging is where the significant amount of the effort and time is invested in the verification activity. Good logging can save lot of debug time and headache.

Audience of Log files

The audiences for the logging are verification and design engineers.  can be developers, who are doing debug or verification engineers who are not directly developing but helping with debug. Yes debug requires more engineers so team may consists of additional verification engineers who are not developing but are helping with the debug. It may also be a case where, IP/SOC companies have adopted the third party verification solutions. So the developers and debuggers can be different.

Larger debug team size is required for constrained random verification where a single test can be seeded to create variations. This can also result in different types of failures with single test. Also for constrained random tests there isn’t a linear test execution flow that mostly remains constant which can be easily followed and correlate it to log files. The stimulus generated in pseudo random in nature and it has to be found out by looking into the log files.

It may also be case where the testbench is in maintenance mode and design is undergoing the changes, which may not be directly functional like RTL changes for timing or low power etc. The full regression could be seeing multiple failures.

Logging and Verification methodologies

When we hear term “logging” it’s hard to resist thinking of the utilities classes provided by the verification methodologies for the logging. With multi level verbosity control, hierarchical log handle stitching and controlling the verbosity of verification components at various levels, message-formatting controls etc.

This is useful but often verification engineers are overly focused on these details than looking at what fundamentally are we to achieve by using all these. Remember they are just enablers. Verification methodologies are just frameworks. Just their proper use and using all their features is no guarantee that you will meet the objectives of logging.

Logging objectives and verbosity

Primary objective for logging is to provide clues into failure and help root cause the issue for failure. Logging is made up two types of messages.

They are error messages from checks or assertions and debug messages from verification components. Read more about objective of logging messages and how to map to there level verbosity here.

Debug Log content and formatting

Good logging format is not just about pleasing eye. Beauty is necessary but its definition is not always eye pleasing. Bottom line is logging should adapt seamlessly to it’s use case model. Beauty of logging is in the eyes of it’s use cases.

Read more about when to use the ASCII table format, when to use single line analysis friendly messages over here.

Debug logging tips

These guidelines are designed to help make debug effective, not only for selected geniuses in team but for everyone. Everyone should be at ease and comfortable with the debug. Debug activity affects entire team and carried out throughout product’s life cycle. Logging is one of the very important contributors in making debug efficient and simple.

Read more about these actionable tips here.