Similar to how assertions can catch runtime exceptions in software, mflowgen allows you to define Python snippets that assert preconditions and postconditions before and after steps to catch unexpected situations at build time. Assertions are in Python to keep them concise and yet powerful. The assertions are collected and run with pytest to allow customization and user extensibility.
These assertions can be statically defined in a step configuration file or defined at graph construction time. For example, say we have a simple synthesis node with a configuration like this:
name: synopsys-dc-synthesis inputs: - adk # Technology files - design.v # RTL outputs: - design.v # gate-level netlist commands: - bash run.sh
We can assert that synthesis should not start unless it sees the technology files and the RTL design present. If either is missing, the build will not continue.
preconditions: - assert File( 'inputs/adk' ) # Technology files must exist - assert File( 'inputs/design.v' ) # RTL must exist
Similarly, after synthesis has completed we can assert that the gate-level netlist exists and that there were no issues resolving references (a common synthesis error that breaks the build). Again, the build would stop if the postcondition were to fail.
postconditions: - assert File( 'outputs/design.v' ) # Gate-level netlist must exist - assert 'Unable to resolve' not in File( 'logs/dc.log' )
Each of these items is valid Python code and you can use Python any way
you like to build your own assertions. For convenience, mflowgen natively
File class that overrides both boolean evaluation and
containment, enabling the concise syntax you see here for checking whether
or not a file exists as well as for using “in” and “not in” to search
within a file. There is also a
Tool class natively available that
overrides boolean evaluation to concisely assert whether a tool exists or
preconditions: - assert Tool( 'dc_shell-xg-t' ) # check for Design Compiler
File Class and
File class internally handles boolean evaluation simply by calling
os.path.exists(), so it can be used to check for existence of both
files and directories.
Additional knobs are available to enable case sensitivity (default is case-insensitive) and regular expression search (default is disabled):
>>> assert 'warning' in File( 'logs/dc.log', enable_case_sensitive = True ) >>> assert 'warn.*' in File( 'logs/dc.log', enable_regex = True )
Tool class handles boolean evaluation using the
function from the shell utilities Python library. This is equivalent to
% which foo on the command line.
>>> assert Tool( 'dc_shell-xg-t' ) # This statement is roughly # equivalent to this % which dc_shell-xg-t # shell statement
Adding Assertions When Constructing Your Graph¶
The assertions defined in a step configuration file can be extended at
graph construction time, meaning you can add your own design-specific
assertions in each step. You can use the
Step.extend_postconditions methods to extend either list.
For example, say we wanted to add a check for clock-gating cells as a postcondition in our synthesis step. We can assert that this cell appears in the gate-level netlist like this:
dc = Step( 'synopsys-dc-synthesis', default=True ) dc.extend_postconditions([ "assert 'CKGATE' in File( 'outputs/design.v' )" ])
Escaping Special Characters¶
Certain characters are special in YAML syntax and must be escaped if you want to use them. For example, the following postcondition in the Mentor Calibre GDS merge step (i.e., “mentor-calibre-gdsmerge”) asserts that the report does not warn about duplicate module definitions (a dangerous warning that can corrupt your layout):
postconditions: - assert 'WARNING: Ignoring duplicate structure' not in File( 'merge.log' )
: character is a reserved character in YAML syntax
since it is used for key-value stores (i.e., dictionaries in Python). The
easiest way to escape this is not to explicitly escape the character, but
to wrap the entire string in double quotes instead as shown below:
postconditions: - "assert 'WARNING: Ignoring duplicate structure' not in File( 'merge.log' )"
You can search for YAML syntax online to find more information on escaping characters in YAML files.
Writing Python assertions in a single line of Python code can be very limiting. You can write assertions with multiple lines, but it requires using the YAML syntax for a block literal (i.e., a multiline string that preserves newline characters):
preconditions: - | import math assert math.pi > 3.0
Indentation matters in Python. Fortunately, YAML syntax uses the
indentation of the first line after the
| character to derive the
indentation of all the following lines. So this entry correctly represents
the following Python code:
>>> import math >>> assert math.pi > 3.0
The pytest function that mflowgen generates looks like this:
def test_0_(): import math assert math.pi > 3.0
Note that if you write a multiline entry without the
| marker, YAML
will simply wrap the lines as if there were no newlines:
preconditions: - import math assert math.pi > 3.0
This is read as a single string, which is not valid Python:
>>> import math assert math.pi > 3.0
Defining Python Helper Functions¶
You can provide your own Python helper functions to extract information about your build which you can use in assertions.
For example, suppose we want to assert that synthesis has successfully clock-gated the majority of registers in the design. The clock-gating report looks like this:
Clock Gating Summary ------------------------------------------------------------ | Number of Clock gating elements | 2 | | | | | Number of Gated registers | 32 (94.12%) | | | | | Number of Ungated registers | 2 (5.88%) | | | | | Total number of registers | 34 | ------------------------------------------------------------
You can write a Python helper function that extracts the 94.12% figure:
# assertion_helpers.py # percent_clock_gated # # Reads the clock-gating report and returns a float representing the # percentage of registers that are clock gated. # def percent_clock_gated(): # Read the clock-gating report with open( glob('reports/*clock_gating.rpt') ) as fd: lines = fd.readlines() # Get the line with the clock-gating percentage, which looks like this: gate_line = [ l for l in lines if 'Number of Gated registers' in l ] # Extract the percentage between parentheses percentage = float( re.search( r'\((.*?)%\)', gate_line ).group(1) )/100 return percentage
Then you can assert a postcondition in the step configuration for a clock-gating percentage of at least 80%:
postconditions: # Check that at least 80% of registers were clock-gated - | from assertion_helpers import percent_clock_gated assert percent_clock_gated() > 0.80
Using Custom pytest Files¶
You can write your own pytest functions and include them in your Step (or
attach them as inputs). Then you can drop them in your step configuration
file using the
pytest: key as special syntax:
preconditions: - pytest: test_foo.py - pytest: inputs/test_bar.py
These tests will then be collected and automatically run with all the other assertions.
Assertion Scripts in mflowgen¶
When executing a step, mflowgen generates two scripts,
mflowgen-check-postconditions.py, puts them in the build directory,
and then runs these scripts before and after executing the step. At
runtime if the postcondition check fails, re-running the step (e.g.,
make 4) will only re-run the postcondition check. It will not
re-execute the step. This gives you the chance to enter the sandbox and
fix things until the postconditions pass. The build status will not be
marked “done” until all postcondition checks pass.
To completely re-run a step, you should clean that step. For example
if synthesis is step 4,
make clean-4 and
make 4 will do a
clean rebuild of synthesis.
The two assertion scripts can also be run independently with pytest. The
example below shows a precondition assertion firing and saying that
Synopsys Design Compiler (i.e.,
dc_shell-xg-t) is missing. You can
re-run the check yourself with default pytest options:
% cd 4-synopsys-dc-synthesis % ./mflowgen-check-preconditions.py > Checking preconditions for step "synopsys-dc-synthesis" pytest -q -rA --disable-warnings --tb=no --color=no ./mflowgen-check-preconditions.py F... [100%] ==================================== short test summary info ==================================== PASSED mflowgen-check-preconditions.py::test_1_ PASSED mflowgen-check-preconditions.py::test_2_ PASSED mflowgen-check-preconditions.py::test_3_ FAILED mflowgen-check-preconditions.py::test_0_ - AssertionError: assert Tool( 'dc_shell-xg-t' ) 1 failed, 3 passed in 0.05s
Or you can call pytest explicitly with your own arguments for a longer traceback (although this traceback does not say very much):
% cd 4-synopsys-dc-synthesis % pytest -q --tb=short mflowgen-check-preconditions.py F... [100%] =========================================== FAILURES ============================================ ____________________________________________ test_0_ ____________________________________________ mflowgen-check-preconditions.py:44: in test_0_ assert Tool( 'dc_shell-xg-t' ) E AssertionError: assert Tool( 'dc_shell-xg-t' ) E + where Tool( 'dc_shell-xg-t' ) = Tool('dc_shell-xg-t') ==================================== short test summary info ==================================== FAILED mflowgen-check-preconditions.py::test_0_ - AssertionError: assert Tool( 'dc_shell-xg-t' ) 1 failed, 3 passed in 0.17s