5. Test List Format¶
Each generator plugin integrated with RiVer Core, apart from generating just the test artifacts, must also generate a test list YAML. The test list has the following syntax:
<test-name>:
asm_file: <path to assembly/C/test file generated>
cc: <optional compile command to be used to compile the tests>
ccargs: <optional compile arguments to be used>
extra_compile: [<list of supplementary files to be compiled. Provided as absolute paths>]
include: [<list of directories containing any required header file>]
isa: <the isa string for which this test was generated for>
linker_args: <arguments to be provided to the linker command>
linker_file: <absolute path of the linker file to be used>
result: <set to Unvailable during generation. Will change to Pass or Fail based on the simulation runs>
generator: <name of the generator plugin used to generate this test>
march: <the march argument to be supplied to the compiler>
mabi: <the mabi argument to be supplied to the compiler>
compile_macros: <list of strings indicating compile time macros that need to be enabled>
Note
While we capture the ISA, it may seem redundant to capture the march and mabi. However, the tests can be generated to check a subset features like - no compressed instructions in targets which do support compressed instructions. Hence the redundancy.
Note
cc and ccargs are optional here because typically the target/DUT will have its own compiler and toolchain setup and may ignore these fields. Also most of the test generators are independent of the choice of toolchain and may leave these fields blank.
Warning
All the files contain an absolute path.
5.1. Test-List Validation¶
The test-list YAML generated by a generator plugin is validated against the above rule via schema using the Cerberus python package.
-
river_core.constants.
YamlValidator
(*args, **kwargs)[source]¶ Validator class. Normalizes and/or validates any mapping against a validation-schema which is provided as an argument at class instantiation or upon calling the
validate()
,validated()
ornormalized()
method. An instance itself is callable and executes a validation.All instantiation parameters are optional.
There are the introspective properties
types
,validators
,coercers
,default_setters
,rules
,normalization_rules
andvalidation_rules
.The attributes reflecting the available rules are assembled considering constraints that are defined in the docstrings of rules’ methods and is effectively used as validation schema for
schema
.- Parameters
ignore_none_values (
bool
) – Seeignore_none_values
. Defaults toFalse
.allow_unknown (
bool
or any mapping) – Seeallow_unknown
. Defaults toFalse
.require_all (
bool
) – Seerequire_all
. Defaults toFalse
.purge_unknown (
bool
) – Seepurge_unknown
. Defaults to toFalse
.purge_readonly (
bool
) – Removes all fields that are defined asreadonly
in the normalization phase.error_handler (class or instance based on
BaseErrorHandler
ortuple
) – The error handler that formats the result oferrors
. When given as two-value tuple with an error-handler class and a dictionary, the latter is passed to the initialization of the error handler. Default:BasicErrorHandler
.
The current schema looks like as follows:
testlist_schema = '''
asm_file:
type: string
nullable: False
required: True
check_with: filecheck
cc:
type: string
nullable: True
cc_args:
type: string
nullable: True
generator:
type: string
required: True
isa:
type: string
required: True
linker_file:
type: string
check_with: filecheck
linker_args:
type: string
required: True
mabi:
type: string
required: True
march:
type: string
required: True
work_dir:
type: string
required: True
check_with: dircheck
result:
type: string
compile_macros:
type: list
schema:
type: string
nullable: True
empty: True
default: []
extra_compile:
type: list
schema:
type: string
nullable: True
check_with: filecheck
empty: True
default: []
Note
the filecheck function will confirm if the paths to various files are valid or not