5. Test List Format¶
Each generator plugin integrated with RiVer Core, apart from generating just the test artifacts, must also generate a test list YAML. The test list has the following syntax:
<test-name>: asm_file: <path to assembly/C/test file generated> cc: <optional compile command to be used to compile the tests> ccargs: <optional compile arguments to be used> extra_compile: [<list of supplementary files to be compiled. Provided as absolute paths>] include: [<list of directories containing any required header file>] isa: <the isa string for which this test was generated for> linker_args: <arguments to be provided to the linker command> linker_file: <absolute path of the linker file to be used> result: <set to Unvailable during generation. Will change to Pass or Fail based on the simulation runs> generator: <name of the generator plugin used to generate this test> march: <the march argument to be supplied to the compiler> mabi: <the mabi argument to be supplied to the compiler> compile_macros: <list of strings indicating compile time macros that need to be enabled>
While we capture the ISA, it may seem redundant to capture the march and mabi. However, the tests can be generated to check a subset features like - no compressed instructions in targets which do support compressed instructions. Hence the redundancy.
cc and ccargs are optional here because typically the target/DUT will have its own compiler and toolchain setup and may ignore these fields. Also most of the test generators are independent of the choice of toolchain and may leave these fields blank.
All the files contain an absolute path.
5.1. Test-List Validation¶
The test-list YAML generated by a generator plugin is validated against the above rule via schema using the Cerberus python package.
Validator class. Normalizes and/or validates any mapping against a validation-schema which is provided as an argument at class instantiation or upon calling the
normalized()method. An instance itself is callable and executes a validation.
All instantiation parameters are optional.
There are the introspective properties
The attributes reflecting the available rules are assembled considering constraints that are defined in the docstrings of rules’ methods and is effectively used as validation schema for
bool) – See
ignore_none_values. Defaults to
bool) – See
require_all. Defaults to
bool) – See
purge_unknown. Defaults to to
bool) – Removes all fields that are defined as
readonlyin the normalization phase.
error_handler (class or instance based on
tuple) – The error handler that formats the result of
errors. When given as two-value tuple with an error-handler class and a dictionary, the latter is passed to the initialization of the error handler. Default:
The current schema looks like as follows:
testlist_schema = ''' asm_file: type: string nullable: False required: True check_with: filecheck cc: type: string nullable: True cc_args: type: string nullable: True generator: type: string required: True isa: type: string required: True linker_file: type: string check_with: filecheck linker_args: type: string required: True mabi: type: string required: True march: type: string required: True work_dir: type: string required: True check_with: dircheck result: type: string compile_macros: type: list schema: type: string nullable: True empty: True default:  extra_compile: type: list schema: type: string nullable: True check_with: filecheck empty: True default: 
the filecheck function will confirm if the paths to various files are valid or not