5. Test List Format

Each generator plugin integrated with RiVer Core, apart from generating just the test artifacts, must also generate a test list YAML. The test list has the following syntax:

<test-name>:
 asm_file: <path to assembly/C/test file generated>
 cc: <optional compile command to be used to compile the tests>
 ccargs: <optional compile arguments to be used>
 extra_compile: [<list of supplementary files to be compiled. Provided as absolute paths>]
 include: [<list of directories containing any required header file>]
 isa: <the isa string for which this test was generated for>
 linker_args: <arguments to be provided to the linker command>
 linker_file: <absolute path of the linker file to be used>
 result: <set to Unvailable during generation. Will change to Pass or Fail based on the simulation runs>
 generator: <name of the generator plugin used to generate this test>
 march: <the march argument to be supplied to the compiler>
 mabi: <the mabi argument to be supplied to the compiler>
 compile_macros: <list of strings indicating compile time macros that need to be enabled>

Note

While we capture the ISA, it may seem redundant to capture the march and mabi. However, the tests can be generated to check a subset features like - no compressed instructions in targets which do support compressed instructions. Hence the redundancy.

Note

cc and ccargs are optional here because typically the target/DUT will have its own compiler and toolchain setup and may ignore these fields. Also most of the test generators are independent of the choice of toolchain and may leave these fields blank.

Warning

All the files contain an absolute path.

5.1. Test-List Validation

The test-list YAML generated by a generator plugin is validated against the above rule via schema using the Cerberus python package.

river_core.constants.YamlValidator(*args, **kwargs)[source]

Validator class. Normalizes and/or validates any mapping against a validation-schema which is provided as an argument at class instantiation or upon calling the validate(), validated() or normalized() method. An instance itself is callable and executes a validation.

All instantiation parameters are optional.

There are the introspective properties types, validators, coercers, default_setters, rules, normalization_rules and validation_rules.

The attributes reflecting the available rules are assembled considering constraints that are defined in the docstrings of rules’ methods and is effectively used as validation schema for schema.

Parameters
  • schema (any mapping) – See schema. Defaults to None.

  • ignore_none_values (bool) – See ignore_none_values. Defaults to False.

  • allow_unknown (bool or any mapping) – See allow_unknown. Defaults to False.

  • require_all (bool) – See require_all. Defaults to False.

  • purge_unknown (bool) – See purge_unknown. Defaults to to False.

  • purge_readonly (bool) – Removes all fields that are defined as readonly in the normalization phase.

  • error_handler (class or instance based on BaseErrorHandler or tuple) – The error handler that formats the result of errors. When given as two-value tuple with an error-handler class and a dictionary, the latter is passed to the initialization of the error handler. Default: BasicErrorHandler.

The current schema looks like as follows:

testlist_schema = '''
asm_file:
  type: string
  nullable: False
  required: True
  check_with: filecheck
cc:
  type: string
  nullable: True
cc_args:
  type: string
  nullable: True
generator:
  type: string
  required: True
isa: 
  type: string
  required: True
linker_file:
  type: string
  check_with: filecheck
linker_args:
  type: string
  required: True
mabi:
  type: string
  required: True
march:
  type: string
  required: True
work_dir:
  type: string
  required: True
  check_with: dircheck
result:
  type: string
compile_macros:
  type: list
  schema:
    type: string
    nullable: True
  empty: True
  default: []
extra_compile:
  type: list
  schema:
    type: string
    nullable: True
    check_with: filecheck
  empty: True
  default: []

Note

the filecheck function will confirm if the paths to various files are valid or not