Building an EU AI Act compliance matrix: traceability from obligation to evidence

This guide builds the obligation to test to evidence matrix by hand from the obligation records of Art. 11 with Annex IV and Art. 9: one row per requirement element, an id per row, a test column, an evidence column with a path and a content hash, and a status. The procedure is step 43 of PR-11-DOC v1.0, with the document control of prEN 18286Published. At the end the reader holds a CSV file, a script that fails on a missing or altered evidence file, and a count of open rows per article.

What the obligation asks for

Art. 11(1) second subparagraph of Regulation (EU) 2024/1689 requires the technical documentation to demonstrate that the high-risk AI system meets the requirements of Chapter III Section 2 and to give national competent authorities and notified bodies the information they need to assess the system against those requirements. Annex IV section 7 requires a list of the standards and specifications applied and, where no harmonised standard is applied, a description of the solutions adopted for each requirement. Annex IV section 2(g) requires the test logs and test reports, dated and signed. Neither article names a matrix; the matrix is the index that connects each requirement element to the record that answers it, and the Art. 11 obligation page lists every element with its split into testable, documentation and process. The duty applies from the Annex III application date 2027-12-02Deferred for a system classified under Art. 6(2).

The guide covers three testable elements of the Art. 11 record: the declared metric values match the dated and signed test reports for the same version and conditions (Annex IV section 2(g)); every test log and report carries a date and a responsible signature (Annex IV section 2(g)); and each entry in the standards list maps to the requirement it addresses (Annex IV section 7). It covers two testable elements of the Art. 9 record: each test in the risk management file runs against a prior defined metric and probabilistic threshold (Art. 9(8)), and the residual risk of each hazard is compared with the acceptance criterion recorded for that hazard (Art. 9(5)). Art. 9(10) adds a reason for the matrix on the Art. 9 side: a provider that combines Art. 9 with a risk management procedure under other Union law records a mapping that shows where each aspect of Art. 9(1) to 9(9) is covered, and the matrix rows for Art. 9 are that mapping. The matrix does not run a test; the row records which protocol step produced the evidence and where the evidence sits.

Inputs

The inputs are the subset of the PR-11-DOC v1.0 input table that step 43 uses, plus the tools the examples call. The full table is on the protocol page.

The obligation records for the articles in scope: the testable, documentation and process lists on each obligation page, or the CSV export of the traceability matrix template for the role and tier.
The technical documentation as it stands, with a version identifier per document and a file path per section.
The test reports, test logs, consistency tables and records that the evidence column will point to, each with a version identifier.
The risk management file with the hazard register, the threshold declaration and the residual risk record.
The frozen input set from step 6 of the protocol: one identifier and one content hash per document version.
A shell with sha256sum, python and jq.

Procedure

Each step names the protocol step or the clause it derives from and the record it produces. Code blocks follow the step they belong to. Every path, identifier, hash and count in the examples is synthetic.

Export the CSV from the browser tool for the role and tier, or list every element of each obligation record.
Record seed file: one row per element of every obligation record in scope
[AIAssuranceLabs procedure; PR-11-DOC step 7]
Assign each row one id in the form TM- followed by the article number and the row number, numbered in record order.
Record id column: TM-<article number>-<row number>, assigned once and never reused
[AIAssuranceLabs procedure; PR-11-DOC step 43]
Mark each row test, document or process record, from the list the element sits in on the obligation page.
Record kind column: test, document or process record per row
[AIAssuranceLabs procedure; PR-11-DOC step 43]
Name the protocol and the step that produce the evidence for each test row.
Record test column: protocol and step per test row; PR-11-DOC step per document and process row
[AIAssuranceLabs procedure; PR-11-DOC step 43]
Compute a content hash for each evidence file version and record it in the frozen input set.
Record frozen input set: identifier, path and content hash per evidence file version
[AIAssuranceLabs procedure; PR-11-DOC step 6]
Fill the evidence column with the record name, its path, its hash and its Annex IV section.
Record evidence, evidence_path, evidence_sha256 and location columns per row
[Regulation (EU) 2024/1689, Annex IV section 2(g) and section 7; PR-11-DOC step 43]
Write the regulation entry id, not a date, in the applies-from column of each row.
Record applies_from column: regulation entry id per row
[AIAssuranceLabs procedure; regulation data rule]
Set each row to missing, draft, ready or not-applicable, with the reason for each not-applicable row.
Record status column: missing, draft, ready or not-applicable with the reason in the evidence column
[Regulation (EU) 2024/1689, Annex IV introductory sentence; PR-11-DOC step 10]
Run the evidence check and correct each row whose path is missing or whose hash differs.
Record check output: one line per failing row with the reason, exit code 1 on any failure
[AIAssuranceLabs procedure; PR-11-DOC step 6]
Count the open rows per article and record the count as the coverage summary.
Record coverage summary per article: rows, open, ready, not applicable
[AIAssuranceLabs procedure]
Record the matrix version, the system version it describes, the owner and the update trigger.
Record matrix change log: matrix version, system version, owner, update trigger
[prEN 18286, published, clauses on document and record control; PR-11-DOC step 42]
File the matrix with the technical documentation as the index from requirement element to record.
Record technical file location of the matrix, named in the documentation index
[Regulation (EU) 2024/1689, Art. 11(1) second subparagraph; PR-11-DOC step 43]

Example for steps 1 to 4. The browser tool exports one CSV with the columns id, system, article, requirement, kind, test, evidence, applies from and status, and the column and id scheme it uses are published in the traceability logic file. The hand-built file below keeps those columns and adds three: evidence_path, evidence_sha256 and location. The export writes the date of the regulation entry in the applies-from cell; the hand-built file keeps the entry id instead, so that the date resolves from the regulation data at the time the file is read and no date is typed into the matrix. The six rows are synthetic: four for Art. 11 with Annex IV and two for Art. 9, the system name and version belong to no system, and the hashes are the digests of six placeholder files.

id,system,article,requirement,kind,test,evidence,evidence_path,evidence_sha256,location,applies_from,status
TM-11-01,demo-system 2.4.0,art-11,"Annex IV section 2(g): declared metric values match the dated and signed test reports for the same version and conditions",test,PR-11-DOC steps 16 and 20,Declared metric to test report consistency table,evidence/art-11/dmc-table-2.4.0.csv,334849e88c0801418f2eb7737ea24b1bc1331cb1fd33e6e700a8ff0939cdc55f,Annex IV section 2(g),annex-iii-high-risk,ready
TM-11-02,demo-system 2.4.0,art-11,"Annex IV section 2(g): every test log and test report carries a date and a responsible signature",test,PR-11-DOC step 33,Test report register,evidence/art-11/test-report-register.csv,96cc62677a0d53e2ac4bb661566333a4b98c13c9dba3e5fe55a1ba9b9355ce26,Annex IV section 2(g),annex-iii-high-risk,ready
TM-11-03,demo-system 2.4.0,art-11,"Annex IV section 7: each listed standard maps to the requirement it addresses, with its stage",test,PR-11-DOC step 17,Standards and specifications list with requirement mapping,evidence/art-11/standards-list.md,8bf25baff17410bc7b86c76c1407b45ca911b377bc5f9b70378754f331b3b103,Annex IV section 7,annex-iii-high-risk,draft
TM-11-04,demo-system 2.4.0,art-11,"Annex IV section 5: detailed description of the risk management system under Art. 9",document,PR-11-DOC step 19,Annex IV section 5 draft with source references,,,Annex IV section 5,annex-iii-high-risk,missing
TM-9-01,demo-system 2.4.0,art-9,"Art. 9(8): each test in the risk management file runs against a prior defined metric and probabilistic threshold",test,PR-15-ROB threshold-declaration step,Prior defined metrics and probabilistic thresholds declaration,evidence/art-9/threshold-declaration-2.4.0.md,0dd2ba0f41667a1849e8736c2c14bbdd3029ebda357c5b8af16548780a4105b2,Annex IV section 5,annex-iii-high-risk,ready
TM-9-02,demo-system 2.4.0,art-9,"Art. 9(5): the residual risk of each hazard is compared with the acceptance criterion recorded for that hazard",test,PR-17-QMS acceptance-criteria step,Residual risk acceptance record per hazard and overall,evidence/art-9/residual-risk-record.csv,a50852451c934ced618dd5eb810c9fb2c3883aed46aeec5a679f50023da2eb1a,Annex IV section 5,annex-iii-high-risk,draft

The requirement cell quotes the element as the obligation record states it, with its Annex IV section or Art. 9 paragraph in front. The test cell of a test row names the protocol and the step; the browser tool writes the first protocol in the record's procedures list, and the hand-built file narrows that to the step, which for the Art. 9 rows is the threshold-declaration step of PR-15-ROB v1.0 and the acceptance-criteria step of PR-17-QMS v1.0, as the Art. 9 record's threshold note names them. A row with status missing has an empty path and an empty hash; the check in step 9 treats a path on a missing row as a failure, because a path without a status that claims the file is a row nobody has read.

Example for step 5. The frozen input set is the list of evidence file versions with their digests. The command writes one line per file; the digest in the matrix row is copied from this list, never computed from a file at a later date, so that a later change to the file shows up as a mismatch in step 9.

# Synthetic example: freeze the evidence files the matrix rows point to.
sha256sum evidence/art-11/*.csv evidence/art-11/*.md evidence/art-9/*.csv evidence/art-9/*.md \
  > frozen-input-set.txt
cat frozen-input-set.txt

Example for step 9. The script reads the matrix, and for each row with status ready or draft checks that the evidence path exists and that its digest equals the evidence_sha256 cell. A row with status missing or not-applicable passes only when its path is empty. The script prints one line per failing row and exits with code 1 when any row fails, so that it can run in a repository check before the matrix is filed.

# Synthetic example: check that every evidence path in the matrix exists and its hash matches.
import csv
import hashlib
import sys

CHECKED = {"ready", "draft"}


def sha256(path):
    h = hashlib.sha256()
    with open(path, "rb") as f:
        for chunk in iter(lambda: f.read(65536), b""):
            h.update(chunk)
    return h.hexdigest()


failures = []
with open("traceability-matrix.csv", newline="", encoding="utf-8") as f:
    for row in csv.DictReader(f):
        status = row["status"]
        path = row["evidence_path"].strip()
        if status not in CHECKED:
            if path:
                failures.append((row["id"], "path given on a row with status " + status))
            continue
        if not path:
            failures.append((row["id"], "no evidence path on a row with status " + status))
            continue
        try:
            digest = sha256(path)
        except FileNotFoundError:
            failures.append((row["id"], "missing file " + path))
            continue
        if digest != row["evidence_sha256"].strip().lower():
            failures.append((row["id"], "hash mismatch for " + path))

for matrix_id, reason in failures:
    print(f"{matrix_id}\t{reason}")
print(f"{len(failures)} failing row(s)", file=sys.stderr)
sys.exit(1 if failures else 0)

Run against the six synthetic rows with the six placeholder files in place, the script prints 0 failing row(s) and exits 0. After one byte is appended to the residual risk record, it prints TM-9-02 hash mismatch for evidence/art-9/residual-risk-record.csv and exits 1; the row stays at draft until the file is re-frozen in step 5 and the hash cell is updated.

Example for step 10. jq reads JSON, so the one-line python call converts the CSV first. The filter groups the rows by article and counts the open rows, which are the rows at missing or draft, next to the ready rows and the rows recorded as not applicable. The output is the coverage summary per article that the deliverable specification names as its second section.

# Synthetic example: open rows per article, where open means status missing or draft.
python3 -c "import csv, json, sys; json.dump(list(csv.DictReader(open('traceability-matrix.csv', newline='', encoding='utf-8'))), sys.stdout)" \
  | jq 'group_by(.article)
        | map({ article: .[0].article,
                rows: length,
                open: map(select(.status == "missing" or .status == "draft")) | length,
                ready: map(select(.status == "ready")) | length,
                not_applicable: map(select(.status == "not-applicable")) | length })'

For the six synthetic rows the filter returns two objects: art-11 with four rows, two open and two ready, and art-9 with two rows, one open and one ready. The counts are outputs of the example rows and no count is a requirement of the Act.

Example for step 11. The change log of the matrix is separate from the lifecycle change record of the system under Annex IV section 6: the first records versions of the index, the second records changes to the system. One synthetic entry:

# Synthetic example: change log of the matrix itself, one entry per matrix version.
- matrix_version: 3
  system_version: 2.4.0
  owner: documentation-control
  trigger: new PR-15-ROB test report for version 2.4.0 replaced the evidence on TM-11-01 and TM-9-01
  rows_changed: [TM-11-01, TM-9-01]
  frozen_input_set: frozen-input-set.txt

A pitfall that vendor documentation omits

Document management and GRC tooling describe a requirement as covered when a file is attached to it. A matrix built that way has one row per document: "Annex IV section 2(g): test-report.pdf", "Art. 9: risk-management-file.docx". Art. 11(1) second subparagraph asks for documentation that demonstrates that the system meets each requirement of Chapter III Section 2, and Annex IV section 7 asks for a mapping from each requirement to the standard or the described solution that meets it; the unit of both sentences is the requirement element, not the file. A test report attached to section 2(g) can say nothing about the element "declared metric values match the test reports for the same version and conditions" when the report covers version 2.3.0 and the documentation declares values for version 2.4.0; the Art. 11 record names that gap as a common finding, together with a standards list that names draft standards without their stage and without the requirement each one addresses. On the Art. 9 side the record names a risk management file that describes measures without a test that shows their effect: one row "Art. 9: risk management file, present" hides that Art. 9(5) and Art. 9(8) each need a record of their own. The synthetic row below is what a document-level matrix looks like next to the rows in the example for steps 1 to 4.

id,article,requirement,evidence,status
TM-11-01,art-11,Annex IV section 2(g),test-report-final.pdf,covered
TM-9-01,art-9,Art. 9 risk management,risk-management-file.docx,covered

The element-level row fixes this by quoting the element as the obligation record states it and by pointing to the record that answers that element, with the version and the hash of the file. Step 20 of PR-11-DOC v1.0 then compares the declared value with the report row for the same version and conditions, and the matrix row carries the identifier of that consistency row, not the identifier of the report. Where a file answers several elements, it appears in several rows, one per element, each with the section or table inside the file that answers the element; the deliverable specification's location column exists for that reason.

Regulation (EU) 2024/1689 sets no numeric threshold for any check in this guide, and no published document sets a required coverage level for a matrix. The hierarchy is the one the obligation record states: no harmonised standard is cited in the Official Journal as of the anchor date, so no presumption of conformity attaches; the declaration structure for the metric rows comes from prEN 18229-2Draft on the standards page, at draft stage; the document control routine that keeps the matrix current comes from prEN 18286Published; the risk management records on the Art. 9 rows follow prEN 18228Enquiry, under enquiry, with ISO/IEC 23894:2023Published as the published reference process; the target for each declared value and each acceptance criterion is the provider's justified choice, recorded in the Annex IV section 4 statement and in the residual risk acceptance record.

What the result shows and what it does not

The matrix shows presence and traceability: for each element, whether a record exists, where it sits and that the file has not changed since it was frozen. It does not show that a declared value is correct; a row at ready whose evidence is a report the provider produced without an independent test is only as reliable as that report. It does not show that the documentation is clear and comprehensive to a national competent authority or a notified body, and a row at ready is not a pass under Art. 43; the file itself shows whether the test passed. The matrix works on a frozen input set, so a change to the system, the documentation or a test report after step 5 is outside the result until the affected rows are re-frozen, which is why step 11 records the update trigger. The result is evidence for the provider's own technical file under Annex IV; the decision whether the system meets Art. 11 or Art. 9 remains with the provider, their notified body where one is required, and their counsel. The technical documentation and evidence-pack preparation service delivers the matrix with the coverage summary and the change log, and its scope limits are on what we cannot do.

The two obligations, the protocol, the related guide on the test reports the Art. 9 rows point to, the tool and the hubs for the two roles this guide serves are listed below; every protocol on the site is on the methodology index.

Obligations: Art. 11
Art. 9
Protocols: PR-11-DOC v1.0
Related guides: Robustness testing with ISO/IEC TR 24029-1
ICP hubs: Enterprises
Startups

Every inquiry receives a reply within 24 hours. Scope and availability are discussed per request.

Ask about scope hello@aiassurancelabs.com