Our legal team sent us the AI Act: what do we test
Legal has forwarded Regulation (EU) 2024/1689 and asks engineering what is tested. For a provider whose system matches an Annex III use case, the answer is the high-risk set of Chapter III Section 2, Art. 9 to Art. 15 and Art. 17, from the Annex III application date in the regulation data Deferred. Regulation (EU) 2026/1744 moved that date and left the obligations unchanged. This page splits the set into tests, documents and processes, gives the sequence, and names the evidence each step leaves in the file.
Where this starts
The forwarded text is the whole Regulation, and the first sorting question is scope. A provider under Art. 3(3) develops the system or has it developed under its own name. A system is high-risk under Art. 6(2) when its intended purpose matches a use case in Annex III: recruitment screening under point 4, admission to education under point 3, creditworthiness evaluation under point 5(b), or the dispatch of emergency services under point 5(d). One of the four conditions in Art. 6(3) removes the classification when the system performs no profiling of natural persons, and Art. 6(4) requires the provider to document that assessment before placing the system on the market. The risk classifier runs the Art. 6 routes on the published logic; the Annex III provider hub carries the obligation set with its dates.
What engineering receives from that classification is the Section 2 set: a risk management system under Art. 9, data governance under Art. 10, technical documentation in the Annex IV structure under Art. 11, automatic logging under Art. 12, information to deployers under Art. 13, human oversight design under Art. 14, accuracy, robustness and cybersecurity under Art. 15, and a quality management system under Art. 17. Each obligation record on this site splits the article into three lists. A testable element has a procedure that runs on the system or its data and produces a result that meets a stated criterion or does not: accuracy on a held-out set at or above the declared level under Art. 15(1) and 15(3) is testable. A documentation element is met by a document with a stated content: the accuracy declaration in the instructions for use under Art. 15(3) is documentation. A process element is met by an activity repeated over the lifecycle: the continuous iterative risk management process under Art. 9(2) is process. Each obligation page lists the split for its article.
The second sorting question is where the pass criteria come from, and the Act gives almost none. Art. 9(8) requires testing against prior defined metrics and probabilistic thresholds and leaves the values to the provider; Art. 15(3) requires the provider to declare the accuracy level and metrics in the instructions for use without stating what that level has to be. No harmonised standard is cited in the Official Journal as of the anchor date, so no presumption of conformity is available and the first level of the source hierarchy is empty. The metric categories and the declaration structure in use today come from the JTC 21 drafts prEN 18229-2Draft for accuracy and robustness and prEN 18228Enquiry for risk management, with ISO/IEC TR 24029-1:2021Published for robustness assessment methods and NIST AI RMF 1.0Published for the metric selection rationale. None of these documents publishes a numeric threshold for a high-risk system. The provider sets each target, justifies it against the context of use and records the justification; the sequence below places that record before any test runs.
The sequence
The work runs in the order below, classification first, then the obligation split, then the tests, then the consistency check and the file.
Match the intended purpose of each function against Annex III and test the Art. 6(3) conditions.
Record High-risk classification record and Art. 6(3) assessment record per function
[Regulation (EU) 2024/1689, Art. 6(2), 6(3) and Annex III; PR-CLS steps 13 to 17]
Compile the obligation register: each Section 2 element marked testable, documentation or process.
Record Obligation register with a regulation entry id per date and a protocol per testable element
[PR-CLS steps 39 to 41]
Define each metric, its threshold and the justification against the context of use before any run.
Record Prior defined metrics and probabilistic thresholds declaration (Art. 9(8))
[Regulation (EU) 2024/1689, Art. 9(8) and Art. 15(3); PR-15-ROB steps 8 to 9]
Run the held-out, shifted, perturbation, fault-injection and Art. 15(5) attack sets on the frozen version.
Record Accuracy, robustness, and adversarial and cybersecurity test reports (PR-15-ROB output)
[Regulation (EU) 2024/1689, Art. 15(1), 15(4) and 15(5); PR-15-ROB steps 14 to 30]
Disaggregate performance by protected and context-relevant subgroup on the testing data set.
Record Bias examination report with subgroup disaggregation tables (PR-10-BIAS output)
[Regulation (EU) 2024/1689, Art. 10(2)(f); PR-10-BIAS]
Reconstruct a sampled decision from the logs alone and verify override, reversal and stop.
Record Decision reconstruction record and override, reversal and stop verification report
[Regulation (EU) 2024/1689, Art. 12(2) and Art. 14(4); PR-12-LOG and PR-14-OVR]
Compare each declared value in the instructions for use with the test result for the same version.
Record Declared metric to test report consistency table (PR-11-DOC output)
[Regulation (EU) 2024/1689, Art. 13(3)(b)(ii); PR-11-DOC steps 20 to 26]
Sample the quality management records against each Art. 17(1) aspect and the stated frequency.
Record QMS gap analysis report (PR-17-QMS output)
[Regulation (EU) 2024/1689, Art. 17(1)(a) to (m); PR-17-QMS steps 9 to 23]
Assemble the Annex IV sections from the test reports and trace each requirement to its evidence.
Record Annex IV technical documentation set with the traceability matrix
[Regulation (EU) 2024/1689, Art. 11(1) and Annex IV; PR-11-DOC steps 43 to 45]
Evidence this situation ends with
| Artifact | Answers | Produced by |
|---|---|---|
| Prior defined metrics and probabilistic thresholds declaration (Art. 9(8)) | Art. 9(8): the metric and threshold recorded before the test run, with the reason each fits the intended purpose | the provider |
| Accuracy test report on held-out and distribution-shifted sets | Art. 15(1) and 15(3): accuracy at or above the declared level and consistent across versions | PR-15-ROB v1.0 |
| Adversarial and cybersecurity test report covering the Art. 15(5) attack list | Art. 15(5): resistance to each named attack class, with the threat model | PR-15-ROB v1.0 |
| Bias examination report with subgroup disaggregation tables | Art. 10(2)(f): performance and outcomes per subgroup, each metric defined | PR-10-BIAS v1.0 |
| Decision reconstruction record | Art. 12(2): input, model version, configuration, output and timestamp recovered from the logs alone | PR-12-LOG v1.0 |
| Override, reversal and stop verification report | Art. 14(4)(d) and (e): the recorded decision carries the person's choice, and the stop reaches the declared safe state | PR-14-OVR v1.0 |
| Declared metric to test report consistency table | Art. 13(3)(b)(ii) and Annex IV section 2(g): each declared value matches the dated and signed report for the same version | PR-11-DOC v1.0 |
| QMS gap analysis report | Art. 17(1)(a) to (m): each aspect marked conforms, partial or gap, with the sampled record | PR-17-QMS v1.0 |
Each artifact follows a specification with a synthetic sample on the deliverables page: the test reports follow the test report specification, the consistency table and the traceability rows follow the traceability matrix specification, and the gap report follows the findings register specification.
If nothing is done
A high-risk system placed on the market or put into service after the Annex III application date without the Section 2 evidence is subject to the market surveillance procedure of Art. 79: the market surveillance authority evaluates the system and, where it finds that the requirements of Chapter III are not met, requires the provider to take corrective action, to withdraw the system or to recall it within a period the authority sets, under Art. 79(2). A finding against the provider obligations in Art. 16, which include the Section 2 requirements and the Annex IV documentation, falls in the penalty tier of Art. 99(4), and the amount in a given case is set by the national authority under the Art. 99(7) criteria. The provider's action under Art. 20 is the same corrective action, taken on its own initiative once it has reason to consider that the system does not meet the requirements. The sequence above runs before the conformity assessment under Art. 43 and produces the information and documentation that Art. 21(1) requires the provider to supply on a reasoned request from a competent authority.
Services, guides and tools for this situation
The classification and readiness assessment produces the obligation register that scopes the rest, within the limits stated on what we cannot do; the robustness and adversarial testing service runs the Art. 15 tests under PR-15-ROB v1.0, and the technical documentation service runs the consistency check and the Annex IV assembly under PR-11-DOC v1.0.
- Services
- Guides
- ICP hubs
Questions
What does "high-risk AI" mean under the EU AI Act?
High-risk is a classification under Art. 6 of Regulation (EU) 2024/1689, and it determines which obligations apply to the system. A high-risk system carries Chapter III Section 2: risk management under Art. 9, data governance under Art. 10, technical documentation under Art. 11 and Annex IV, logging under Art. 12, information to deployers under Art. 13, human oversight under Art. 14, accuracy, robustness and cybersecurity under Art. 15, and a quality management system under Art. 17. A market surveillance authority can review the classification under Art. 79.
Which AI systems are high-risk under the EU AI Act?
Art. 6 of Regulation (EU) 2024/1689 sets two routes. Under Art. 6(1), a system is high-risk when it is a product, or a safety component of a product, listed in Annex I and subject to third-party conformity assessment. Under Art. 6(2), a system is high-risk when its intended purpose matches an Annex III use case, such as recruitment screening under point 4, unless one of the four Art. 6(3) conditions applies and the system performs no profiling of natural persons. PR-CLS steps 12 to 16 record each route per function.
Every inquiry receives a reply within 24 hours. Scope and availability are discussed per request.