Logging and human-oversight verification

Deliverables

The four deliverables answer what an assessor or a market surveillance authority asks for under Art. 12 and Art. 14, and what a deployer keeps under Art. 26(6): a log design reviewed against the minimum content of the article, a test report on completeness, integrity and retention, a reconstruction record with a synthetic sample, and a human oversight verification report. The specifications and synthetic samples are on the deliverables page, under assessment memo for the design review and test report for the other three.

Who this is for

• Annex III providers
• Deployers

A provider of an Annex III system, for example recruitment screening, credit scoring, education admission or emergency-service triage, carries the logging design under Art. 12 and the oversight design under Art. 14 as part of the Chapter III Section 2 set, and neither article sets a numeric threshold. A deployer that bought or licensed such a system and uses it in its own decisions holds the operational side under Art. 26: use within the instructions for use, oversight assigned to persons with the competence and authority to act, and the logs under its control retained for the period in Art. 26(6). The provider's evidence shows the design; the deployer's evidence shows that an assigned person can stop or override a decision and that the retained logs allow a decision to be rebuilt. The scope limits of the service are stated on what we cannot do.

The duties of both roles follow the Annex III application date for systems classified under Art. 6(2); the entry below carries the date and its source.

• 2027-12-02DeferredAnnex III standalone high-risk obligations

Scope

The engagement covers one high-risk AI system version and, where the client is a deployer, one deployment of it. On the logging side the scope is the event catalogue and the fields per event, automatic recording during normal operation, continuity after a model, software or configuration update, the tamper-evidence mechanism of the log store, the configured retention period, and the reconstruction of sampled decisions from the logs alone on the provider side and on the deployer side. On the oversight side the scope is the human-machine interface tools, the visibility of an injected anomaly to the assigned person, the measure against automation bias, the interpretation tools, the decision not to use and the override or reversal of an output, the stop function and its declared safe state, and, for a remote biometric identification system in Annex III point 1(a), the two-person verification under Art. 14(5). In a deployment the scope adds the elements of Art. 26(1) to (3) and Art. 26(5): use within the instructions for use, implementation of the provider's deployer-side measures, the authority and support of the assigned persons, and the suspension of use.

The expected event set and the field definitions for reconstruction come from prEN 18229-1Enquiry, which is under enquiry, and prEN ISO/IEC 24970Approval, which is at approval stage. The categories of oversight measures come from prEN 18229-3Draft, which is at drafting stage with no public text. None of the three is cited in the Official Journal, so no presumption of conformity attaches to any result; each document is evidence of the state of the art, and its stage and source are on the standards page.

The engagement does not cover the relevance and representativeness of input data under Art. 26(4), which PR-10-BIAS v1.0 covers. It does not test the content of the risk management file under Art. 9 or the post-market monitoring plan under Art. 72; both are inputs, and the logging test checks whether the logs supply what each document names. It does not assess the retention period a provider or deployer selects under other Union or national law; the protocol records the stated legal basis and tests whether the configuration matches the policy. It does not assess the competence and training of the persons assigned to oversight, which a record and a repeated activity show, and it does not decide whether a system falls under Annex III point 1(a) or whether the Art. 14(5) exception for law enforcement, migration, border control or asylum applies. The test reports are evidence for the provider's conformity assessment under Art. 43 or for the deployer's file; the decision whether the system meets Art. 12, Art. 14 or Art. 26 remains with the provider or deployer, their notified body where one is required, and their counsel.

A deployer that also performs the fundamental rights impact assessment under Art. 27, through FRIA facilitation and evidence structuring, can use the human oversight verification report as the description of oversight implementation that Art. 27(1)(e) asks for. The application date of that assessment is an unsettled point.

What we test

The table lists every testable element of the three obligation records, the protocol that produces the observable result, and the artifacts it produces for the record.

Regulation (EU) 2024/1689 sets no numeric performance threshold for Art. 12, Art. 14 or Art. 26. Each protocol takes its criterion from the highest level available in a six-level hierarchy: a harmonised standard cited in the Official Journal, of which none exists as of the anchor date; a JTC 21 draft standard, stated with its stage; an ISO/IEC standard or technical report; a Commission guideline or code of practice; NIST AI RMF or a sector framework; and the state of the art that the provider justifies and documents. For logging, the event set and the fields come from the second level, prEN 18229-1Enquiry and prEN ISO/IEC 24970Approval, read against the intended purpose in the instructions for use, and the scope step of PR-12-LOG v1.0 records the traceability level the provider justifies. For oversight, no published document states what counts as an effective override, stop or monitoring measure, so the provider's justification under Art. 14(3) is the criterion, and the rationale-recording step of PR-14-OVR v1.0 captures it. The retention period in Art. 26(6) is a legal period; the retention test compares the configured period against the policy and its legal basis.

Protocols used

Two published protocols cover the service, each at the version shown.

• PR-12-LOG v1.0Logging completeness and reconstruction protocolv1.0
• PR-14-OVR v1.0Human oversight verification protocolv1.0

PR-12-LOG v1.0 covers the logging design review, completeness and continuity, tamper evidence, retention and decision reconstruction under Art. 12 and Art. 26(6). It produces the logging design review record, the completeness, integrity and retention test report, the retention configuration evidence and the decision reconstruction record with one synthetic sample row. It cannot show that the event set is sufficient for a purpose the provider has not stated, that records survive the whole retention period, or that the production deployment records the same events as the test environment; only the deployer export and the deployer-side reconstruction read production logs.

PR-14-OVR v1.0 covers the seven testable elements of Art. 14 and five of Art. 26 in a scripted campaign on the tested system version and deployment configuration. It produces the human oversight verification report, the override, reversal and stop verification report, the automation bias measure record, the instructions-for-use conformance record and an oversight action log extract that feeds the log completeness test. It does not show that the assigned persons oversee the system effectively during live use, which varies in load, case mix and attention, and it does not assess their competence and training.

Engagement shape

The engagement is fixed-scope, scoped per request. The entry service, Classification and readiness assessment, is the usual starting point where the role of the organisation or the risk tier of the system is not yet established. The client provides the instructions for use with the log mechanism description under Art. 13(3)(f), the log schema and event catalogue from Annex IV section 3, the risk management file and the post-market monitoring plan, the retention policy with its legal basis, read access to the log store, a test environment with an update to apply, the oversight measure inventory and safe-state definition, and, for a deployer, the assignment record and the suspension procedure. The work follows the order of each protocol: preparation builds the expected event set and the test-case set from the inputs; execution runs the system with the update, the injected anomaly, the override, stop and suspension cases and the tamper test; analysis reconstructs the sampled decisions and classifies each gap by element and article; reporting assembles the four deliverables with the protocol deviation log and the handover manifest.

What we cannot do

• We do not perform conformity assessments where a notified body is required.
• We do not issue CE marking or the EU Declaration of Conformity.
• We are not a notified body.
• We do not give legal opinions on classification or liability.
• We test against requirements, prepare evidence, and structure documentation; compliance decisions remain with the provider or deployer, their notified body, and their counsel.

Read the full statement

For this service the test reports show what the logs record, what a reader can rebuild from them and which oversight measures had an observable effect on the test date. They are inputs to the provider's conformity assessment or the deployer's file, and no statement of conformity follows from them.

What the client must have ready

The documentation and process elements below are inputs to the tests and are recorded, not tested.

The log schema is accepted as JSON Schema, Avro, Protobuf or a table with the event list; the risk indicators and the post-market monitoring data points are accepted as one row per item with its condition or field definition; the assignment record names the persons with their competence, training and authority.

Questions

+−What does Article 12 of the EU AI Act require?

Art. 12(1) requires a high-risk AI system to technically allow the automatic recording of events over its lifetime. Art. 12(2) requires the logs to give traceability appropriate to the intended purpose for three purposes: identifying a risk within the meaning of Art. 79(1) or a substantial modification, post-market monitoring under Art. 72, and monitoring by the deployer under Art. 26(5). Art. 12(3) adds minimum log content for remote biometric identification systems. PR-12-LOG tests automatic recording, continuity after an update and reconstruction of a sampled decision from the logs alone.

+−What are the EU AI Act human oversight requirements?

Art. 14(1) requires the provider to design the system so that natural persons can oversee it effectively during use, with appropriate human-machine interface tools. Art. 14(4) names the capabilities of the assigned person: understand the capacities and limitations, monitor operation, stay aware of automation bias, interpret the output, decide not to use or override the output, and stop the system in a safe state. Art. 26(2) requires the deployer to assign oversight to natural persons with competence, training, authority and support. PR-14-OVR tests each capability for an observable effect.

+−How many staff do we need for human oversight?

Regulation (EU) 2024/1689 sets no headcount for human oversight. Art. 14(3) requires the provider to identify measures commensurate with the risks, the level of autonomy and the context of use, and Art. 26(2) requires the deployer to assign oversight to natural persons with the necessary competence, training, authority and support. The one count in the text is Art. 14(5): two natural persons separately verify a remote biometric identification before any action follows. PR-14-OVR tests whether an assigned person can stop or override the system and sets no number.

+−Does human oversight conflict with automation efficiency?

Art. 14(1) requires that natural persons can oversee the system effectively during use, and the text names no per-output review. Under Art. 14(3) the measures are commensurate with the risks, the level of autonomy and the context of use, so a high-autonomy design carries more built-in measures, for example a monitoring view in which an injected anomaly is visible and a stop that reaches a declared safe state. The measure against automation bias under Art. 14(4)(b) changes the presentation of an output. PR-14-OVR tests each measure for an observable effect.

+−What does ISO/IEC 24970 cover?

prEN ISO/IEC 24970, Logging and traceability, is at approval stage. The document supplies common capabilities, requirements and an information model for logging events in AI systems. PR-12-LOG uses the document as the reference event model for the expected event set and the field definitions a decision reconstruction needs, read together with prEN 18229-1, which is under enquiry. Neither document is cited in the Official Journal, so a test result rests on both as evidence of the state of the art and carries no presumption of conformity.

Every inquiry receives a reply within 24 hours. Scope and availability are discussed per request.

Ask about scopehello@aiassurancelabs.com