EU AI Act Article 12 logging: verifying log completeness

What the obligation asks for

Art. 12(1) of Regulation (EU) 2024/1689 requires a high-risk AI system to technically allow the automatic recording of events over its lifetime. Art. 12(2) requires the recorded events to give a level of traceability appropriate to the intended purpose, for three purposes: identifying a situation that may present a risk within the meaning of Art. 79(1) or a substantial modification, post-market monitoring under Art. 72, and monitoring by the deployer under Art. 26(5). Art. 12(3) adds four minimum items for the remote biometric identification systems in Annex III point 1(a): the period of each use, the reference database checked, the input data that led to a match, and the identification of the natural persons who verified the results under Art. 14(5). The Art. 12 obligation page lists every element with its split. The duty applies from the Annex III application date 2027-12-02Deferred for systems classified under Art. 6(2).

The guide covers six testable elements of the obligation record: events recorded automatically without operator action (Art. 12(1)); logging that continues with the same event set and format after an update (Art. 12(1)); a sampled decision reconstructed from the logs alone with its input, model version, configuration, output and timestamp (Art. 12(2)); a log event for each risk indicator and for each candidate substantial modification (Art. 12(2)(a)); logs readable and exportable by the deployer (Art. 12(2)(c)); and, for Annex III point 1(a) systems, the four Art. 12(3) fields. Retention sits outside Art. 12: Art. 19 sets the provider's period and Art. 26(6) the deployer's, each at least six months unless other Union or national law provides otherwise. The Act names no event list for systems outside Annex III point 1(a); the expected event set comes from the two draft standards on the standards page, read against the intended purpose.

Inputs

The inputs below are the subset of the PR-12-LOG v1.0 input table that this guide uses. The full table is on the protocol page.

• The log schema and event catalogue from the technical documentation, Annex IV section 3, as JSON Schema, Avro, Protobuf or a table with the event list.
• The instructions for use with the intended purpose and the log mechanism description under Art. 13(3)(f).
• The risk management file under Art. 9 with one row per risk indicator and its condition.
• The retention policy with the claimed period per log class and the legal basis for any period that differs from Art. 19 or Art. 26(6).
• Read access to the log store and to the request log of the gateway or API layer in front of the system.
• A test environment that runs the version under test and accepts one model, software or configuration update.
• Read access to the system output store as the record of truth for the reconstruction comparison.
• For Annex III point 1(a) systems: a test reference database, a matching input and an assigned verifier role under Art. 14(5).

Procedure

Each step names the protocol step or the standard clause it derives from and the record it produces. Code blocks follow the step they belong to. Every path, identifier, count and digest in the examples is synthetic.

1. Collect the log schema and event catalogue from the technical documentation and record the version.

   Record log-schema-input: document reference, version, storage location

   [Regulation (EU) 2024/1689, Annex IV section 3; PR-12-LOG step 3]

2. Build the expected event set from the reference event model and the purposes in Art. 12(2)(a) to (c).

   Record expected-event-set: one row per event with purpose, trigger and fields

   [prEN ISO/IEC 24970, approval, clause on the logging information model; PR-12-LOG step 6]

3. Assign to each expected event its fields, including the four Art. 12(3) fields for Annex III point 1(a) systems.

   Record expected-event-set: field list per event, source clause per field, Art. 12(3) fields where they apply

   [prEN 18229-1, enquiry, clause on log content and fields; Regulation (EU) 2024/1689, Art. 12(3); PR-12-LOG steps 7 and 13]

4. Run the test input set through the system with no operator action on logging and export the run log.

   Record run-log-A: exported events for the run, run identifier, version under test

   [Regulation (EU) 2024/1689, Art. 12(1); PR-12-LOG step 14]

5. Validate each exported decision line against the schema and record each line with a missing reconstruction field.

   Record completeness-table-A: decision lines with a missing reconstruction field, each line listed

   [prEN ISO/IEC 24970, approval, clause on log record structure; PR-12-LOG steps 15 and 30]

6. Join the request log to the decision log by correlation identifier and record each request without a decision event.

   Record completeness-table-A: requests without a decision event, decision events without a request

   [prEN 18229-1, enquiry, clause on completeness of recorded events; PR-12-LOG step 15]

7. Apply the update in the test environment, repeat step 4, and compare the event set and format against run-log-A.

   Record run-log-B and continuity-table: events and fields before and after the update, differences listed

   [Regulation (EU) 2024/1689, Art. 12(1); prEN ISO/IEC 24970, approval, clause on logging across the AI system lifecycle; PR-12-LOG steps 16 and 17]

8. Change the model version, one threshold and one configuration value, and record the event each change produces.

   Record modification-event-table: change applied, event observed, candidate substantial modification marker present yes or no

   [Regulation (EU) 2024/1689, Art. 12(2)(a); PR-12-LOG step 19]

9. Select the reconstruction sample with the recorded method and rebuild each decision from the logs alone.

   Record sample-list and reconstruction-record: one row per sampled decision, each of the five elements found or missing

   [prEN ISO/IEC 24970, approval, clause on traceability of AI system events; PR-12-LOG steps 29 and 30]

10. Compare each reconstructed decision against the output store and classify each gap by the five gap classes.

    Record gap-classification (PR-12-LOG steps 32 and 33): gap identifier, class (missing event, missing field, unreadable format, broken linkage or retention loss), affected element, affected step

    [AIAssuranceLabs procedure]

11. Read the retention configuration of each log store and locate the oldest retained record in each store.

    Record retention-config-record and retention-observation: store, configured period, deletion rule, oldest record timestamp

    [Regulation (EU) 2024/1689, Art. 19 and Art. 26(6); prEN 18229-1, enquiry, clause on log retention; PR-12-LOG steps 24 and 25]

12. Write the completeness report with one row per expected event, one row per gap class and one retention row.

    Record Log completeness and continuity test report (PR-12-LOG output, step 38)

    [AIAssuranceLabs procedure]

Example for step 3. The schema below describes one log line in JSON Lines format. The five elements the protocol reconstructs under Art. 12(2) are input_ref, model_version, config_hash, output and ts, and the schema requires the first four on every decision.issued line; the two hash fields carry the tamper-evidence chain. The verifier_ids array is required on the result.verified event of an Annex III point 1(a) system, so it appears as a conditional branch. The schema is synthetic.

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "title": "log line (synthetic)",
  "type": "object",
  "required": ["event_id", "ts", "event_type", "source_id", "correlation_id",
               "prev_hash", "record_hash"],
  "properties": {
    "event_id":        { "type": "string", "pattern": "^evt-[0-9]{6,}$" },
    "ts":              { "type": "string", "format": "date-time" },
    "event_type":      { "enum": ["decision.issued", "risk.indicator.fired",
                                  "config.changed", "result.verified"] },
    "source_id":       { "type": "string" },
    "correlation_id":  { "type": "string", "pattern": "^req-[0-9]{6,}$" },
    "input_ref":       { "type": "string", "pattern": "^sha256:[0-9a-f]{64}$" },
    "model_version":   { "type": "string", "minLength": 1 },
    "config_hash":     { "type": "string", "pattern": "^sha256:[0-9a-f]{64}$" },
    "output":          {},
    "prev_hash":       { "type": "string", "pattern": "^sha256:[0-9a-f]{64}$" },
    "record_hash":     { "type": "string", "pattern": "^sha256:[0-9a-f]{64}$" },
    "session_id":      { "type": "string" },
    "reference_db_id": { "type": "string" },
    "verifier_ids":    { "type": "array", "items": { "type": "string" }, "minItems": 1 }
  },
  "allOf": [
    { "if":   { "properties": { "event_type": { "const": "decision.issued" } } },
      "then": { "required": ["input_ref", "model_version", "config_hash", "output"] } },
    { "if":   { "properties": { "event_type": { "const": "result.verified" } } },
      "then": { "required": ["session_id", "reference_db_id", "verifier_ids"] } }
  ]
}

Example for step 5. The jq filter lists every decision line in the export from step 4 that lacks one of the three reconstruction fields the schema does not already require on every line (ts is required on every line, output by the conditional branch). A full schema validation with a JSON Schema validator replaces this filter where one is installed; the filter is the minimum check.

# Synthetic example: run-log-A.jsonl is the export from step 4.
jq -c 'select(.event_type == "decision.issued")
       | select(.model_version == null or .input_ref == null or .config_hash == null)
       | { event_id, correlation_id,
           missing: [ (if .model_version == null then "model_version" else empty end),
                      (if .input_ref     == null then "input_ref"     else empty end),
                      (if .config_hash   == null then "config_hash"   else empty end) ] }' \
   run-log-A.jsonl > missing-fields.jsonl
wc -l missing-fields.jsonl

Example for step 6. The request log comes from the gateway or API layer in front of the system; the decision log comes from the system. A request with no decision event is a gap of class "missing event". A decision event with no request is either an internal call or a correlation identifier that was not propagated, and the reason is recorded in the completeness table.

# Synthetic example: request-log.jsonl from the gateway, run-log-A.jsonl from the system.
jq -r 'select(.event_type == "request.received") | .correlation_id' request-log.jsonl | sort -u > requests.txt
jq -r 'select(.event_type == "decision.issued")  | .correlation_id' run-log-A.jsonl   | sort -u > decisions.txt
comm -23 requests.txt decisions.txt > missing-decisions.txt   # request, no decision event
comm -13 requests.txt decisions.txt > orphan-decisions.txt    # decision event, no request
wc -l requests.txt decisions.txt missing-decisions.txt orphan-decisions.txt

Example for step 9. The script reads the sample list and the export, rebuilds each sampled decision from the log lines with the same correlation identifier, checks the five elements, and recomputes the record hash with the canonical serialisation the schema documents. The hash check is the integrity test of the protocol applied to the sampled lines; where the producer serialises differently, the recomputed hash fails for that reason alone, and the difference goes into the deviation log.

# Synthetic example: rebuild sampled decisions from the log export alone.
import hashlib
import json
import sys

REQUIRED = ("input_ref", "model_version", "config_hash", "output", "ts")


def load(path):
    with open(path, encoding="utf-8") as f:
        return [json.loads(line) for line in f if line.strip()]


def record_hash(rec):
    body = {k: v for k, v in rec.items() if k != "record_hash"}
    canonical = json.dumps(body, sort_keys=True, separators=(",", ":")).encode("utf-8")
    return "sha256:" + hashlib.sha256(canonical).hexdigest()


events = load("run-log-A.jsonl")
with open("sample-list.txt", encoding="utf-8") as f:
    sample = [line.strip() for line in f if line.strip()]

by_correlation = {}
for e in events:
    by_correlation.setdefault(e.get("correlation_id"), []).append(e)

rows = []
for decision_id in sample:
    decisions = [e for e in by_correlation.get(decision_id, [])
                 if e.get("event_type") == "decision.issued"]
    if not decisions:
        rows.append({"decision_id": decision_id, "result": "gap: missing event"})
        continue
    rec = decisions[-1]
    missing = [k for k in REQUIRED if rec.get(k) in (None, "")]
    integrity = "ok" if record_hash(rec) == rec.get("record_hash") else "hash mismatch"
    if not missing and integrity == "ok":
        result = "reconstructed"
    else:
        result = "gap: " + (", ".join(missing) if missing else integrity)
    rows.append({
        "decision_id": decision_id,
        "found": [k for k in REQUIRED if k not in missing],
        "missing": missing,
        "integrity": integrity,
        "source_records": [e.get("event_id") for e in by_correlation[decision_id]],
        "result": result,
    })

json.dump(rows, sys.stdout, indent=2)

Example for step 12. The completeness report has one row per expected event, one row per gap class the analysis found, and one retention row. Every count below is a synthetic output of the run in the examples above; no count is a requirement of the Act.

The five gap classes are those of the protocol: missing event, missing field, unreadable format, broken linkage and retention loss. A report row carries the affected element and the protocol step, so that the logging design review record and the test report can be traced to the observation that produced each finding.

A pitfall that vendor documentation omits

Observability documentation describes a log as complete when every request produces a line with a status and a response body. Art. 12(2) asks for traceability of the functioning of the system, and the obligation record names the first common gap: logs record the output but not the model version and configuration that produced it, so a decision cannot be reconstructed after a retrain. A line that carries the output alone passes the request-to-decision join in step 6 and fails reconstruction in step 9, because the model version and the configuration in force are two of the five elements the protocol reconstructs under Art. 12(2), with the record structure taken from prEN ISO/IEC 24970Approval, at approval stage. The same applies to the input: a line that stores the raw request body without a digest cannot be matched to an input store, and a line that stores a digest without a retained input store cannot be rebuilt into the input. The second case is the "broken linkage" class in the protocol: the digest proves which input was used only while the input store keeps the record for the same period the log store keeps the line, which for the deployer is the period in Art. 26(6) and for the provider the period in Art. 19. The synthetic line below is what such a log looks like next to the schema of step 3.

{ "ts": "<local server time, no offset>", "request_id": "req-000017",
  "status": 200, "response": { "score": 0.81, "decision": "refer" } }

For an Annex III point 1(a) system the gap has a third form. Art. 12(3)(d) requires the log to record the identification of the natural persons involved in the verification of the results under Art. 14(5), so a line that carries a boolean verified: true and no identifier does not meet the text. The verifier identification is a field on the result.verified event in step 3, and step 28 of PR-12-LOG v1.0 reads it after an assigned person verifies one result. A timestamp without a UTC offset, as in the line above, is the fourth form: Art. 12(3)(a) asks for the start and end of each use, and a local clock without a zone cannot bound a session across two components.

Regulation (EU) 2024/1689 sets no numeric threshold for any of these checks, and no published document sets a sample size for the reconstruction exercise. The hierarchy is the one the obligation record states: no harmonised standard is cited in the Official Journal as of the anchor date, so no presumption of conformity attaches; the event set and fields come from prEN 18229-1Enquiry, under enquiry, and prEN ISO/IEC 24970Approval, at approval stage; the traceability level is the provider's justified choice, which step 5 of the protocol records and step 34 tests; the sample size is recorded with its reason in step 10 of the protocol, because no published document sets one.

What the result shows and what it does not

The completeness report and the reconstruction record show what the logs recorded and what a reader could rebuild from them on the test date. The protocol cannot show that the event set is sufficient for a purpose the provider has not stated; step 34 of the protocol tests against the recorded traceability rationale only. The retention row observes one point in time and cannot show that records survive for the whole period. Steps 4 to 8 run in a test environment and cannot show that the production deployment records the same events; only the deployer export and the deployer-side reconstruction read production logs. The integrity check detects an alteration in a copy and cannot show that the production store rejects alteration by a privileged operator. The result is evidence for the provider's own technical file under Annex IV or the deployer's file under Art. 26(6); the decision whether the system meets Art. 12 remains with the provider or deployer, their notified body where one is required, and their counsel. The logging and human-oversight verification service runs the full protocol, and its scope limits are on what we cannot do.

Related pages

The obligation, the protocol and the hubs for the two roles this guide serves are listed below; every protocol on the site is on the methodology index.

Obligations

• Art. 12

Protocols

• PR-12-LOG v1.0

Related guides

• Override testing: what AI human oversight under Article 14 looks like in a test

ICP hubs

• Annex III providers
• Deployers

Questions

+−What does AI Act logging require of a high-risk system?

Art. 12(1) requires the system to record events automatically over its lifetime, and Art. 12(2) requires the events to give traceability appropriate to the intended purpose for risk identification, post-market monitoring and deployer monitoring. The Act names no event list outside Art. 12(3), which covers remote biometric identification. Retention sits in Art. 19 for the provider and Art. 26(6) for the deployer. A completeness test validates the schema, joins the request log to the decision log, and rebuilds sampled decisions from the logs alone.

Every inquiry receives a reply within 24 hours. Scope and availability are discussed per request.

Ask about scopehello@aiassurancelabs.com