Proving AI literacy under the AI Act: the Article 4 evidence file
This guide builds the evidence file for the AI literacy duty in Art. 4(1). For a provider, the resource and accountability steps of PR-17-QMS v1.0 sample the file under Art. 17(1); for a deployer, the human oversight assignment record under Art. 26(2) draws on it. The competence records follow the clauses of ISO/IEC 42001:2023Published, published. At the end the reader holds a role-to-competence matrix, an indexed set of evidence records, and a check that fails on any role without a record.
What the obligation asks for
Art. 4(1) of Regulation (EU) 2024/1689, as amended by Regulation (EU) 2026/1744, requires providers and deployers of AI systems to take measures to support the development of AI literacy of their staff and other persons dealing with the operation and use of AI systems on their behalf, taking into account their technical knowledge, experience, education and training and the context the AI systems are to be used in, and considering the persons or groups of persons on whom the AI systems are to be used. The amended text adds that no specific level of AI literacy of any individual is owed. The pre-Omnibus wording asked for measures towards a sufficient level, to the best extent of the provider or deployer; the Omnibus replaced that wording and added Art. 4(2) and 4(3), which task the Commission and the Member States with support for providers and deployers and the Board with recommendations. Art. 3(56) defines AI literacy as the skills, knowledge and understanding that allow providers, deployers and affected persons to make an informed deployment and to gain awareness of the opportunities and risks of AI and of the possible harm it can cause. The duty covers every AI system in the organisation, whatever its risk tier, and both roles. It applies from the Art. 4 application date In force; the amended wording took effect on the Omnibus entry into force In force. The Commission's published questions and answers on AI literacy state that national market surveillance authorities supervise Art. 4 from the general application date under Art. 113, the date on which Art. 50 also applies In force, and that any penalty is proportionate; Art. 4 is not among the provisions Art. 99(4) lists with a maximum fine, and Art. 99(1) leaves the penalty rules to the Member States. The same questions and answers state that no mandatory training and no formal proof document per person is required, and that an internal record of the training and other guiding initiatives is the expected form of evidence. Art. 4 has no obligation page on this site; the guide grounds each step in the article text and in the two obligations the literacy measures support.
For a deployer of a high-risk system, the Art. 26 record holds the elements the literacy measures feed. Art. 26(2) requires the deployer to assign human oversight to natural persons who have the necessary competence, training, authority and support; the record lists the assignment record with competence, training and authority as documentation, the support in the deployed workflow as a testable element verified in a walk-through, and the renewal of training and support when the system or the assigned persons change as a process element. Art. 14(4)(a) to (e) names what the oversight measures enable the assigned person to do: understand the capacities and limitations, remain aware of automation bias, interpret the output, decide not to use it or disregard it, and intervene or stop the system. The competence element for an oversight role is written against those five abilities. For a provider, Art. 17(1) places the measures in the quality management system: the resource management description under Art. 17(1)(l) and the accountability framework under Art. 17(1)(m), which the Art. 17 record lists as documentation, with the role walk-through under (m) as a testable element. An organisation that is provider of some systems and deployer of others keeps one evidence file and marks the role per system, as the enterprise hub describes.
Inputs
The inputs are the subset of the PR-17-QMS v1.0 input table that the resource management and accountability steps use, plus the records the literacy measures produce. The full table is on the protocol page.
- The AI system inventory with the organisation's role per system, provider or deployer, and the risk tier where classified.
- The role list: every function that operates, configures, procures, oversees or uses an AI system on the organisation's behalf, including contractors and agency staff.
- The human oversight assignment record for each deployed high-risk system, where one exists (Art. 26(2)).
- For a provider: the accountability framework and the resource management description of the quality management system (Art. 17(1)(l) and (m)).
- The instructions for use of each deployed system, which state the context of use and the persons or groups on whom the system is used.
- The records of measures already taken: attendance logs, learning-platform exports, briefing notes, guidance documents, onboarding checklists, each with a location and a system-side entry that carries its date.
- A shell with
pythonand the PyYAML parser installed.
Procedure
Each step names the article, clause or protocol step it derives from and the record it produces. Code blocks follow the step they belong to. Every role name, system name, record identifier, path and count in the examples is synthetic.
Record each AI system in the inventory with the organisation's role for that system.
Record scope record: system list with the organisation's role per system, provider or deployer
[Regulation (EU) 2024/1689, Art. 4(1), Art. 3(3) and Art. 3(4); PR-17-QMS step 1]
List every role that deals with the operation or use of each system, staff and other persons alike.
Record role inventory: role, system, relation to the organisation, context of use, persons or groups affected
[Regulation (EU) 2024/1689, Art. 4(1), 'staff and other persons dealing with the operation and use of AI systems on their behalf']
Record the technical knowledge, experience, education and training the organisation assumes for each role.
Record starting-point record per role: technical knowledge, experience, education and training assumed for the role
[Regulation (EU) 2024/1689, Art. 4(1), 'technical knowledge, experience, education and training']
Write one competence element per role from the system's context of use and the persons affected.
Record role-to-competence matrix: one row per role and system with the competence element and the evidence record
[Regulation (EU) 2024/1689, Art. 4(1) and Art. 3(56); the Commission's published questions and answers on AI literacy]
For each oversight role, cross-reference the assignment record and write the competence element against Art. 14(4).
Record oversight cross-reference: role, human oversight assignment record identifier, competence element against the five Art. 14(4) abilities
[Regulation (EU) 2024/1689, Art. 26(2) and Art. 14(4)(a) to (e)]
For a provider, map each measure to the resource management description and to a responsibility in the accountability framework.
Record QMS cross-reference: measure to the resource management entry and the accountability framework responsibility it supports
[Regulation (EU) 2024/1689, Art. 17(1)(l) and (m); ISO/IEC 42001:2023 §7.2 and Annex A control A.4.6; PR-17-QMS steps 22 and 23]
Give each measure an evidence record with an identifier, an owner role, a location and a date reference.
Record evidence record: identifier, measure type, roles covered, owner role, date reference, location
[ISO/IEC 42001:2023 §7.2 and §7.5, documented information retained as evidence of competence]
Build the evidence file index with one entry per record.
Record evidence file index: one entry per evidence record
[AIAssuranceLabs procedure; PR-17-QMS step 2, document inventory]
Run the coverage check and resolve every role that has no record or whose record has no date reference.
Record coverage check output: roles without a record, records without a date reference, exit code
[AIAssuranceLabs procedure; Regulation (EU) 2024/1689, Art. 4(1)]
Open one record per measure type from the index and record whether each is found at its location.
Record retrieval test record: record identifier, found or not found, location
[ISO/IEC 42001:2023 §7.5.3; PR-17-QMS step 21, retrieval test]
Record the triggers after which a role's competence element is reviewed and a new evidence record is required.
Record update trigger list: system change, instructions-for-use change, role change, new person in a role
[Regulation (EU) 2024/1689, Art. 26(2), renewal of training and support as restated in the Art. 26 record; ISO/IEC 42001:2023 §7.2]
File the index with a control block and a limits statement naming the records reviewed.
Record control block: index version, owner role, update triggers; limits statement
[prEN 18286, published, clause on document and record control; PR-17-QMS step 45]
Example for steps 2 to 5. The matrix has one row per role and system. The context_of_use and persons_affected columns come from the instructions for use of the deployed system or, for a provider's own system, from its intended purpose statement. The competence_element column states what the role needs to know for that context; for an oversight role it references the Art. 14(4) abilities. The evidence_record column holds the identifier of the record in the index, or is empty when no measure has been taken, which the check in step 9 reports. The roles, the system names and the identifiers are synthetic.
role,system,relation,context_of_use,persons_affected,competence_element,evidence_record
recruiter,cv-screen-3,staff,shortlisting of applicants for advertised roles,job applicants,"ranking output is a recommendation; known error modes; when to disregard the output",LIT-0007
hiring-manager,cv-screen-3,staff,final decision on shortlisted applicants,job applicants,"reads the ranking rationale; decides without the output where the rationale is absent",LIT-0007
oversight-officer,cv-screen-3,staff,human oversight assigned under Art. 26(2),job applicants,"Art. 14(4)(a) to (e): capacities and limitations; automation bias; interpretation; disregard; intervene and stop",LIT-0011
support-agent,helpdesk-assistant-2,agency,drafting replies to customer tickets,customers,"generated text is a draft; checks facts against the ticket; escalation route",LIT-0003
ml-engineer,cv-screen-3,staff,training and release of the provider's own system,job applicants,"data governance duties under Art. 10; the QMS procedures the role executes",
procurement-lead,helpdesk-assistant-2,staff,selection and contracting of vendor systems,customers,"which vendor statements to obtain; the deployer duties the contract allocates",LIT-0003
Example for steps 7 and 8. The index holds one entry per evidence record. No entry carries a date literal; the date_ref field points to the entry in the system of record that holds the date, so that the date is kept once and the index cannot drift from it. The type field takes one of the measure types the organisation defines; the four shown are examples. The owner is a role. The location is where the record is opened in the retrieval test of step 10.
# Synthetic example: ai-literacy-evidence-index.yaml
organisation_roles:
cv-screen-3: provider-and-deployer
helpdesk-assistant-2: deployer
records:
- id: LIT-0003
type: briefing
title: generative assistant use briefing for support and procurement
roles: [support-agent, procurement-lead]
systems: [helpdesk-assistant-2]
owner_role: customer-operations-lead
date_ref: hr-ledger#0412
location: evidence/literacy/LIT-0003-briefing-notes.pdf
- id: LIT-0007
type: training
title: shortlisting with a ranking model, error modes and the disregard rule
roles: [recruiter, hiring-manager]
systems: [cv-screen-3]
owner_role: talent-acquisition-lead
date_ref: lms-export#row-118
location: evidence/literacy/LIT-0007-attendance.csv
- id: LIT-0011
type: oversight-induction
title: oversight officer induction against Art. 14(4) for cv-screen-3
roles: [oversight-officer]
systems: [cv-screen-3]
owner_role: hr-systems-owner
oversight_assignment_ref: HO-ASSIGN-cv-screen-3-v2
date_ref: hr-ledger#0455
location: evidence/literacy/LIT-0011-induction-record.md
- id: LIT-0012
type: guidance
title: engineer guidance on Art. 10 data governance procedures
roles: [ml-engineer]
systems: [cv-screen-3]
owner_role: ml-platform-lead
location: evidence/literacy/LIT-0012-guidance.md
control:
index_version: 4
owner_role: ai-governance-lead
update_triggers:
- new AI system in the inventory or change of the organisation's role for a system
- change to the instructions for use or the intended purpose of a system
- new role dealing with a system, or a new person in an oversight role
- change to the human oversight assignment record
Example for step 9. The script reads the matrix and the index, and fails on three conditions: a role row with no evidence record, a role row whose record is not in the index or does not list the role, and a record without a date_ref. A record without a date reference cannot show that the measure was taken before the person started dealing with the system, so the check treats it as absent. The script prints one line per failure and exits with code 1 when any failure exists, so that it can run as a repository check. It needs the PyYAML parser.
# Synthetic example: check every role in the matrix has a dated-by-reference evidence record.
import csv
import sys
import yaml
with open("ai-literacy-evidence-index.yaml", encoding="utf-8") as f:
index = yaml.safe_load(f)
records = {r["id"]: r for r in index.get("records", [])}
failures = []
rows = 0
with open("role-competence-matrix.csv", encoding="utf-8", newline="") as f:
for row in csv.DictReader(f):
rows += 1
key = "%s/%s" % (row["role"], row["system"])
rid = (row.get("evidence_record") or "").strip()
if not rid:
failures.append((key, "no evidence record"))
continue
rec = records.get(rid)
if rec is None:
failures.append((key, "record %s is not in the index" % rid))
continue
if row["role"] not in rec.get("roles", []):
failures.append((key, "record %s does not list this role" % rid))
if row["system"] not in rec.get("systems", []):
failures.append((key, "record %s does not list this system" % rid))
if not rec.get("date_ref"):
failures.append((key, "record %s has no date reference" % rid))
for rid, rec in records.items():
if not rec.get("date_ref"):
failures.append((rid, "record has no date reference"))
for key, reason in failures:
print("FAIL %-40s %s" % (key, reason))
print("%d role rows checked; %d failures" % (rows, len(failures)))
sys.exit(1 if failures else 0)
On the six rows and four records above the script reports two failures: the ml-engineer row has no evidence record, and LIT-0012 has no date reference. Both counts are outputs of this example, never a requirement of the Act. Resolving them means taking the measure for the engineer role and entering the guidance record in the system of record that carries its date, then re-running the check.
A pitfall that vendor documentation omits
Learning-platform documentation presents Art. 4 as a course assignment: one AI awareness module, assigned to every employee, with a completion percentage on a dashboard. Art. 4(1) asks for measures that take into account the technical knowledge, experience, education and training of the persons, the context the systems are to be used in, and the persons or groups on whom the systems are used. A completion record per employee carries none of those three: it does not say which system the person deals with, in which context, or who is affected by its output. The Commission's published questions and answers on AI literacy state the same in their own terms, that no single format fits every organisation and that the measures are tailored to the role, the knowledge of the staff and the context of use; they also state that no formal proof document per person is required, so the dashboard's completion figure is not the evidence the article describes. The role-to-competence matrix in step 4 records the three factors per row, and a module assigned to everyone is one evidence record among others, listed against the roles whose competence element it covers.
The second gap is the boundary of the population. Art. 4(1) covers staff and other persons dealing with the operation and use of AI systems on the organisation's behalf. A learning platform is fed from the employee directory, so contractors, agency staff and the vendor's on-site operators are outside it; the relation column in the matrix records the employment relation so that those persons are in the inventory. The third gap is the oversight role. Art. 26(2) requires the deployer to assign human oversight to persons with the necessary competence, training, authority and support, and the Art. 26 record names the finding that oversight is assigned on an organisation chart without a record of the competence, training or authority of the named persons. A general awareness module is not evidence of competence against the five Art. 14(4) abilities; step 5 writes the competence element for an oversight role against those abilities and cross-references the assignment record, so that the oversight verification under PR-14-OVR v1.0 and the literacy file name the same persons.
Regulation (EU) 2024/1689 sets no level for AI literacy. The pre-Omnibus text asked for a sufficient level to the best extent of the provider or deployer; the amended Art. 4(1) asks for measures that support development and states that no specific level of any individual is owed. Neither version defines a curriculum, a duration or a pass mark, and Art. 4(2) and 4(3) place practical examples and recommendations with the Commission and the Board rather than in the article. The hierarchy for what a measure contains is the one the obligation records state. No harmonised standard is cited in the Official Journal as of the anchor date, so no presumption of conformity exists for any literacy measure. For a provider's quality management system, prEN 18286Published, published and not cited in the Official Journal, supplies the clause structure of the resource management description and the accountability framework; the standards page shows its stage. ISO/IEC 42001:2023Published, published, supplies the competence clause §7.2, the awareness clause §7.3 and the Annex A control A.4.6 on human resources, and requires documented information retained as evidence of competence, which is the form the evidence records in step 7 take. The Commission's published questions and answers on AI literacy sit at the guideline level and give the content areas a measure covers: a general understanding of AI and its opportunities and risks, the organisation's role as provider or deployer, the risks of the systems it uses, and measures tailored to the staff and the context. NIST AI RMF 1.0Published, published, supplies the Govern function where an organisation reuses its workforce and accountability artifacts. The competence element per role, and the depth of each measure, are the organisation's justified choice, and the starting-point record in step 3 holds the justification.
What the result shows and what it does not
The matrix, the index and the check show coverage and traceability: for each role that deals with an AI system, that a measure was taken, that the record of it exists at its location, that the record carries a date by reference, and, for an oversight role, that the record and the Art. 26(2) assignment record name the same persons. They do not show that any person understands the system, interprets its output correctly or would intervene when needed; a record of a measure is evidence of the measure, and the amended Art. 4(1) states that no specific level of any individual is owed. The PR-17-QMS v1.0 limitations apply: the review reads documents and sampled records at one point, and cannot show that the measures were applied between the sampled records or that they will be renewed after the review. The walk-through of an oversight role under Art. 26(2) is a separate test under PR-14-OVR v1.0, and this guide does not replace it. The result is evidence for the organisation's own file; whether the measures are adequate for the context of use is a decision that remains with the provider or deployer and their counsel. The QMS gap analysis service samples the resource management and accountability records of a provider's quality management system, its scope limits are on what we cannot do, and the findings register it delivers holds the gap rows for Art. 17(1)(l) and (m).
Related pages
The two obligations the literacy measures support, the protocol whose resource and accountability steps sample the file, the related guides on the Art. 17 management system and on the provider and deployer roles, and the hubs for the two roles this guide serves are listed below.
- Protocols
- Related guides
- ICP hubs
Every inquiry receives a reply within 24 hours. Scope and availability are discussed per request.