Bias and subgroup performance protocol (PR-10-BIAS)

PR-10-BIAS v1.0 is built from ISO/IEC TR 24027:2021Published, prEN 18283Draft, prEN 18284Draft, Art. 10(2) to Art. 10(4) of Regulation (EU) 2024/1689, and Art. 4a of Regulation (EU) 2024/1689, the sensitive-data provision that Regulation (EU) 2026/1744 moved from the former Art. 10(5).

Scope

The protocol applies to a high-risk AI system whose provider trains AI models with data, and to its training, validation and testing data sets. Where the system does not train AI models, Art. 10(6) applies the same criteria to the testing data sets only, and the protocol runs on those sets. The protocol also applies to a deployer of a high-risk AI system that controls the input data, for the representativeness duty in Art. 26(4). The Art. 10 duties apply from the Annex III application date 2027-12-02Deferred for Annex III systems; the Art. 4a legal basis applies from the Omnibus entry into force 2026-07-27In force.

The protocol covers the testable elements of Art. 10: representativeness, error rate and completeness, statistical properties, coverage of the setting of use, the bias examination by subgroup, the effect of mitigation measures, the feedback-loop check, and the observable Art. 4a(1) safeguards. It covers one testable element of Art. 26: the representativeness of deployer-controlled input data under Art. 26(4). The other testable elements of Art. 26 belong to PR-14-OVR v1.0 and PR-12-LOG v1.0.

The protocol does not cover the documentation and process elements of Art. 10, except where a step records the provider's evidence as an input. It does not decide whether processing of special-category data is lawful under Regulation (EU) 2016/679; it records what the provider states and what the pipeline shows. It does not select the protected attributes for the provider; it records the provider's selection and the reason for it. No step in the protocol is a conformity assessment; the scope limits are stated on what we cannot do.

Articles covered

Inputs required

Steps

Preparation

1. Record the system identifier, the model version and the intended-purpose statement from the instructions for use.

   Record Test scope record

   [AIAssuranceLabs procedure]

2. Record whether the system uses techniques that train AI models with data.

   Record Training-technique statement, which sets the data sets in scope under Art. 10(1) or Art. 10(6)

   [Regulation (EU) 2024/1689, Art. 10(1) and 10(6)]

3. List each training, validation and testing data set in scope and compute a hash of each file.

   Record Data set inventory with one file hash per data set

   [AIAssuranceLabs procedure]

4. Extract from the technical documentation the persons or groups the system is intended to be used for.

   Record Target population statement

   [Regulation (EU) 2024/1689, Art. 10(3)]

5. Extract the geographical, contextual, behavioural and functional setting of intended use from the technical documentation.

   Record Setting-of-use statement

   [Regulation (EU) 2024/1689, Art. 10(4)]

6. Extract the provider's assumptions about the information the data measure and represent.

   Record Assumptions statement

   [Regulation (EU) 2024/1689, Art. 10(2)(d)]

7. Define with the provider the protected attributes and the context-relevant subgroups for disaggregation.

   Record Subgroup definition table with the coding of each attribute value and the reason for each attribute

   [ISO/IEC TR 24027:2021 §5.3, fairness]

8. Classify the candidate bias sources for the system as human cognitive, data or engineering bias.

   Record Bias source register

   [ISO/IEC TR 24027:2021 §6, sources of unwanted bias]

9. Select the fairness metrics for the context of use and write the definition of each metric.

   Record Metric selection record with one definition per metric

   [ISO/IEC TR 24027:2021 §7.2, fairness metrics]

10. Record the provider's coverage, error, completeness and disparity targets with the justification for each target.

    Record Threshold justification record; the step the Art. 10 threshold note names

    [AIAssuranceLabs procedure]

11. Record whether special categories of personal data are present in any data set or pipeline stage.

    Record Special-category data presence statement

    [Regulation (EU) 2024/1689, Art. 4a(1)]

12. Where special-category data is present, obtain the records of processing activities and the safeguards description.

    Record Legal-basis input set

    [Regulation (EU) 2024/1689, Art. 4a(1)(f)]

Execution

13. Trace each data set to its collection source, collection process and original purpose of collection.

    Record Data set lineage record

    [prEN 18284, draft, clause on data collection and origin]

14. List each data-preparation operation applied to each data set in the order applied.

    Record Preparation operations log covering annotation, labelling, cleaning, updating, enrichment and aggregation

    [Regulation (EU) 2024/1689, Art. 10(2)(c)]

15. Count the records per subgroup in each data set and compare the counts with the target population statement.

    Record Subgroup coverage table

    [prEN 18284, draft, clause on representativeness]

16. Count the records per setting-of-use category and compare the counts with the setting-of-use statement.

    Record Setting coverage table

    [Regulation (EU) 2024/1689, Art. 10(4)]

17. Draw a random sample of labelled records from each data set and re-label it with an independent annotator.

    Record Label-error sample with the sample size and the sampling method

    [ISO/IEC TR 24027:2021 §6.3, data bias, labelling process]

18. Compute the label disagreement rate per subgroup from the label-error sample.

    Record Label error rate table

    [ISO/IEC TR 24027:2021 §6.3, data bias]

19. Profile the missing values per feature and per subgroup in each data set.

    Record Completeness profile

    [prEN 18284, draft, clause on dataset quality characteristics]

20. Compute the distribution of each feature and of the label, and compare each with the assumptions statement.

    Record Statistical properties comparison

    [Regulation (EU) 2024/1689, Art. 10(2)(d) and 10(3)]

21. Check for duplicate records within each data set and for overlap between the data sets.

    Record Duplicate and overlap report

    [AIAssuranceLabs procedure]

22. Run the model on the testing data set and record each output with its record identifier.

    Record Output log with the model version

    [AIAssuranceLabs procedure]

23. Compute the confusion matrix per subgroup for each protected attribute.

    Record Disaggregated confusion matrices

    [ISO/IEC TR 24027:2021 §7.2, confusion matrix]

24. Compute each selected fairness metric per subgroup and the disparity between each subgroup pair.

    Record Fairness metric and disparity table

    [ISO/IEC TR 24027:2021 §7.2, fairness metrics]

25. Compute each selected fairness metric for intersectional subgroups and flag each cell below the recorded minimum sample size.

    Record Intersectional metric table with sample-size flags

    [prEN 18283, draft, clause on bias measures]

26. Where outputs influence future inputs, compare the subgroup distributions of consecutive operation cycles.

    Record Feedback-loop comparison

    [Regulation (EU) 2024/1689, Art. 10(2)(f)]

27. Observe the pipeline and verify that special-category fields are pseudonymised and blocked from re-use.

    Record Safeguards observation, condition (b)

    [Regulation (EU) 2024/1689, Art. 4a(1)(b)]

28. Verify that the access control configuration limits special-category data to the authorised persons listed.

    Record Access control check, condition (c)

    [Regulation (EU) 2024/1689, Art. 4a(1)(c)]

29. Verify from the data flow diagram and egress controls that no other party receives special-category data.

    Record Transfer check, condition (d)

    [Regulation (EU) 2024/1689, Art. 4a(1)(d)]

30. Where the deployer controls the input data, profile the live input population and feature distributions.

    Record Input data representativeness profile

    [Regulation (EU) 2024/1689, Art. 26(4)]

31. Compare the live input profile with the intended purpose and with the training data set profile.

    Record Input population comparison

    [AIAssuranceLabs procedure]

The subgroup coverage table from step 15 has one row per data set, attribute and subgroup. The values below are synthetic.

data_set,attribute,subgroup,records,share,target_share,flag
train,sex,female,4120,0.41,0.50,below_target
train,sex,male,5880,0.59,0.50,above_target
train,age_band,18-34,3010,0.30,0.30,at_target
train,age_band,35-49,3980,0.40,0.35,above_target
train,age_band,50+,3010,0.30,0.35,below_target
test,sex,female,530,0.53,0.50,at_target
test,sex,male,470,0.47,0.50,at_target

Analysis

32. Compare each coverage, error and completeness value with its target in the threshold justification record.

    Record Data set quality findings, pass or fail per target

    [AIAssuranceLabs procedure]

33. Compare each disparity value with its justified target and mark each result pass or fail.

    Record Disparity findings, pass or fail per metric and subgroup pair

    [AIAssuranceLabs procedure]

34. Attribute each failed finding to one or more bias sources in the bias source register.

    Record Root-cause attribution

    [ISO/IEC TR 24027:2021 §6, sources of unwanted bias]

35. Record each bias mitigation measure the provider applied and the lifecycle stage where it was applied.

    Record Bias mitigation measures record

    [ISO/IEC TR 24027:2021 §8, treatment of unwanted bias across the life cycle]

36. Repeat the disaggregated evaluation on the mitigated model or data set and record each metric delta.

    Record Post-mitigation re-test table

    [Regulation (EU) 2024/1689, Art. 10(2)(g)]

37. Check that the legal-basis record states why other data, including synthetic or anonymised data, could not achieve the objective.

    Record Legal-basis review, conditions (a) and (f)

    [Regulation (EU) 2024/1689, Art. 4a(1)(a) and (f)]

38. Check the retention period and the deletion trigger recorded for the special-category data.

    Record Retention and deletion check, condition (e)

    [Regulation (EU) 2024/1689, Art. 4a(1)(e)]

Each fairness metric result from step 24 is stored as one record that carries the metric definition and the target it is compared with. The values below are synthetic.

{
"protocol": "PR-10-BIAS",
"step": 24,
"data_set": "test",
"attribute": "age_band",
"metric": "equal_opportunity_difference",
"definition": "TPR(subgroup) minus TPR(reference subgroup)",
"reference_subgroup": "35-49",
"values": {
  "18-34": -0.021,
  "50+": -0.084
},
"target": {
  "source": "threshold justification record, step 10",
  "max_absolute_difference": 0.05
},
"result": {
  "18-34": "pass",
  "50+": "fail"
}
}

Reporting

39. Assemble the data set quality profile from the lineage, coverage, error, completeness and statistical property tables.

    Record Dataset quality profile (PR-10-BIAS output)

    [AIAssuranceLabs procedure]

40. Assemble the bias examination report with the metric definitions, disaggregation tables, disparities and findings.

    Record Bias examination report with subgroup disaggregation tables (PR-10-BIAS output)

    [AIAssuranceLabs procedure]

41. Assemble the input data representativeness profile for the deployer from the live input comparison.

    Record Input data representativeness profile (PR-10-BIAS output)

    [AIAssuranceLabs procedure]

42. Assemble the legal-basis record and the safeguards and deletion record from the Art. 4a checks.

    Record Bias detection legal-basis record and special-category data safeguards and deletion record

    [AIAssuranceLabs procedure]

43. Write one mitigation recommendation with a re-test criterion for each failed finding.

    Record Mitigation recommendation list with re-test criteria

    [AIAssuranceLabs procedure]

44. Record the protocol version, the file hashes and the model version in every report.

    Record Report provenance block

    [AIAssuranceLabs procedure]

Thresholds and criteria

Art. 10 sets no numeric threshold. It states no minimum sample size per subgroup, no maximum label error rate, no completeness level and no acceptable disparity between groups. Every pass criterion in this protocol therefore comes from the source hierarchy in the obligation record, and the record for each criterion names its level.

1. No harmonised standard for Art. 10 is cited in the Official Journal, so the first level is empty; the standards page shows the current stage of each document.
2. prEN 18284Draft supplies the checklist of data governance practices and the dataset quality characteristics behind steps 13, 15 and 19. Its text is not public at draft stage, and it publishes no pass level.
3. prEN 18283Draft supplies the bias management concepts and candidate measures behind step 25. It is a draft and publishes no pass level.
4. ISO/IEC TR 24027:2021Published supplies the bias source categories behind steps 8 and 34 and the fairness metrics behind steps 9, 23 and 24. It is a technical report; it lists candidate metrics and states no acceptable disparity.
5. The Act itself supplies the criteria for the Art. 4a(1) conditions: each condition in points (a) to (f) is present in the pipeline and the records, or it is absent. Steps 27 to 29, 37 and 38 record presence or absence per condition.
6. For coverage, error rate, completeness and disparity, no published threshold exists; the provider justifies the target against the context of use and the persons or groups the system is intended to be used for. Step 10 records that justification, and steps 32 and 33 compare each measured value with it.

A disparity target that the provider adopts from a sector rule or a national practice is recorded in step 10 with its source and its condition of use; the protocol does not adopt it as its own criterion.

The necessity threshold for the Art. 4a legal basis is an unsettled point. Step 37 checks the record against the working reading below.

Outputs

Each artifact below is named as an assessor asks for it and is specified on the deliverables page.

Limitations

• The protocol shows disparity on the subgroups and metrics defined in steps 7 and 9. It does not show the absence of bias on subgroups that the data sets do not identify or on metrics that were not selected.
• Several fairness metrics cannot be met at the same time on the same data set. The metric selection record in step 9 states which metrics were selected and why; the protocol does not resolve the conflict between them.
• Subgroup metrics on small cells carry wide uncertainty. Step 25 flags each cell below the recorded minimum sample size; the protocol does not report a value for a flagged cell as a finding.
• Where the provider holds no protected attribute data, steps 15, 23 and 24 cannot run on that attribute. The protocol records the gap for the Art. 10(2)(h) register; it does not infer protected attributes.
• The Art. 4a checks in steps 27 to 29, 37 and 38 record what the pipeline and the records show. They do not decide whether the processing is lawful under Regulation (EU) 2016/679; that decision remains with the provider and its counsel.
• prEN 18284Draft and prEN 18283Draft have no public text at draft stage. The clause references in steps 13, 15, 19 and 25 name the subject of the clause, and the protocol is revised when the texts are published.
• The pass or fail results in steps 32 and 33 are results against the provider's justified targets. A pass is not a statement that the system meets Art. 10; that decision remains with the provider, its notified body where one is required, and its counsel.

Sources

• ISO/IEC TR 24027:2021Published, bias in AI systems and AI aided decision making, a published technical report: ISO catalogue
• prEN 18283Draft, bias management, a JTC 21 draft: JTC 21 standards tracker
• prEN 18284Draft, dataset quality and governance, a JTC 21 draft: JTC 21 standards tracker
• Regulation (EU) 2024/1689, consolidated text: EUR-Lex
• Regulation (EU) 2024/1689, Art. 4a, rendering of the post-Omnibus text: AI Act Explorer

Revision history

The table lists each version of PR-10-BIAS with its note and the verified-on stamp of the regulation entry the revision names.