EU AI Act watermark survivability: testing a watermark against re-encoding

What the obligation asks for

Art. 50(2) of Regulation (EU) 2024/1689 requires the provider of an AI system that generates synthetic audio, image, video or text to mark the outputs in a machine-readable format and to make them detectable as artificially generated or manipulated, with technical solutions that are effective, interoperable, robust and reliable as far as technically feasible. The Art. 50 obligation record lists survivability as one testable element: the mark survives a defined set of ordinary transformations for the modality (re-encoding, resizing, cropping, screenshot, format conversion, transcoding, copy and paste for text), verified per transformation with the survival result recorded. Two more elements of the record feed the same run: the detection rate and the false-positive rate recorded against the target the provider justified, and readability of the mark by a tool other than the provider's own. Art. 50 applies from the general application date 2026-08-02In force; a system that generates synthetic content and was placed on the market before that date has a grace period for the Art. 50(2) marking duty only 2026-12-02Deferred.

The Act names no transformation and no survival rate. The published Code, Code of Practice on Transparency of AI-Generated ContentPublished, gives the operations in Measure 3.3: in-place modifications such as noise, filtering, recompression, screenshot and change of file format; desynchronisation such as mirroring, cropping, scaling, rotation, aspect ratio change, pitch shift and time stretch; and the analogue hole, such as print-and-scan or screen recording. The Commission guidelines, Commission guidelines on Article 50 transparency obligationsPublished, published, read robustness in paragraph 79 as the capability to identify the provider's content under varying conditions, covering common alterations and adversarial attacks, and paragraph 80 asks the combination of marking and detection solutions to meet the four criteria holistically. This guide covers the common alterations only; adversarial removal is outside the current protocol version. Which layer set counts as adequate under Art. 50(2) is unsettled, and the guide tests the layers the marking architecture description declares without taking a position on that point.

Inputs

The inputs below are the subset of the PR-50-MARK v1.0 input table that the survivability steps use. The full table is on the protocol page.

• Access to the generating system on every output path listed in the scope record: API, user interface, SDK or batch export.
• The marking architecture description per modality: which layers are applied (signed metadata, imperceptible watermark, provenance manifest), at which pipeline point, and which reader detects each.
• The provider's own watermark detector, as a command-line tool or a local API, with its version.
• An independent manifest validator with a recorded version and trust list, for the metadata and provenance layer.
• The target justification record: detection rate target, false-positive tolerance and the transformation set the provider justified against the context of use.
• The list of distribution surfaces where outputs are published, with each surface's published upload specification where one exists.
• Non-generated control samples per modality with a statement of origin that excludes the system under test.
• A workstation with ffmpeg, ImageMagick and Python with Pillow installed, versions recorded in the tool register.

Procedure

Each step names the protocol step or the source clause it derives from and the record it produces. Code blocks follow the step they belong to. Every path, sample identifier, parameter, count and rate in the examples is synthetic.

1. Record the provider's detection rate target, false-positive tolerance and transformation set with the written justification.

   Record target-justification-record: detection rate target, false-positive tolerance, transformation set, justification

   [Regulation (EU) 2024/1689, Art. 50(2); PR-50-MARK step 6]

2. Define the transformation set per modality from the Measure 3.3 operations, with one parameter value per row.

   Record transformation-set: one row per modality and transformation with parameters

   [Code of Practice on Transparency of AI-Generated Content, Section 1, Measure 3.3; PR-50-MARK step 7]

3. Add one chained row per distribution surface with the operations its upload pipeline applies, in order.

   Record transformation-set: one chain row per distribution surface with the surface's upload parameters

   [Code of Practice on Transparency of AI-Generated Content, Section 1, Measure 4.2; PR-50-MARK step 7]

4. Generate one output per sample case on each output path; store the file with a hash.

   Record generation-log: sample id, output path, SHA-256 hash, prompt or seed, model version

   [AIAssuranceLabs procedure; PR-50-MARK step 11]

5. Run the provider's detector on each untransformed sample; record the baseline result per sample.

   Record watermark-detection-log: sample id, detector version, result on the untransformed file

   [Code of Practice on Transparency of AI-Generated Content, Section 1, Sub-measure 1.1.2; PR-50-MARK step 15]

6. Apply each transformation row to each marked sample and to each control sample; hash each output.

   Record transformed-sample-register: source sample, transformation, parameters, output hash

   [Code of Practice on Transparency of AI-Generated Content, Section 1, Measure 3.3; PR-50-MARK step 19]

7. Take a screenshot of each image and video sample on the display surface; store it with a hash.

   Record screenshot-register: source sample, display surface, capture tool, output hash

   [AIAssuranceLabs procedure; PR-50-MARK step 20]

8. Re-run the detector and the manifest validator on each transformed file; record survived or lost per layer.

   Record survival-matrix: sample, transformation, layer, survived or lost

   [C2PA Content Credentials specification §15.1; Code of Practice on Transparency of AI-Generated Content, Section 1, Measure 3.3; PR-50-MARK step 22]

9. Run the detector on each transformed control sample; record each result per transformation.

   Record control-detection-log: control sample id, transformation, result

   [AIAssuranceLabs procedure; PR-50-MARK steps 9 and 25]

10. Compute per transformation the detection rate on marked samples and the false-positive rate on control samples.

    Record detection-rate-summary: per transformation, detection rate on marked samples, false-positive rate on controls, sample counts

    [Code of Practice on Transparency of AI-Generated Content, Section 1, Measures 3.2 and 3.3; PR-50-MARK step 36]

11. Compare each per-transformation rate with the target justification record; record met or not met.

    Record target-comparison: transformation, rate, target, met or not met

    [PR-50-MARK step 37]

12. Write the survivability section of the test report with the survival matrix, the rates and the tool versions.

    Record Marking presence and survivability test report (PR-50-MARK output), with run-log and tool-register

    [AIAssuranceLabs procedure; PR-50-MARK steps 41 and 45]

Example for step 2. The transformation set is a table the run reads; each row is one operation with one parameter value, so that a lost mark can be attributed to a parameter and not to a family of operations. The rows below are synthetic and cover the in-place and desynchronisation classes of Measure 3.3 for one image path and one video path; the analogue-hole class is recorded as out of set for this run, with the reason.

modality,transformation,parameters,source_class,in_set
image,re-encode,jpeg-q90,in-place (recompression),yes
image,re-encode,jpeg-q75,in-place (recompression),yes
image,re-encode,jpeg-q50,in-place (recompression),yes
image,format-convert,webp-q80,in-place (change of file format),yes
image,metadata-strip,exif-xmp-removed,in-place,yes
image,resize,0.5x,desynchronisation (downscaling),yes
image,resize,1.5x,desynchronisation (upscaling),yes
image,crop,center-80pct,desynchronisation (cropping),yes
image,rotate,2deg,desynchronisation (rotation),yes
image,screenshot,display-native,in-place (screenshot),yes
video,transcode,h264-crf23-aac96k,in-place (recompression),yes
video,transcode,h264-crf35-aac64k,in-place (recompression),yes
video,format-convert,vp9-500k-webm,in-place (change of file format),yes
video,resize,480p,desynchronisation (downscaling),yes
video,crop,center-80pct,desynchronisation (cropping),yes
video,metadata-strip,container-metadata-removed,in-place,yes
audio,transcode,mp3-64k,in-place (recompression),yes
audio,resample,16khz,desynchronisation,yes
image,print-and-scan,not-run,analogue hole,no: no scanner in the test environment

Example for step 6, images. ImageMagick applies one operation per output file and rewrites the file on every operation, which drops the JUMBF container that holds a C2PA manifest, so the metadata result is expected to be lost on every image row and each row tests the watermark layer alone; -strip additionally removes the EXIF and XMP containers. +repage after -crop resets the canvas offset so that a detector reads the cropped pixels and not a virtual canvas. The quality values and scale factors are the synthetic parameters from the table above.

# Synthetic example: IMG-0001.png is one marked output from step 4.
set -e
src=IMG-0001.png
mkdir -p out
for q in 90 75 50; do
  magick "$src" -quality "$q" "out/IMG-0001_jpeg-q${q}.jpg"
done
magick "$src" -quality 80 "out/IMG-0001_webp-q80.webp"
magick "$src" -strip "out/IMG-0001_exif-xmp-removed.png"
magick "$src" -resize 50%  "out/IMG-0001_resize-0.5x.png"
magick "$src" -resize 150% "out/IMG-0001_resize-1.5x.png"
magick "$src" -gravity center -crop 80%x80%+0+0 +repage "out/IMG-0001_crop-center-80pct.png"
magick "$src" -background white -rotate 2 +repage "out/IMG-0001_rotate-2deg.png"
sha256sum out/IMG-0001_* > out/IMG-0001.sha256

Example for step 6, video and audio. ffmpeg re-encodes with a constant rate factor for H.264 and a target bitrate for VP9; -map_metadata -1 drops the container metadata, which again isolates the watermark layer. scale=-2:480 keeps the aspect ratio and an even width, which the H.264 encoder requires. For audio, the MP3 bitrate and the resampling rate are the two rows from the table.

# Synthetic example: VID-0002.mp4 and AUD-0003.wav are marked outputs from step 4.
set -e
mkdir -p out
ffmpeg -y -i VID-0002.mp4 -c:v libx264 -crf 23 -c:a aac -b:a 96k out/VID-0002_h264-crf23-aac96k.mp4
ffmpeg -y -i VID-0002.mp4 -c:v libx264 -crf 35 -c:a aac -b:a 64k out/VID-0002_h264-crf35-aac64k.mp4
ffmpeg -y -i VID-0002.mp4 -c:v libvpx-vp9 -b:v 500k -c:a libopus out/VID-0002_vp9-500k.webm
ffmpeg -y -i VID-0002.mp4 -vf "scale=-2:480" -c:a copy out/VID-0002_resize-480p.mp4
ffmpeg -y -i VID-0002.mp4 -vf "crop=iw*0.8:ih*0.8" -c:a copy out/VID-0002_crop-center-80pct.mp4
ffmpeg -y -i VID-0002.mp4 -map_metadata -1 -c copy out/VID-0002_metadata-stripped.mp4
ffmpeg -y -i AUD-0003.wav -c:a libmp3lame -b:a 64k out/AUD-0003_mp3-64k.mp3
ffmpeg -y -i AUD-0003.wav -ar 16000 out/AUD-0003_resample-16khz.wav
sha256sum out/VID-0002_* out/AUD-0003_* > out/VID-AUD.sha256

Example for step 7. The screenshot is taken on the display surface a person would use, at the display's native resolution, with the operating system's own capture tool; on macOS screencapture -x out/IMG-0001_screenshot.png captures the screen without the shutter sound, and the capture tool and display are recorded in the screenshot register. A screenshot is a re-rasterisation of the displayed pixels: every metadata container is gone, and the watermark layer is tested at the scale the display applied.

Example for step 8. The script walks the output folder, runs the provider's detector and the independent validator on each file, and writes the survival matrix. The detector here is a synthetic command-line tool detector-cli that prints a JSON object with a detected boolean; the provider's real detector is substituted, and its version goes into the tool register. The validator call is the same shape and is left as a second function so that each layer has its own column. Pillow reads the image back before the detector call, which catches a file that a transformation left unreadable and records it as a lost row with the reason.

# Synthetic example: write survival-matrix.csv from the files in out/.
import csv
import json
import pathlib
import subprocess

from PIL import Image, UnidentifiedImageError

OUT = pathlib.Path("out")
DETECTOR = ["detector-cli", "--json", "--input"]          # provider's detector, synthetic name
VALIDATOR = ["c2patool", "--detailed"]                    # independent manifest validator

IMAGE_SUFFIXES = {".png", ".jpg", ".jpeg", ".webp"}


def run_json(cmd, path):
    proc = subprocess.run(cmd + [str(path)], capture_output=True, text=True)
    if proc.returncode != 0:
        return None
    try:
        return json.loads(proc.stdout)
    except json.JSONDecodeError:
        return None


def watermark_result(path):
    if path.suffix.lower() in IMAGE_SUFFIXES:
        try:
            with Image.open(path) as im:
                im.verify()
        except (UnidentifiedImageError, OSError):
            return "lost: unreadable file"
    data = run_json(DETECTOR, path)
    if data is None:
        return "lost: detector error"
    return "survived" if data.get("detected") is True else "lost"


def metadata_result(path):
    data = run_json(VALIDATOR, path)
    if data is None:
        return "lost: no manifest"
    status = data.get("validation_state")
    return "survived" if status in ("Valid", "Trusted") else "lost"


rows = []
for f in sorted(OUT.iterdir()):
    if f.suffix == ".sha256":
        continue
    sample_id, _, transformation = f.stem.partition("_")
    rows.append([sample_id, transformation, "watermark", watermark_result(f)])
    rows.append([sample_id, transformation, "metadata", metadata_result(f)])

with open("survival-matrix.csv", "w", newline="", encoding="utf-8") as fh:
    w = csv.writer(fh)
    w.writerow(["sample_id", "transformation", "layer", "result"])
    w.writerows(rows)
print(len(rows), "rows written")

Example for step 10. The summary below is a synthetic output of the run above for one image path with a sample of marked outputs and a sample of control images per transformation. Every count and rate is an output of this example; none is a requirement of the Act or of the Code. The chained row is the surface row from step 3, with the operations applied in the order the surface's upload specification states.

The metadata row is lost on every transformation in this synthetic run, which is the first common finding of the obligation record: metadata as the only layer is removed by a screenshot or a re-encode. The row is the reason the survival matrix carries one row per layer: the report states which layer carried detectability through each operation.

A pitfall that vendor documentation omits

The documentation of a watermarking library or of an upstream model provider reports survival on the transformation set its authors chose: one JPEG quality, one scale factor, one crop ratio, each applied alone to the unmodified output. Measure 4.2 of the published Code, Code of Practice on Transparency of AI-Generated ContentPublished, lets a downstream provider rely on the results of testing performed by an upstream model provider or a third-party marking vendor, without prejudice to the downstream provider's own responsibility under Art. 50(2) and (5), and it asks for testing under conditions that are representative of real-world use. A distribution surface does not apply one operation alone. A surface's published upload specification, where one exists, lists the operations its pipeline applies in sequence, for example metadata removal, downscaling to a maximum edge length, re-encoding at the surface's quality setting and, for video, transcoding into the surface's bitrate ladder; the order and parameters are taken from that specification in step 3. A watermark that survives each of these at the vendor's parameter can be lost under the chain, because the losses compose: the synthetic table above shows a chained row below every single-operation row that feeds it. A survivability report that carries only the vendor's rows therefore shows survival under conditions the content will not meet once published.

The protocol's Limitations section states the same boundary from the other side: the result does not show survival under transformations outside the recorded set. Step 3 of this guide is the correction. One chained row per distribution surface goes into the transformation set, with the parameters taken from the surface's published upload specification where one exists, or from an observed upload where none does, and the source of each parameter is recorded in the row. Where the surface's parameters are unknown, the row is recorded as unknown, which is a finding for the remediation list, since the provider cannot state survival on that surface.

Regulation (EU) 2024/1689 sets no survival threshold and no transformation list. The Code publishes no survival rate either: Measure 3.3 names the operations and refers to the reliability metrics of Measure 3.2, such as the error rate of detection, and Measure 4.2 defers to internal benchmarks and industry practice until evaluation methods endorsed by the AI Office emerge. The hierarchy is therefore the one the obligation record states: no harmonised standard is cited in the Official Journal as of the anchor date, so no presumption of conformity attaches to any result; no JTC 21 draft and no ISO/IEC document on the standards page covers Art. 50; the operations and the metric come from the Code and from paragraphs 79 to 83 of Commission guidelines on Article 50 transparency obligationsPublished, published; and the pass rate per transformation is the provider's justified state of the art, recorded in step 1 of this guide, which is step 6 of the protocol.

What the result shows and what it does not

The survival matrix and the per-transformation rates show what the declared layers did on the sampled outputs, under the recorded transformation rows, with the recorded detector and validator versions, on the test date. The result does not show survival under an operation or a parameter outside the set, and it does not show resistance to adversarial removal, regeneration or modification of the mark, which Measure 3.3 of the Code also names and which the current protocol version does not cover. The false-positive rate is measured on the control samples collected for the run; a different control population can give a different rate. A rate compared with the provider's own justified target does not show that the Commission or a market surveillance authority accepts either the target or the technique set, and the layer-set question above remains unsettled. The result is evidence for the provider's own file; the decision whether the marking meets Art. 50(2) remains with the provider and their counsel. The Article 50 transparency and marking/watermark testing service runs the full protocol, and its limits are on what we cannot do; the report format is specified on deliverables.

Related pages

The obligation, the protocol and the hub for the role this guide serves are listed below; every protocol on the site is on the methodology index.

Obligations

• Art. 50

Protocols

• PR-50-MARK v1.0

Related guides

• Verify content credentials: C2PA manifest validation step by step

ICP hubs

• Generative AI features

Every inquiry receives a reply within 24 hours. Scope and availability are discussed per request.

Ask about scopehello@aiassurancelabs.com