2510.09263v1
Model: healer-alpha-free
# SynthID-Image: Image watermarking at internet scale
**Authors**: Sven Gowal, Rudy Bunel, Florian Stimberg, David Stutz, Guillermo Ortiz-Jimenez, Christina Kouridi, Mel Vecerik, Jamie Hayes, Sylvestre-Alvise Rebuffi, Paul Bernard, Chris Gamble, MiklĂłs Z. HorvĂĄth, Fabian Kaczmarczyck, Alex Kaskasoli, Aleksandar Petrov, Ilia Shumailov, Meghana Thotakuri, Olivia Wiles, Jessica Yung, Zahra Ahmed, Victor Martin, Simon Rosen, Christopher SavÄak, Armin Senoner, Nidhi Vyas, Pushmeet Kohli
> Google DeepMind Core contributor (randomized order) Lead
> Google DeepMind Core contributor (randomized order)
> Google DeepMind Contributor
> Google DeepMind Support
> Google DeepMind Lead Sponsor
sgowal@google.com, pushmeet@google.com
## Abstract
We introduce SynthID-Image, a deep learning-based system for invisibly watermarking AI-generated imagery. This paper documents the technical desiderata, threat models, and practical challenges of deploying such a system at internet scale, addressing key requirements of effectiveness, fidelity, robustness, and security. SynthID-Image has been used to watermark over ten billion images and video frames across Googleâs services and its corresponding verification service is available to trusted testers. For completeness, we present an experimental evaluation of an external model variant, SynthID-O, which is available through partnerships. We benchmark SynthID-O against other post-hoc watermarking methods from the literature, demonstrating state-of-the-art performance in both visual quality and robustness to common image perturbations. While this work centers on visual media, the conclusions on deployment, constraints, and threat modeling generalize to other modalities, including audio. This paper provides a comprehensive documentation for the large-scale deployment of deep learning-based media provenance systems.
keywords: provenance, watermarking, robustness
## 1 Introduction
In the wake of powerful generative artificial intelligence (AI) systems and their accessibility in products such as Gemini, ChatGPT, Midjourney or ElevenLabs including their corresponding open-source alternatives (Rombach et al., 2022; Touvron et al., 2023; Scao et al., 2022), leading AI companies including Google committed to advancing responsible practices in deploying AI. One such practice consists of establishing the provenance of generated media artifacts. In this context, the Coalition for Content Provenance and Authenticity (C2PA; Coalition for Content Provenance and Authenticity, 2023), describes provenance as serving two main purposes: allowing leading AI companies to responsibly disclose that content is AI-generated and allowing users to verify the authenticity of these artifacts. The latter, in particular, has received considerable attention due to worries of widespread misinformation (Zellers et al., 2019; Weidinger et al., 2021) or impersonation (e.g., so-called âdeep fakesâ; Agarwal et al., 2019; Carlini and Farid, 2020; Vaccari and Chadwick, 2020; Yu et al., 2021). Importantly, we note that establishing provenance is materially different from detecting AI-generated content, as the public discourse tends to focus on. https://www.technologyreview.com/2023/11/06/1082996/the-inside-scoop-on-watermarking-and-content-authentication Not only are such detectors https://contentatscale.ai/ai-content-detector/, https://www.compilatio.net/en/blog/best-ai-detectors, https://gptzero.me/ generally less reliable, but even if we could reliably detect AI-generated content, they do not necessarily establish provenance. Fine-grained origins are typically still unknown and we believe that the increase of fidelity of future generative models will also increase the complexity of establishing provenance through detection alone.
While C2PA establishes a standard for encoding and verifying provenance through metadata that is attached to the content, this approach is vulnerable to removing said metadata (Zhao et al., 2025). Metadata is often stripped accidentally and can also be trivially removed. Watermarking is a promising responsible disclosure mechanism that allows to embed provenance information within the generated content directly. As visible watermarks (Braudaway, 1997) have been shown to be ineffective in the age of deep learning (Dekel et al., 2017), there has been a recent revival of deep learning based methods for invisible watermarking. Originating in the information hiding community (Anderson, 1996; Cox, 2007), invisible watermarks aim to hide information within a multimedia artifact. Classical methods use wavelet or Fourier transforms for this purpose and recently deep learning based methods have been introduced for this task (Zhu et al., 2018; Adi et al., 2018) as they promise to store more information with higher robustness to removal attempts. However, this exposes these methods to a wide array of attacks that exploit vulnerabilities in the underlying deep neural networks. Furthermore, many methods still lack robustness to common post-processing or manipulations of the content and, unsurprisingly, they are vulnerable to adversarial attempts to remove or fake the watermark, e.g. by utilizing a rich body of work on adversarial examples (Biggio et al., 2013; Szegedy et al., 2014). Building (adversarially) robust watermarking systems remains an extremely challenging problem. More importantly, there is little to no guidance on how to deploy such systems in a robust manner at internet scale.
To successfully enable provenance of AI-generated content at internet scale, a watermarking system needs to balance a number of potentially conflicting desiderata. As the main objective during the development of generative AI models is quality, it is crucial for watermarking systems not to impact it. This is commonly implied by demanding invisible watermarking. Then, we want the produced watermarks to be effective in verifying provenance which also includes robustness to expected and unexpected changes in the underlying media artifact due to every day use. Deviating from every day use, we explicitly want watermarks to be secure against malicious use, ranging from watermark removal or forgery attacks to attempts on model extraction. To enable watermarking at internet scale, it also needs to be sufficiently efficient in terms of runtime, with minimal overhead during generation and high throughput during detection and verification. A rich literature on watermarking, see surveys by Wan et al. (2022); Cox (2007) and benchmarks by An et al. (2024); Petitcolas (2000), hints that many of these desiderata are conflicting and trade-offs have to be made. This is particularly important in light of our final desiderata, large-scale deployment. This means that, ultimately, decisions have to be made on how to handle trade-offs and deal with imperfect robustness and security in practice.
With SynthID-Image, https://deepmind.google/technologies/synthid/ we built a watermarking system that is currently watermarking all of Googleâs AI-generated imagery. Throughout development, we found that many existing surveys, benchmarks and methods do not realistically capture the above desiderata and their trade-offs, often ignoring key threat models, evaluations or deployment considerations. In this paper, we want to share a detailed account of our learnings from developing, deploying and maintaining SynthID-Image since its first public announcement in August 2023. https://deepmind.google/discover/blog/identifying-ai-generated-images-with-synthid/ Moreover, with this work, we evaluate our external variant of SynthID-Image which is available via partnerships. Our contributions can be summarized along the structure of this paper: SynthID-Image (Section 2)
follows a post-hoc and model-independent approach to watermarking. This means the watermark is applied on top of the AI-generated content using an encoder, not as part of the generation process, and detected using a corresponding decoder. This makes SynthID applicable to any generative model and thus maximizes utility, deployability and organizational flexibility at the cost of stringent constraints on quality and efficiency. Invisibility (Section 3):
Because quality dictates many trade-off among relevant desiderata, we meticulously benchmark the watermarks impact on quality by predominantly using human judgement on corner-case content. We frequently run internal studies and complement this with external ones. Framing these studies correctly is key. These studies are complemented using proxy metrics that enable auto-evaluation. Robustness (Section 4)
requires comprehensive and reproducible benchmarking of numerous possible transformations. This is informed by manually deciding on appropriate ranges, essentially defining when transformed content can visibly be determined as heavily tampered or when transformations change the underlying (semantic) content significantly. Payload (Section 5):
Tracking and establishing provenance goes beyond âjustâ detecting a watermark. While many approaches allow multi-bit watermarks, it is often neglected how a multi-bit payload is best used. Adapting SynthID-Image âs payload to its use case allows us to serve different generative models, users and deployment scenarios. Ensuring security (Section 6)
of the watermarking system involves understanding a wide range of threat models, setting them up for benchmarking and prioritizing them. The relevance of each threat model depends on how the system is deployed, i.e., what information of is publicly available or can be extracted under reasonable assumptions. Achieving perfect security is impossible; thus, we focused our efforts on making key attacks as difficult and expensive as possible. Deployment (Section 7):
Aiming for internet-scale deployment requires solving a wide range of additional problems, including decision making (i.e., what constitutes a positive and when to abstain), defining payload use, and putting watermarking in context of other methods such as C2PA or retrieval (e.g., reverse image search). New state-of-the-art (Section 8):
We present experimental results for one variant of SynthID-Image. This model achieves state-of-the-art performance across the most comprehensive range of transformations to date. We also provide a usage recommendation of how to adopt this external variant securely. Figure 1 demonstrates that our models establish a new state-of-the-art in terms of quality, detection performance and robustness across a wide range or transformations.
We believe this to be the first comprehensive account of deploying image watermarking models at internet scale.
<details>
<summary>x1.png Details</summary>
### Visual Description
## Scatter Plot: Watermarking Method Evaluation (Perceptibility vs. Brittleness)
### Overview
The image is a scatter plot comparing eight different digital watermarking or steganography methods. The plot evaluates each method on two axes: "Perceptibility" (y-axis) and "Brittleness" (x-axis). The data points are plotted as circles, with one method highlighted in blue and the others in gray. A legend is present in the top-left corner.
### Components/Axes
* **Chart Type:** Scatter Plot.
* **Y-Axis:**
* **Label:** "Perceptibility (percent point increase in artifact rates)"
* **Scale:** Linear, from 0 to 40, with major tick marks every 5 units (0, 5, 10, 15, 20, 25, 30, 35, 40).
* **X-Axis:**
* **Label:** "Brittleness (FNR at 0.1% FPR)"
* **Scale:** Linear, from 0% to 40%, with major tick marks every 5% (0%, 5%, 10%, 15%, 20%, 25%, 30%, 35%, 40%).
* **Legend:**
* **Position:** Top-left corner of the plot area.
* **Entries:**
1. A blue circle labeled "SynthID-Q".
2. A gray circle labeled "WAM".
* **Data Points (Labeled Methods):** Each method is represented by a single circle. The approximate coordinates (Brittleness, Perceptibility) are as follows:
* **StegaStamp:** Gray circle at approximately (28%, 38).
* **TrustMark-Q:** Gray circle at approximately (22%, 27).
* **VideoSeal 0.0:** Gray circle at approximately (23%, 26).
* **TrustMark-P:** Gray circle at approximately (34%, 10).
* **InvisiMark:** Gray circle at approximately (30%, 7).
* **WAM:** Gray circle at approximately (10%, 9).
* **VideoSeal 1.0:** Gray circle at approximately (11%, 5).
* **SynthID-Q:** Blue circle at approximately (0%, 0).
### Detailed Analysis
The plot visualizes a trade-off between two undesirable properties of a watermark: how easily it is perceived (Perceptibility) and how easily it is broken or fails (Brittleness, measured as False Negative Rate at a fixed False Positive Rate).
* **Trend Verification:** There is a general positive correlation visible. Methods with higher perceptibility (higher on the y-axis) also tend to have higher brittleness (further right on the x-axis). However, the relationship is not perfectly linear, and there is significant spread.
* **Data Series & Spatial Grounding:**
* **High Perceptibility / High Brittleness Cluster:** `StegaStamp` is the most perceptible and brittle method, positioned in the top-right quadrant. `TrustMark-Q` and `VideoSeal 0.0` form a cluster nearby, with slightly lower values on both axes.
* **Moderate Brittleness / Lower Perceptibility:** `TrustMark-P` and `InvisiMark` are positioned further right on the brittleness axis (30-34%) but have significantly lower perceptibility scores (7-10%).
* **Low Brittleness / Low Perceptibility Cluster:** `WAM` and `VideoSeal 1.0` are grouped in the lower-left area, indicating relatively good performance on both metrics.
* **Ideal Point (Origin):** `SynthID-Q` is plotted at the origin (0%, 0), indicating it has the lowest possible scores for both perceptibility and brittleness according to this chart. It is the only data point colored blue, matching its entry in the legend.
### Key Observations
1. **SynthID-Q is an Outlier:** It occupies the ideal position at (0,0), suggesting it introduces no measurable increase in artifact rates and has a 0% false negative rate at the given threshold. This makes it appear vastly superior to all other methods plotted.
2. **Clear Performance Tiers:** The methods cluster into distinct groups:
* **Tier 1 (Best):** SynthID-Q (blue).
* **Tier 2 (Good):** VideoSeal 1.0 and WAM (low perceptibility <10%, brittleness ~10-11%).
* **Tier 3 (Mixed):** InvisiMark and TrustMark-P (low perceptibility but high brittleness >30%).
* **Tier 4 (Poor):** TrustMark-Q, VideoSeal 0.0, and StegaStamp (high perceptibility >25% and moderate-to-high brittleness >20%).
3. **Version Comparison:** `VideoSeal 1.0` shows a dramatic improvement over `VideoSeal 0.0`, moving from the poor-performance cluster to the good-performance cluster, reducing both perceptibility and brittleness significantly.
4. **Legend Ambiguity:** The legend lists "WAM" with a gray circle, which matches the color of all non-SynthID-Q data points. This could imply that gray circles represent a category or baseline that includes WAM, but since all other methods are also gray, it primarily serves to identify the color for the WAM data point specifically.
### Interpretation
This chart is a performance benchmark for watermarking/steganography techniques, likely from a research paper. The axes define the core challenge: a good watermark should be imperceptible (low perceptibility) and robust (low brittleness/FNR).
* **What the Data Suggests:** The plot demonstrates that achieving both low perceptibility and low brittleness is difficult. Most methods make a trade-off. `SynthID-Q` is presented as a breakthrough that seemingly overcomes this trade-off, achieving near-perfect scores. The significant improvement from `VideoSeal 0.0` to `1.0` highlights successful iterative development.
* **Relationship Between Elements:** The spatial distribution creates a "Pareto front" of sorts. Methods closer to the origin are better. The diagonal spread suggests that as developers try to make watermarks less perceptible (moving down the y-axis), they often become more brittle (moving right on the x-axis), and vice-versa. `SynthID-Q`'s position suggests it uses a fundamentally different or superior approach.
* **Notable Anomalies/Caveats:**
* The perfect (0,0) score for `SynthID-Q` is highly unusual for empirical data and may represent a theoretical best-case, a result under very specific/ideal conditions, or a different measurement baseline. It warrants scrutiny.
* The metric "percent point increase in artifact rates" for perceptibility is specific. It measures how much more often artifacts are detected, not an absolute quality score.
* The brittleness metric is conditional on a 0.1% False Positive Rate (FPR), meaning the threshold is set to be very strict about not misidentifying unwatermarked content. Performance could change at a different FPR.
**Language:** All text in the image is in English.
</details>
Figure 1: Quality as evaluated by human raters against FNR at 0.1% FPR averaged across various image transformations. Each model is calibrated to obtain 0.1% FPR (more details to follow in Section 8). The quality is measured on the y-axis by looking at the difference in artifact rates between watermarked and non-watermarked content. SynthID-O (our external variant of SynthID-Image available via partnerships) achieves the highest quality (i.e., lowest perceptibility) and robustness (i.e., lowest brittleness) compared to other baselines.
## 2 Our approach
Digital watermarking is commonly defined as âthe practice of hiding a message about an image, audio clip, video clip, or other work of media within that work itself (Cox, 2007).â Generally, this implies the ability to detect and extract this hidden message, which we refer to as the watermark payload, in a way that is robust to post-processing of the watermarked artifact. While watermarking can be used for many modalities (e.g., text; Dathathri et al., 2024), in this paper, we solely consider multimedia watermarking and focus on images. Besides its original use for steganography (i.e., the practice of âundetectably embedding a secret messageâ; Cox, 2007), watermarking in the context of generative AI is primarily associated with tackling misinformation, impersonation (Yu et al., 2021) and copyright protection (Kadian et al., 2021; Xuehua, 2010).
The application to AI-generated content provenance is the dominant motivation of recent deep learning based watermarking methods. It deviates significantly from earlier research (Wan et al., 2022). For example, we exclusively consider blind watermarking which assumes that we only have access to the watermarked content for detection, not pairs of original and watermarked content. In the context of text-to-image models, blind watermarking also excludes knowledge of the used prompt. Moreover, in contrast to steganography, we may not care about users or the general public knowing that content gets watermarked. In fact, companies have been vocal about their plans to watermark AI-generated content as demonstrated through the ongoing dialogue with legislators and participation in C2PA. In the context of soft-binding schemes (https://c2pa.org/specifications/specifications/1.0/specs/C2PA_Specification.html##_soft_bindings).
Irrespective of the approach, there are several desiderata that we expect SynthID-Image to fulfill. Most prominently, these include quality preservation, robustness to every-day changes to the watermarked content, and hiding a sufficiently large payload in the watermark. The difficulty of developing any watermarking system stems from unfavorable trade-offs between these desiderata, namely the quality-robustness-payload trade-off. For example, without requiring an invisible watermark, it becomes trivial to perform robust watermarking with large payload. Thus, in the following, we start with an overview of these desiderata and trade-offs.
### 2.1 Desiderata
#### Quality and diversity.
Product and research teams (especially those that build generative models) are keen to demonstrate and release their most powerful models. This race to higher quality content means that watermarking can be an afterthought and must, as a result, maintain the quality and diversity of the generated content. While some ad-hoc watermarking schemes are provably lossless (Christ et al., 2024; Yang et al., 2024), they can reduce diversity (Gunn et al., 2025). Post-hoc schemes do not suffer from diversity degradation by construction, but may affect quality. In practice, all methods require careful empirical assessment. Unfortunately, quality is the most difficult requirement to properly measure and ensure. Common automated metrics are unreliable and can only be used as proxies during development. Thus, in developing SynthID-Image, we focused heavily on human studies to evaluate quality, as discussed in detail in Section 3.
#### Effectiveness and robustness.
It is important to properly define under which assumptions a watermarking system is expected to work reliably. For a start, the watermark should always be perfectly detectable when the watermarked content is not transformed or manipulated. Good detection performance is commonly evaluated using the true positive rate (TPR) at a specific (low) false positive rate (FPR). Achieving high TPR at low FPR is what we call an effective watermarking scheme. Beyond effectiveness, we require watermark detection to be robust to general purpose, everyday changes such as compression, quantization, basic geometric transformations or image effects. While evaluating watermarking schemes against various such transformations is common practice, as shown in older as well as more recent benchmarks (Petitcolas, 2000; An et al., 2024; Piet et al., 2025), there is little agreement on which exact transformations to consider (across modalities) and evaluations are often not exhaustive, leading to misleading claims about the effectiveness of watermarking (Zhang et al., 2023). For SynthID-Image, we hand-picked and -tuned a large variety of transformations, as detailed in Section 4.
#### Payload.
Beyond detection, we require watermarks to include an additional multi-bit payload. Again, we expect this payload to be robust to everyday content changes. This is commonly evaluated using bit accuracy, i.e., accuracy of predicting individual bits, or code accuracy, i.e., accuracy of predicting a full multi-bit code. While most state-of-the-art methods (Fernandez et al., 2023; Zhang et al., 2019; Yang et al., 2024) directly tackle this multi-bit watermarking problem, the exact use of the payload at test time is rarely discussed and thus evaluation usually focuses on detection only. For SynthID-Image, we tackle the detection and payload problems separately and found the payload to play a crucial role in how provenance is established, see Section 5.
#### Security.
The security of a watermarking system entails both worst-case robustness of the watermark as well as security of the watermarking scheme itself. This includes a variety of threat models, including evasion of watermark detection, watermark forgery, model stealing or secret extraction attacks. Depending on the threat model, ensuring security might result in a number of secondary desiderata. Avoiding evasion and forgery attacks on the decoder generally requires an adversarially robust detector (see Section 2.3). Thus, adversarial robustness is an important but not the only part of security. For example, to avoid model extraction attacks we may require generated watermarks to be sufficiently diverse across different payloads or hard to train surrogate models on (Orekondy et al., 2020). Some aspects of security can also be addressed using infrastructure changes such as limiting the number of queries per second for the detector or providing only an encoder to trusted customers. We give a comprehensive overview of threat models and their practical relevance for SynthID-Image in Section 6.
#### Efficiency.
Deploying watermarking at scale requires reasonably low latency for both encoding and decoding the watermark. For encoding, this is usually measured relative to the latency of content generation. Because this is often user-facing, we aim for a single-digit percentage of additional overhead. For decoding, it is important to realize that it may be run orders of magnitude more frequently than encoding. This fact is commonly ignored in the literature where decoding is framed as an ad-hoc action initiated by a user who wants to check for the watermark. However, we expect watermark decoders to be run at increasingly large scale to check the majority of content that is shared on the web, e.g., as part of social media, web search and scraping, training data creation, etc. Thus, decoding efficiency might often be the bottleneck. It is also important to realize that decoding may be run in addition to other checks such as verifying C2PA (Coalition for Content Provenance and Authenticity, 2023) metadata and content pre- or post-processing (e.g., video encoding/decoding). We benchmark efficiency in terms of throughput per second and hardware unit (e.g., GPU). At encoding time, batch size will be pre-determined by generation requests; for decoding, batch size can be chosen to optimize throughput.
#### Deployment.
As SynthID-Image is intended for internet-scale deployment, it clearly distinguishes itself from prior work (see Section 9 for a discussion). This introduces a number of additional requirements not sufficiently considered previously. Most importantly, different deployment settings â including internal deployment, watermarking-as-a-service or open watermarking models â have significant influence on the above desiderata and key design choices. This includes proper decision making, versioning of watermarks and how SynthID-Image fits into a larger landscape of methods for provenance, e.g., C2PA or search-based methods (Balan et al., 2023; Bharati et al., 2021; Zhang et al., 2020b; Nguyen et al., 2021). We found that such deployment considerations had the largest impact on deciding what approach to use and which operating point in terms of the above desiderata to choose.
### 2.2 Post-hoc, model-independent watermarking
Watermarking schemes can be categorized along several, partly orthogonal dimensions. Because we focus on blind watermarking only, watermarking has to shift the distribution of generated content in a detectable manner. Thus, blind watermarking is also referred to as distributional watermarking. Additionally, recent literature typically distinguishes between ad-hoc and post-hoc watermarking depending on whether the process of watermarking happens during the generative process (ad-hoc) (Fernandez et al., 2023; Wen et al., 2023; Yang et al., 2024) or as a post-processing step (post-hoc) (Wen and Aydöre, 2019; Hayes and Danezis, 2017; Hayes et al., 2020; Tancik et al., 2020; Luo et al., 2020; Bui et al., 2025). Ad-hoc methods are typically integrated into the corresponding generative model and thus require at least partial access to the underlying generative model. Post-hoc watermarking, in contrast, can be performed without any knowledge or assumptions on the generative model and is thus model-independent. We deliberately designed SynthID-Image as a post-hoc, model-independent approach, a choice largely based on deployment considerations while ensuring the best trade-off between the remaining desiderata.
#### Advantages.
First and foremost, SynthID-Image must consistently watermark all of Googleâs current and future AI-generated content, stemming from a variety of different models. We believe that the promise of watermarking in the context of preventing misinformation stems from this consistency, i.e., being able to mark all content using a single (or as few as possible) watermarking scheme(s). Because ad-hoc approaches are inherently tied to the respective generative model architectures and versions, they are unable to fulfill this promise at scale. For example, having different watermarks for different generative models or versions necessitates running many multiple detectors or cleverly training âmeta-detectorsâ. With a post-hoc approach, this can be handled using a payload. We also found that this creates less organizational dependencies and allows more flexibility in how the watermarking system is deployed internally and externally. For example, in contrast to most ad-hoc approaches, SynthID-Image could be open-sourced or deployed as-a-service without implications for any generative model. Moreover, it is easier to debug, update or simply turn on or off. We appreciate that some ad-hoc methods can be used in a post-hoc way (Fernandez et al., 2023); but the dependence on details of the underlying generative model remains, combining the disadvantages of ad- and post-hoc approaches rather than their advantages.
#### Drawbacks.
The main disadvantage of post-hoc approaches is that they are inherently lossy, i.e., they impact the quality of the generated content. This is in contrast to ad-hoc approaches which can be completely lossless (Yang et al., 2024). However, we found that reaching a level of quality where the watermark is essentially invisible to (expert) users is possible in practice. We also believe that the impact of watermarking on quality is also easier to measure for post-hoc approaches. This is because we can directly compare watermarked and non-watermarked content to judge invisibility, as done in our human studies, see Section 3. For ad-hoc approaches, by construction, judging quality is more entangled with the quality of the underlying generative model. Ad-hoc approaches risk reducing diversity of generations (Gunn et al., 2025), which is incredibly difficult to test for.
There are various other implications of our choice. For example, post-hoc watermarking causes additional computational overhead for adding the watermark compared to ad-hoc approaches. This overhead can be limited using architectural changes, but cannot be entirely eliminated. This is rather critical for image generation where watermarking has to be performed after generation; for audio and video, generation and watermarking can be parallelized to a large extent. On the other hand, watermark detection can be less efficient for ad-hoc methods, for example, if they require expensive inversion of the diffusion process (Wen et al., 2023; Yang et al., 2024). As we expect detection to be run several orders of magnitude more often, this is an important consideration. We also noticed that robustness of post-hoc approaches is easier to benchmark, since content to be watermarked can be fixed independent of the generative model, and more controllable. Watermark robustness of ad-hoc approaches may be entangled with internal representations of the underlying generative models.
### 2.3 Training formulation
As a post-hoc scheme, we used an encoder-decoder approach for SynthID-Image following prior work (Wen and Aydöre, 2019; Hayes and Danezis, 2017; Hayes et al., 2020; Tancik et al., 2020; Luo et al., 2020; Bui et al., 2025, 2023). To be more specific, let ${\bm{x}}âX$ be an image, where $Xââ^n$ represents the set of all images (in practice, the set of all images is finite and countable). We assume there exists a metric $d:XĂXââ_+$ that encodes the perceptual similarity between two images. We will use this metric to formalize the concept of invisibility of a watermark. A post-hoc watermarking scheme is a pair $âš f,gâ©$ consisting of an encoder function $f:XâX$ , which adds an identification mark, and a decoder function $g:Xâ\{± 1\}$ , which tries to detect if the mark is present. Let $â_X$ be a target distribution that we would like to watermark. Given a perceptual threshold $Δ>0$ , the optimal watermarking scheme on $â_X$ solves
$$
\displaystyle\min_f,g \displaystyleE_{\bm{x}âŒâ_X}â€ft[\ell(g(f({\bm{x}})),+1)+\ell(g({\bm{x}}),-1)\right] \displaystyle \displaystyle d({\bm{x}},f({\bm{x}}))â€Î”, \tag{1}
$$
where $\ell$ represents a suitable classification loss function. Assuming that $g$ makes a decision by thresholding some logit using a threshold $Îș$ , the classification loss we are ideally interested in is $\ell(g_Îș(f({\bm{x}})),k)=\mathbbm{1}[sign(g_Îș(f({\bm{x}})))=k]$ . In practice, an appropriate approximation such as cross-entropy is used to train $g$ and the threshold $Îș$ is calibrated for a target FPR post-training. Minimizing this loss will ensure that the watermarking scheme is effective, i.e., is able to watermark content and detect them. The constraint on $d$ , in contrast, will ensure quality, i.e., the watermark being invisible. It is important to note that Equation (1) assumes that images to watermark and non-watermarked images come from the same distribution $â_X$ . While we make this assumption to keep our watermarking scheme future-proof and universal across various generative models, this is clearly sub-optimal when targeting specific AI-generated content (e.g., coming from a family of text-to-image models). We account for this discrepancy during the calibration process which is applied post-training.
We are particularly interested in robust watermarking schemes. That is, schemes that are robust to everyday alterations of watermarked samples that still preserve the usefulness and semantic content of the samples. We define a distribution of valid transformations $â_T$ that semantically preserve the content and require robustness on average, i.e.,
$$
\displaystyle\min_f,g \displaystyleE_{\bm{x}âŒâ_X,ÏâŒâ_T}â€ft[\ell(g(Ï(f({\bm{x}}))),+1)+\ell(g(Ï({\bm{x}})),-1)\right] \displaystyle \displaystyle d({\bm{x}},f({\bm{x}}))â€Î”. \tag{2}
$$
The difficulty of this problem is clearly determined by the perceptual threshold $Δ$ and the distribution over transformations $â_T$ , resulting in a robustness-quality trade-off. Clearly, allowing more visible watermarks will render the system more robust to stronger transformations. Note again that Equation (2) assumes that the same transformations are applied to watermarked and non-watermarked images. This is not necessarily the case in practice but it simplifies analysis and evaluation of the resulting scheme.
For a watermarking scheme to be secure, we might also need to be robust to specific malicious modifications of the content. Attackers may want to manipulate the images ${\bm{x}}$ or $f({\bm{x}})$ to provoke false positives and false negatives, respectively. This leads to a worst-case formulation of the above optimization problem that requires adversarial training (Madry et al., 2018) which remains largely unsolved. For SynthID-Image, we found that it is sufficient to enforce adversarial robustness only in settings where adversaries have limited information (e.g., black-box access) and constrained compute. As training a perfectly robust and secure watermarking scheme may be infeasible, this shifts the focus to making malicious attacks as difficult as possible.
An important aspect of a useful provenance system is the ability to discern content from different actors. As such, having the ability to embed a hidden message, called payload, in the watermark is fundamental. In this setting, training a watermarking scheme becomes a multi-bit classification problem rather than a detection problem. More formally, let ${\bm{c}}â\{± 1\}^C$ be a $C$ -bit binary payload. The encoder function $f:XĂ\{± 1\}^CâX$ now crafts a watermark based on both ${\bm{x}}$ and ${\bm{c}}$ and the decoder function $g:Xâ\{± 1\}^C$ is now tasked with recovering the payload ${\bm{c}}$ rather than detecting the presence of the watermark alone. The robust watermarking problem then becomes:
$$
\displaystyle\min_f,g \displaystyleE_{\bm{x}âŒâ_X,{\bm{c}}âŒU_\{± 1\^C},ÏâŒâ_T}â€ft[\ell(g(Ï(f({\bm{x}},{\bm{c}}))),{\bm{c}})\right] \displaystyle \displaystyle d({\bm{x}},f({\bm{x}}))â€Î”, \tag{3}
$$
where $U_A$ represents the uniform distribution over the set ${A}$ . Clearly, this problem becomes more difficult for longer payloads, i.e., larger $C$ . This means we are now dealing with a robustness-quality-payload trade-off. For SynthID-Image, we solve both the detection and the payload problem, i.e., Equations (2) and (3), separately, without making any assumptions on the payload ${\bm{c}}$ such that it can serve a variety of purposes.
### 2.4 Benchmarking to control trade-offs
#### Primary trade-off.
The most important trade-off between our desiderata is the robustness-quality-payload trade-off. More visible watermarks are easier and more robust to detect and can store more bits. In a nutshell, we want the watermark to be as robust and contain as many bits as possible while staying invisible. As we will see in Section 3, the main bottleneck of this approach is objectively ensuring invisibility through appropriate metrics and studies. In practice, we control these trade-offs using a combination of different losses, data augmentation methods and payload encoding strategies. Moreover, the data distribution has a large impact on this trade-off. For example, requiring invisible watermarks can be harder for some images (e.g., with less structure). Requiring high quality on these examples leads to less robustness overall. Thus, deciding to not watermark specific content can improve the trade-off significantly.
#### Secondary trade-offs.
Besides the robustness-quality-payload trade-off, there are various secondary trade-offs. For example, requiring adversarial robustness or preventing watermark extraction or exchange attacks may worsen quality. Another example is using a cascade of decoders, with more efficient decoders offering less robustness first, which allows to trade-off robustness and efficiency.
For SynthID-Image, we found that the most effective way of controlling these trade-offs is proper end-to-end benchmarking of all desiderata of interest. While we will discuss appropriate metrics in detail in the following sections, we want to emphasize that end-to-end benchmarking in itself is a non-trivial endeavor. Critically, benchmarking needs to include all key metrics and threat models to highlight relevant trade-offs. This requires a high degree of automation and tooling around model development. Training and evaluation will also inherently include randomness. It is important to be able to control this randomness appropriately for debugging and reproducibility. For transformations and attacks this also includes appropriately tuning hyper-parameters. Moreover, the choice of datasets for evaluation is important: key considerations are whether to benchmark on natural or generated images and how to include corner-case content, especially for quality evaluation.
## 3 Invisibility
Quality is the most difficult requirement to properly evaluate in practice because âinvisibiliyâ or any quality change in generated content is tricky to objectively quantify. So far, in Section 2.3, we worked with a perceptual metric $d$ (many prior publications also work with an unknown oracle instead; Zhang et al., 2023). Of course, there exist automated metrics and proxies, including PSNR, SSIM (Wang et al., 2004) or pre-trained perceptual models such as L-PIPS (Zhang et al., 2018), FID (Heusel et al., 2017), CLIP (Radford et al., 2021) or CMMD (Jayasumana et al., 2024). However, none of these metrics can ultimately replace human evaluation. This is particularly relevant when dealing with a wide distribution of generated content that might not match benign, natural content. Learned metrics will always be biased to the content distribution that these metrics have been developed for. In the case of images, these are typically ImageNet-style natural images. For SynthID-Image, we decided to go with rather basic metrics such as PSNR or SSIM, using them with extreme caution and mostly to perform relative comparisons while relying heavily on human evaluation, both internally and externally, to decide invisibility in absolute terms. The latter also includes hand-tuning relevant transformations considered during robustness evaluation, as detailed in Section 4. While we also rely heavily on standard datasets, we took particular care to run human studies on actual AI-generated content from state-of-the-art generative models, using a diverse set of prompts.
#### Setups.
Independent of the metric, there are two modes of quality evaluation. First, we can compare pairs of original and watermarked content. This side-by-side comparison allows to directly quantify a perceptual distance and is the default for most post-hoc approaches where we have a one-to-one correspondence between original and watermarked content. In Equation (2), this means directly evaluating the distance $d$ . Second, as users do not typically have access to the original content alongside the watermarked content, we can also compare the distributions of a metrics. This is how ad-hoc approaches are typically evaluated, e.g., by computing FID or CLIP scores for original and watermarked content and performing hypotheses tests to see if both distributions are distinguishable (Yang et al., 2024). However, using CLIP also assumes access to the corresponding prompts and is thus limited to ad-hoc approaches. This approach is typically more difficult as we cannot directly judge, for a fixed prompt or image, to what extent quality changes. For SynthID-Image, we generally found the first approach to be more appropriate for internal evaluation as it allows for more scrutiny. For studies with external users, in contrast, we follow the second approach because it represents the more realistic setting.
#### Internal rater studies.
More specifically, during fast-paced development, we used a reasonably small set of generated images to run side-by-side comparison studies within the team for most model candidates. This involves showing the original and watermarked content side-by-side, randomizing which is the watermarked and asking ourselves to identify the watermarked one. Across the team, we usually target a performance close to random chance $50\$ . This is a particularly high bar since watermarks are easier to spot side-by-side, with the ability to zoom in and out of the image. Moreover, throughout development, we learned to spot watermarks well as we usually know how watermarks of specific model versions look. Over time, as watermarks got less visible, we also added an option to abstain. We included hand-picked examples of generated content, often those that we found difficult to watermark or informed by direct feedback from colleagues working on the corresponding generative models. This also contributed to these studies being extremely conservative. In between these studies, we typically compare models in relative terms using PSNR, SSIM or similar automatic metrics, especially to hill-climb on quality or other desiderata.
\begin{overpic}[width=65.55084pt]{assets/categories/3d_rendering.png} \put(0.0,102.0){\bf 3D Rendering} \end{overpic}
\begin{overpic}[width=65.55084pt]{assets/categories/abstract_art.png} \put(0.0,102.0){\bf Abstract art} \end{overpic}
\begin{overpic}[width=65.55084pt]{assets/categories/animal_painting.png} \put(0.0,102.0){\bf Animal Painting} \end{overpic}
\begin{overpic}[width=65.55084pt]{assets/categories/animal_photo.png} \put(0.0,102.0){\bf Animal Photo} \end{overpic}
\begin{overpic}[width=65.55084pt]{assets/categories/cartoon.png} \put(0.0,102.0){\bf Cartoon} \end{overpic}
\begin{overpic}[width=65.55084pt]{assets/categories/defocused_blurry.png} \put(0.0,102.0){\bf Defocused Blurry} \end{overpic}
\begin{overpic}[width=65.55084pt]{assets/categories/fantasy_painting.png} \put(0.0,102.0){\bf Fantasy Painting} \end{overpic}
\begin{overpic}[width=65.55084pt]{assets/categories/flat_art.png} \put(0.0,102.0){\bf Flat art} \end{overpic}
\begin{overpic}[width=65.55084pt]{assets/categories/historic_photo.png} \put(0.0,102.0){\bf Historic Photo} \end{overpic}
\begin{overpic}[width=65.55084pt]{assets/categories/landscape_painting.png} \put(0.0,102.0){\bf Landscape Painting} \end{overpic}
\begin{overpic}[width=65.55084pt]{assets/categories/landscape_photo.png} \put(0.0,102.0){\bf Landscape Photo} \end{overpic}
\begin{overpic}[width=65.55084pt]{assets/categories/line_drawing.png} \put(0.0,102.0){\bf Line Drawing} \end{overpic}
\begin{overpic}[width=65.55084pt]{assets/categories/logo.png} \put(0.0,102.0){\bf Logo} \end{overpic}
\begin{overpic}[width=65.55084pt]{assets/categories/object_painting.png} \put(0.0,102.0){\bf Object Painting} \end{overpic}
\begin{overpic}[width=65.55084pt]{assets/categories/object_photo.png} \put(0.0,102.0){\bf Object Photo} \end{overpic}
\begin{overpic}[width=65.55084pt]{assets/categories/pencil_drawing.png} \put(0.0,102.0){\bf Pencil Drawing} \end{overpic}
\begin{overpic}[width=65.55084pt]{assets/categories/pixel_art.png} \put(0.0,102.0){\bf Pixel art} \end{overpic}
\begin{overpic}[width=65.55084pt]{assets/categories/portrait_painting.png} \put(0.0,102.0){\bf Portrait Painting} \end{overpic}
\begin{overpic}[width=65.55084pt]{assets/categories/portrait_photo.png} \put(0.0,102.0){\bf Portrait Photo} \end{overpic}
\begin{overpic}[width=65.55084pt]{assets/categories/poster_art.png} \put(0.0,102.0){\bf Poster Art} \end{overpic}
\begin{overpic}[width=65.55084pt]{assets/categories/scifi_photo.png} \put(0.0,102.0){\bf SciFi Photo} \end{overpic}
\begin{overpic}[width=65.55084pt]{assets/categories/shapes.png} \put(0.0,102.0){\bf Shapes} \end{overpic}
\begin{overpic}[width=65.55084pt]{assets/categories/stock_photo.png} \put(0.0,102.0){\bf Stock Photo} \end{overpic}
\begin{overpic}[width=65.55084pt]{assets/categories/text.png} \put(0.0,102.0){\bf Text} \end{overpic}
\begin{overpic}[width=65.55084pt]{assets/categories/wallpaper.png} \put(0.0,102.0){\bf Wallpaper} \end{overpic}
Figure 2: Illustrative images generated using Imagen for different categories of prompt. These images (either watermarked or not) are shown to external human raters and include corner-case images such as grayscale photographs, sketches or close-to-uniform color images.
#### External rater studies.
For evaluation using external raters, this side-by-side approach was replaced by showing either the original or the watermarked content and asking raters to find âartifactsâ. This is the more realistic setup, simulating that users usually do not have access to pairs of original and watermarked content. In contrast to the very specific, hand-picked content examples in our internal evaluation, we constructed a set of hundreds of prompts across various categories. For our image watermark evaluation, example categories are shown in Figure 2 with images generated (and watermarked) using Imagen and SynthID-Image. These categories also illustrate that the distribution of generated images tends to be significantly different from natural images; this, however, reflects many use-cases of such generative models. In the study, we also included training examples for the raters to highlight common types of artifacts. Here, it was important to include both watermark-related artifacts as well as generation-related artifacts. We also had to use dummy examples to double-check that raters pay attention. If, after the study, we cannot distinguish the rate of artifacts between the watermarked and non-watermarked with statistical significance, we concluded the watermark to be invisible.
#### Additional considerations.
There are several considerations that affect quality. For images, this mainly concerns resolution and color space. While resolution could be set by the corresponding generative model whose output must be watermarked; we intend SynthID-Image to be independent of such dependencies as generative models will likely change going forward. While higher resolution allows less visible watermarks at similar robustness and payload size, we found that efficiency constraints ended up determining resolution. This choice required taking care of not introducing any artifacts due to re-sizing during encoding. The color space largely impacts which pixel-level changes are actually perceptible. We found this matters particularly for black-and-white images or images with few colors or low contrast.
#### Corner-cases.
An important realization in terms of quality, but also relevant for robustness and security, is the frequency of corner-case imagery when working at internet scale. We already mentioned the tricky case of black-and-white images â typically ignored in the literature. Other examples include logos with very few but uniform coloring, images with slow gradients, abstract paintings, pixel art, sparse drawings, etc. See Figure 2 for some examples. These types of images are rarely included in standard vision datasets that perceptual metrics are trained on. As corner-cases, we generally define any category of content that poses a challenge to watermarking in terms of quality and is not well represented in many standard datasets. This might mean that such content is difficult to watermark in an invisible way, or that the watermark becomes fragile in terms of robustness. For SynthID-Image, we found it important to identify and track such content throughout development. Large parts of the content used for internal studies included such corner cases.
## 4 Transformation robustness
To consistently detect watermarks, it is important that they are robust to common transformations and content post-processing. This generally includes a pre-determined set of transformations that we expect to be readily available on personal computers or smartphones, including resizing or cropping, quantization and compression, or common image processing filters (e.g. Instagramâs photo filters). This is similar to the notion of corruption robustness in the computer vision literature (Hendrycks and Dietterich, 2019; Hendrycks et al., 2019) and has to be contrasted with adversarial transformations (Engstrom et al., 2017; Alaifari et al., 2019) that assume malicious intent. Concretely, this means that we expect robustness to a random transformation with reasonable strength.
The main metrics to evaluate detection performance are true and false positive rates (TPR and FPR) with and without transformations. Formally, given a distribution ${â}_T$ over content transformations, we can measure the watermarking schemeâs nominal rates as
| | $\displaystyleE_\begin{subarray{c}{\bm{x}}âŒâ_X,ÏâŒ{â}_T\end{subarray}}â€ft[g_Îș(Ï({\bm{x}}))\right] \textrm{(TPR)} E_\begin{subarray{c}\bar{{\bm{x}}}âŒâ_\bar{X}\end{subarray}}â€ft[g_Îș(\bar{{\bm{x}}})\right] \textrm{(FPR)}$ | |
| --- | --- | --- |
with ${â}_\bar{X}$ representing the distribution over all possible non-watermarked content and $g_Îș$ indicating the decoder at the operating point (commonly a threshold) $Îș$ . Note that we can also measure the worst-case across transformations for analysis purposes (replacing $E_ÏâŒ{â_T}$ with $\max_ÏâŒ{â_T}$ ). Some papers also report the area under the receiver operating characteristic (AUROC), considering TPR and FPR across various detection thresholds $Îș$ . However, we found that AUROC is not sensitive enough in practice. This is because we are particularly interested in operating points with very low FPR (i.e., $\ll 1\$ ), disregarding the majority of the ROC curve. Note that prior work (Fernandez et al., 2023; Yang et al., 2024) often simulates results at such low FPRs based on assumptions of the detection/payload logits distribution. But this has been found to be misleading (Fernandez et al., 2023) which is why we rely purely on sufficiently large datasets to avoid simulation. For evaluating TPR and FPR consistently across transformations, it is important to choose the operating point $Îș$ consistently across transformations. This is because, for deployment, we have to choose a single $Îș$ ; evaluation on different subsets of transformations with different operating points can thus be misleading. To the best of our knowledge, this is neglected in prior work, resulting in overly optimistic robustness evaluations.
\begin{overpic}[width=65.55084pt]{assets/transforms/worst/all_rotation_worst.png}\put(0.0,102.0){\bf All rotations}\end{overpic}
\begin{overpic}[width=65.55084pt]{assets/transforms/worst/brightness_worst.png}\put(0.0,102.0){\bf Brightness}\end{overpic}
\begin{overpic}[width=65.55084pt]{assets/transforms/worst/combined_worst.png}\put(0.0,102.0){\bf Combined}\end{overpic}
\begin{overpic}[width=65.55084pt]{assets/transforms/worst/combined_no_crop_worst.png}\put(0.0,102.0){\bf Combined NoCrop}\end{overpic}
\begin{overpic}[width=65.55084pt]{assets/transforms/worst/combined_no_crop_rotate_worst.png}\put(0.0,102.0){\bf Combined NoCrop Rotate}\end{overpic}
\begin{overpic}[width=65.55084pt]{assets/transforms/worst/combined_rotate_worst.png}\put(0.0,102.0){\bf Combined Rotate}\end{overpic}
\begin{overpic}[width=65.55084pt]{assets/transforms/worst/contrast_worst.png}\put(0.0,102.0){\bf Contrast}\end{overpic}
\begin{overpic}[width=65.55084pt]{assets/transforms/worst/crop_resize_worst.png}\put(0.0,102.0){\bf Crop resize}\end{overpic}
\begin{overpic}[width=65.55084pt]{assets/transforms/worst/denoise_worst.png}\put(0.0,102.0){\bf Denoise}\end{overpic}
\begin{overpic}[width=65.55084pt]{assets/transforms/worst/emoji_overlay_worst.png}\put(0.0,102.0){\bf Emoji overlay}\end{overpic}
\begin{overpic}[width=65.55084pt]{assets/transforms/worst/exposure_worst.png}\put(0.0,102.0){\bf Exposure}\end{overpic}
\begin{overpic}[width=65.55084pt]{assets/transforms/worst/file_format_worst.png}\put(0.0,102.0){\bf File format}\end{overpic}
\begin{overpic}[width=65.55084pt]{assets/transforms/worst/flip_left_right_worst.png}\put(0.0,102.0){\bf Flip left-right}\end{overpic}
\begin{overpic}[width=65.55084pt]{assets/transforms/worst/flip_up_down_worst.png}\put(0.0,102.0){\bf flip up-down}\end{overpic}
\begin{overpic}[width=65.55084pt]{assets/transforms/worst/gaussian_blur_worst.png}\put(0.0,102.0){\bf Gaussian blur}\end{overpic}
\begin{overpic}[width=65.55084pt]{assets/transforms/worst/gaussian_noise_worst.png}\put(0.0,102.0){\bf Gaussian noise}\end{overpic}
\begin{overpic}[width=65.55084pt]{assets/transforms/worst/grayscale_worst.png}\put(0.0,102.0){\bf Grayscale}\end{overpic}
\begin{overpic}[width=65.55084pt]{assets/transforms/worst/hue_worst.png}\put(0.0,102.0){\bf Hue}\end{overpic}
\begin{overpic}[width=65.55084pt]{assets/transforms/average/identity.png}\put(0.0,102.0){\bf Identity}\end{overpic}
\begin{overpic}[width=65.55084pt]{assets/transforms/worst/impulse_noise_worst.png}\put(0.0,102.0){\bf Impulse noise}\end{overpic}
\begin{overpic}[width=65.55084pt]{assets/transforms/worst/instagram_worst.png}\put(0.0,102.0){\bf Instagram}\end{overpic}
\begin{overpic}[width=65.55084pt]{assets/transforms/worst/light_text_overlay_worst.png}\put(0.0,102.0){\bf Light text overlay}\end{overpic}
\begin{overpic}[width=65.55084pt]{assets/transforms/worst/resize_worst.png}\put(0.0,102.0){\bf Resize}\end{overpic}
\begin{overpic}[width=65.55084pt]{assets/transforms/worst/rotation_worst.png}\put(0.0,102.0){\bf Rotation}\end{overpic}
\begin{overpic}[width=65.55084pt]{assets/transforms/worst/saturation_worst.png}\put(0.0,102.0){\bf Saturation}\end{overpic}
\begin{overpic}[width=65.55084pt]{assets/transforms/worst/sharpness_worst.png}\put(0.0,102.0){\bf Sharpness}\end{overpic}
\begin{overpic}[width=65.55084pt]{assets/transforms/worst/shot_noise_worst.png}\put(0.0,102.0){\bf Shot noise}\end{overpic}
\begin{overpic}[width=65.55084pt]{assets/transforms/worst/small_rotation_worst.png}\put(0.0,102.0){\bf Small rotation}\end{overpic}
\begin{overpic}[width=65.55084pt]{assets/transforms/worst/speckle_noise_worst.png}\put(0.0,102.0){\bf Speckle noise}\end{overpic}
\begin{overpic}[width=65.55084pt]{assets/transforms/worst/zoom_out_worst.png}\put(0.0,102.0){\bf Zoom out}\end{overpic}
Figure 3: Illustrative examples of the 30 âbasicâ transformations used in our evaluation (alphabetically listed).
Prior work on watermarking quickly realized that properly specifying and training for relevant transformations is key for robustness. Thus, many previous works include concrete transformations and parameters to be tested for robustness (Tancik et al., 2020; Wen et al., 2023; Zhang et al., 2024). However, we found that none of these works are exhaustive, as also seen in the surveyed methods and transforms by Wan et al. (2022). Commonly overlooked transformations can be simple ones, such as small rotations, left/right and up/down flips, or different types of noise (Gaussian, salt&pepper, speckle). For example, GaussianShading (Yang et al., 2024) misses rotations and flips; StableSignature (Fernandez et al., 2023) and WAVES (An et al., 2024) ignore various noise types. This is despite common knowledge that these type of transformations are extremely common and work on corruption robustness indicates that deep neural networks are not inherently robust to them. Moreover, many methods are benchmarked against âfixed-strengthâ transformations, e.g., fixed rotations or crops, or a fixed JPEG quality (Fernandez et al., 2023; Saberi et al., 2024b; Yang et al., 2024). While these usually correspond to rather strong transformations, it is not guaranteed that the methods are also the least robust against them; for example, neural networks may be more vulnerable to small rotations than 90, 180 or 270 degree rotations (Engstrom et al., 2017; Dumont et al., 2018). It has also been shown that combinations of transforms can be effective ways to fool networks. For example, StableSignature has combinations based on cropping, brightness changes and JPEG compression. As generalization of neural networks between transformations is still poorly understood, a key component of SynthID-Image is exhaustively listing, implementing and tuning relevant transformations. We developed a set of âbasicâ transformations that we believe any large-scale watermarking system should be robust to. Our two guiding principles for inclusion in this basic set of transformations are accessibility and detectability. Rotations, flips, brightness changes, etc. are all very accessible on every smartphone camera or social media app. This also includes Instagram-like filters https://github.com/akiomik/pilgram or overlaying texts and logos. To tune strengths, we consider all transformations reasonable if they have reasonable quality and preserve the underlying semantics of the content (as judged by human raters). It is also important to consider any possible transformations that any software might automatically include as pre- or post-processing, such as compression and quantization. All of our basic transformations are shown in Figure 3.
## 5 Payload
For SynthID-Image, where we aim to serve several generative models across different partners, the payload is a crucial component of turning watermarking into a tool for provenance. While some recent work (Yang et al., 2024) simulates multiple users, details are scarce and suggest that a simple integer user identifier is encoded in the payload, without any redundancy or additional logic.
During development, we evaluate the payload robustness in terms of bit and code accuracy, across all relevant transformations. Letting ${\bm{c}}=(c_1,âŠ,c_C)$ be the originally encoded payload code and $g_i(·)$ the logit corresponding to $c_i$ (with $g_0(·)$ corresponding to the detection logit), we define bit accuracy as the accuracy of the predicted payload bits in expectation over bit positions:
$$
\displaystyleE_{\bm{x}âŒâ_X,ÏâŒâ_T}â€ft[\frac{1}{C}â_i=1^C\mathbbm{1}[(g_i(Ï(f({\bm{x}}))))=c_i]\right]. \tag{4}
$$
Bit accuracy is comparable across models with different payload sizes. In some cases, it is also interesting to monitor bit accuracy individually as their accuracy can vary widely by location depending on the exact training procedure. This might inform how to robustly encode information in the payload in practice. Bit accuracy does not explicitly measure what we might be interested in, which is how often we get the entire payload right, i.e., code accuracy:
$$
\displaystyleE_{\bm{x}âŒâ_X,ÏâŒâ_T}â€ft[\mathbbm{1}[â_i=1^C(g_i(Ï(f({\bm{x}}))))=c_i]\right]. \tag{5}
$$
Code accuracy generally degrades with larger payload size and degrades more quickly with challenging transformations than bit accuracy. As before, it is important to monitor these metrics across transformations. This is because bit and code accuracy are highly relevant for many deployment scenarios where pure detection performance in terms of TPR/FPR is not sufficient. Note that both metrics are defined conditional on watermarked examples. That is, we generally neglect what happens to the payload on non-watermarked content. This is somewhat unique to SynthID-Image, as we explicitly separate detection and payload recovery (see Section 2.3). For approaches that directly use the payload for detection, TPR and FPR needs to be defined following Equation (5). For example, Yang et al. (2024); Fernandez et al. (2023) select a random but fixed payload for each watermarked example and make a detection decision by thresholding on the number of matching bits between this payload and the predicted one. Of course, if the reference payload is not available, this test is not applicable.
In the setting from Yang et al. (2024) where each user has an integer identifier, code accuracy becomes the relevant metric for provenance (in this case, watermark detection and payload recovery coincide). Essentially, this turns watermark detection into a multi-class problem, where the number of classes grows with the number of users. For SynthID-Image, where we deliberately disentangled watermark detection and payload decoding for flexibility, we do not always need to invoke the logic for decoding the payload.
## 6 Ensuring security
There is a wide range of attacks on watermarking systems that SynthID-Image has to defend against (Zhao et al., 2025). In the following, we discuss the relevant threat models we considered. A threat model describes an attackerâs objective, capabilities, and knowledge. Even though attackers might have different objectives, they often use a common set of tools that can inform which defenses to prioritize.
### 6.1 Threat Models: Objectives, Capabilities, and Tools
An attackerâs primary objectives are either watermark removal (creating a false negative) to obscure content and possibly claim ownership or watermark forgery (creating a false positive) to wrongly attribute ownership of a potentially incriminating piece of content. Further objectives that aim at facilitating these goals include model extraction (TramĂšr et al., 2016; Oh et al., 2019; Orekondy et al., 2019) to steal the models themselves (or information about them, e.g., hyper-parameters,architecture details), secret extraction (Jovanovic et al., 2024) to find hidden keys (e.g., used for payload encoding), or payload attacks to manipulate the embedded information.
The success of these attacks depends on the attackerâs capabilities. This includes their level of access, ranging from black-box (API-only) to white-box (full model weights and architecture). An attacker may also have privileged access to secrets (e.g., private keys) or possess paired information (both original and watermarked content), which significantly simplifies attacks, especially for post-hoc systems like SynthID-Image. Finally, their available resources, such as query limits and computational power, constrain the feasibility of an attack. To execute an attack, adversaries can employ various tools, largely from the adversarial machine learning community (Papernot et al., 2018). (Universal) adversarial examples
These are inputs with small, often imperceptible, perturbations designed to cause misclassification (Szegedy et al., 2014; Carlini and Wagner, 2017b). For watermarking, they can be used to make the decoder fail (removal) or fire incorrectly (forgery). While defenders aim for invisibility, attackers may tolerate visible artifacts, making attacks easier (Brown et al., 2017). Surrogate models
Since adversarial examples often transfer between different models, an attacker can train a local substitute model to craft attacks in a black-box setting (Papernot et al., 2017; TramĂšr et al., 2016). Re-generation attacks
These attacks use other powerful generative models (like diffusion models) to reconstruct a watermarked image, potentially washing out the watermark in the process (An et al., 2024; Zhao et al., 2024).
### 6.2 Defenses and discussion
The main difficulty of defending against many of the above threat models is anticipating future attacks within relevant threat models (Papernot et al., 2018). These generally depend on how the watermarking scheme is deployed in practice (see Section 7). Our strategy for SynthID-Image has been to defend against attacks that are stronger than what we expect in a real-world deployment. For example, ensuring robustness against white-box attacks provides a strong defense against more realistic black-box threats (Croce and Hein, 2020). We identified relevant attacks through red-teaming and integrated them into our automated evaluation benchmarks to monitor trade-offs.
#### Robust training and fine-tuning.
The foundation of our defense is robust training. The watermark encoder and decoder are trained on a wide range of data augmentations and transformations to help them withstand common image manipulations (Zhu et al., 2018; Hayes, 2020). Robust training may include adversarial training of the decoder (Madry et al., 2018; Kurakin et al., 2017), which aims to harden it against constrained adversarial manipulations. Additionally, for SynthID-Image, we specifically tested and ensured robustness against off-the-shelf weak re-generation attack models (e.g., using variational autoencoders). Ultimately, robustness to a large range of different threat models has been shown to be particularly tricky (TramĂšr and Boneh, 2019; Maini et al., 2020; Stutz et al., 2020), as this will either reduce performance or require a more visible watermark. However, since SynthID-Image is deployed in a proprietary setting, our main goal is to make black-box attacks computationally infeasible at scale, a more achievable goal than defending against a determined white-box adversary.
#### Randomness.
Appropriate randomness in the watermarking process is crucial for countering certain extraction or exchange attacks. For a given generative model, using the same seed and prompt should ideally produce the exact same watermarked output every time. This deterministic behavior prevents an attacker from re-generating the image to reveal differences in watermark patterns, a technique known as a collusion attack. On the other hand, for content that is highly predictable (e.g., due to severe memorization) or simple, an attacker might anticipate the output and attempt to forge or remove a watermark. To mitigate this, the watermark generation process should incorporate stochastic elements, ensuring sufficient diversity in the watermarks applied to semantically similar but non-identical images.
#### Specificity.
To prevent watermark exchange attacks, the watermark pattern should be inextricably linked to the content it marks. We enforce this property by making the watermark generation process content-dependent. During training, we introduce specific loss functions designed to minimize the chance that a watermark extracted from one piece of content can be successfully applied to another. This ensures the watermark is not merely a generic stamp but a specific signature of the content itself.
#### Filtering.
While detecting and filtering various types of adversarial attacks have been shown ineffective (Carlini and Wagner, 2017a; Bryniarski et al., 2022; Stutz et al., 2020), we can decide not to watermark specific content. In text watermarking (Kirchenbauer et al., 2023), for example, zero-entropy generations (e.g., due to memorization) are automatically not watermarked. Similarly, we can decide not to watermark specific corner-case content such as nearly uniform images. Such content can easily be filtered by looking at basic statistics of the content.
## 7 Internet-scale deployment
Our final desiderata, requiring us to actually deploy SynthID-Image in various settings at scale, implies various practical considerations that we found rarely discussed in prior work. Some of these depend on the actual deployment scenario, while others address practicalities like decision making, versioning and combinations with other tools for provenance.
#### Deployment settings.
For SynthID-Image, we considered the following three deployment scenarios: As-an-internal-service:
Here, either encoder, decoder or both are purely used for internal purposes. A common construction is to allow watermarking of content generated by internal models (even if the content is then publicly served) and have detection run only internally, too. A variation of this is to allow internal watermarking but expose detection externally to trusted partners or even the end-user. Note that paired information is only available internally and access to queries and secrets can be tightly controlled. As-a-service:
Here, both encoder and decoder are offered as-a-service to third party companies. In this setting, third parties have access to paired information and potential secrets. Moreover, payloads need to be used to distinguish between different customers all using the same watermarking system. Again, third parties can decide to open the detection part to the end user. Open models:
In this setting, either encoder, decoder or both are publicly available. This makes paired information more widely available and payloads derived from private keys become more important.
In all these scenarios it is important to realize that SynthID-Image has two different components, the encoder and the decoder that are deployed differently. Also, models from the same model family, i.e., multiple SynthID -based models, might be deployed in different scenarios.
The deployment specifics also affect the relevance of certain threat models in practice. This is because the attackerâs knowledge and level of access changes. Internal use has the smallest attack surface as access to models, paired information, and secret keys can be controlled tightly. Giving customers the possibility to produce paired information or opening encoder and/or decoder models increases the attack surface. Along this, threat models like watermark removal/false negatives become less relevant as they are particularly easy to provoke using (universal) adversarial examples against open models. False positives, in contrast, gain in importance and can be reduced through clever payload selection. In the setting where multiple similar models are released through different avenues, open models may also increase the risk to internal models by simplifying surrogate and transfer-based attacks.
#### Decision-making.
Previous work on watermarking used statistical tests for decision making on top of the decoderâs predictions. These approaches are often based on assuming Bernoulli distributions on the individual bits (Fernandez et al., 2023; Yang et al., 2024; Lukas and Kerschbaum, 2023; Yu et al., 2021; Roman et al., 2024) or exploit that diffusion models start from a Gaussian distribution (Zhang et al., 2024; Wen et al., 2023). These are essentially parametric approaches; it has already been shown that individual bits can usually not be modeled using fair coin flips (Fernandez et al., 2023, Appendix B.5). Nevertheless, we believe that proper decision-making is important for deploying watermarking systems at large scale in a responsible and reliable manner. To avoid any parametric assumptions, we found conformal $p$ -values particularly scalable as they are non-parametric and recent work showed how to incorporate robustness considerations (Stutz et al., 2023). Moreover, as this approach is based on a held-out set of calibration examples, it also offers an additional level of configuration before deployment. This can be important when having to react to distributional changes or minor issues for which updating the model is not an option (see Versioning below).
We define the following hypotheses: $H_0$ : âinput is not watermarkedâ and $H_1$ : âinput is watermarkedâ. We then compute valid $p$ -values following Bates et al. (2021, 2023) for both hypotheses using a held-out set of calibration examples $\{{\bm{x}}^(0)_i\}_i=1^n$ and $\{{\bm{x}}^(1)\}_i=1^n$ of non-watermarked and watermarked samples, respectively:
$$
\displaystyleÏ_k({\bm{x}})=\frac{1+â_i=1^nÎŽ[(2k-1)f_0({\bm{x}})â„(2k-1)f_0({\bm{x}}^(k)_i)]}{n+1} \tag{6}
$$
where $f_0(·)$ correspond to the detection logit. As these are valid $p$ -values, i.e., $Ï_k$ is uniformly distributed over $[0,1]$ iff $H_k$ is true, thresholding $Ï_0$ ( $Ï_1$ ) at $α$ guarantees a FPR (TPR) of at most $α$ (at least $1-α$ ). We can control FPR and TPR simultaneously by allowing a variable abstention rate. The calibration set can be updated online if necessary and include different transformations of the same image for robustness. Following Stutz et al. (2023), this reduces the guaranteed FPR (TPR) to $2α$ ( $1-2α$ ); however, this drop is never observed empirically.
Assuming an individual logit $f_j(·)$ for each bit $c_jâ\{± 1\}$ , $jâ\{1,âŠ,C\}$ , we can also compute $p$ -values for each bit $c_j$ being positive or negative. Given an encoded payload ${\bm{c}}=(c_1,âŠ,c_C)$ , we can then test whether the payload is likely the original one using any standard multiple hypothesis testing procedure (Simes, 1986; Hochberg, 1988; Holm, 1979). These methods usually come with a confidence level $α$ which controls the rate of correctly accepting the true payload. However, with larger $C$ , these methods become more conservative. Especially in light of transformations this leads to possibly multiple payloads being accepted. This can be countered using redundant coding techniques. Again, an abstention mechanism can be created easily: if all used payloads ${\bm{c}}$ can be enumerated, we can abstain if multiple payloads are accepted and the number of accepted payloads is often a good mirror of whether the artifact has been tampered with or not.
#### Versioning.
Versioning is an important aspect of deploying at scale and for the long term. In the case of SynthID-Image, the encoder always determines the main version. This is because once watermarked content is out in the wild, we have to be able to detect it. The decoder, in contrast, can be updated much more easily. In a sense this means that we need strict versioning for the encoder while the decoder can be updated continuously. This also implies that threat models that can be addressed in the encoder are more important. For example, once an encoder has been deployed, it is difficult to increase the diversity of watermarks or prevent watermark extraction. The decoder, in contrast, can be updated on the fly to address new attacks. Having the encoder determine the version means that eventually there will be multiple versions in production, assuming that watermarking quality, robustness and payload size will be improved over the coming years. This entails having to detect multiple versions of watermarks over time. This adds additional resource requirements, complicates decision making, and vulnerability might be âinheritedâ between versions.
#### Combination with search and C2PA.
For tackling provenance of AI-generated content, watermarking is a promising but not the only tool at our disposal (Collomosse and Parsons, 2024). Generally, we expect watermarking to be deployed alongside a metadata-based standard like C2PA which major companies vowed to implement. Moreover, there are also technologies based on search and retrieval. For SynthID-Image, we explored various combinations, and focused on fingerprinting, which refers to the practice of computing low-dimensional embeddings for multimedia artifacts and storing them in an internal database. Learning such embeddings in a robust way (Gowal et al., 2021) allows verifying the origin of AI-generated content using similarity search. Various works have applied variations of this idea to attribution and content provenance in the widest sense (Saberi et al., 2024b; Balan et al., 2023; Bharati et al., 2021; Zhang et al., 2020b; Nguyen et al., 2021). In comparison to watermarking, similarity search is more prone to false positives rather than false negatives. On its own, fingerprinting suffers from increased cost for storing embeddings (on top of the embedding cost) and poor scaling as false positives increase with database size. This means that there needs to be retention rules that e.g. store embeddings of generated content only within a specific time-frame. However, as fingerprinting has complementary failure modes, it is natural to combine both approaches. For example, to mitigate forgery attacks, an image must not only contain the correct watermark payload but also match one of the stored embeddings.
## 8 Experimental results
We evaluate the robustness and quality of our external SynthID-Image watermarking model variant, denoted SynthID-O. SynthID-O can encode 136-bit payloads within 512 $Ă$ 512 pixel images. We compare this variant against a wide range of baselines. Our experiments show that SynthID-O establishes new state-of-the-art watermarking performance. Specifically, we find that it outperforms others in terms of quality, considering an extensive human rating study, and shows superior detection performance across a comprehensive list of image transformations. Results are summarized in Figure 1. In this section, we provide further details.
#### Baselines.
To measure detection performance of a given model, we leverage the modelâs detection score if available or use the number of correct payload bits as a score proxy (as proposed previously, e.g., Fernandez et al., 2023). To measure payload recovery performance, we directly measure the bit accuracy for models that provide it. Note that bit accuracy performance depends heavily on the payload length $C$ and this might not be a useful point of comparison. We evaluate all models without any error code correction. We compare against the following: StegaStamp
(Tancik et al., 2020): We use the model checkpoint released in the official repository. https://github.com/tancik/StegaStamp It can encode 100-bit payloads within 400 $Ă$ 400 pixel images. TrustMark
(Bui et al., 2025): Our primary results leverage the TrustMark-Q and TrustMark-P variants. https://github.com/adobe/trustmark/tree/main/python We chose TrustMark-Q over TrustMark-B because, even though it prioritizes image quality over robustness compared to TrustMark-B, our evaluation found it to be more robust overall. All TrustMark variants can encode 100-bit payloads within 256 $Ă$ 256 pixel images. InvisMark
(Xu et al., 2025): We use the model checkpoint released in the official repository. https://github.com/microsoft/InvisMark It can encode 100-bit payloads within 256 $Ă$ 256 pixel images. WAM
(Sander et al., 2025): We use the model checkpoint released in the official repository. https://github.com/facebookresearch/watermark-anything It can encode 32-bit payloads within 256 $Ă$ 256 pixel images. Note that WAM can be used to watermark individual areas within an image, we use it here to watermark full images. As detection score, we use the area of the detection mask returned by the model. For payload recovery, we use the âsemi-hardâ strategy. VideoSeal
(Fernandez et al., 2024): We use the model checkpoints released in the official repository. https://github.com/facebookresearch/videoseal VideoSeal-0.0 and VideoSeal-1.0 can encode 96-bit payloads and 256-bit payloads within 256 $Ă$ 256 pixel images, respectively. Note that VideoSeal was primarily developed to add watermarks to videos. However, it can be used for images. As detection score, we use the number of correct payload bits as using the available detection logit performs badly for the available checkpoints.
For all methods, we use the default parameters provided by the authors in their individual code repositories. We use the watermark rescaling method presented by Bui et al. (2023) with bicubic resizing (which we found to outperform bilinear resizing consistently across all methods).
### 8.1 Quality evaluations
#### Setup.
As explained in Section 3, we perform human evaluations to assess the quality of SynthID-O and the models listed above. For each model, we collect 10,000 ratings (5,000 for watermarked and 5,000 for non-watermarked samples) using 1,000 images generated by Imagen. https://deepmind.google/technologies/imagen-2/ Unlike ImageNet which is used in the robustness evaluation, all images have a resolution of 1536 $Ă$ 1536, which reflects real-world product conditions with the focus placed on quality (as desired by creators). Images are generated across 25 high-level creative categories shown in Figure 2. Each high-level category is further sub-divided into different styles such as âblack & whiteâ and âcolorfulâ for âabstract artâ. All prompts are listed in the appendix (see Section B). We introduce 10 additional dummy images that exhibit clear artifacts. Ratings from raters with less than 70% success rate at correctly identifying dummy images are excluded from the study.
#### Results.
Figure 1 and 4 (top-left) report the average difference in human identified artifact rate between watermarked and non-watermarked samples. A 5 percent point increase indicates that the corresponding watermarking method creates newly visible artifacts in at least 5% of the images (it is a lower bound). SynthID-O performs best, followed by VideoSeal-1.0, InvisMark and WAM. StegaStamp has the largest reported artifacts rate, validating our observation that its high robustness relative to existing watermarking models comes at the cost of a large degradation in quality. We also report computational quality metrics on the same clean and watermarked image pairs in Figure 4. While computational metrics tend to correlate with human judgment, they fail to capture obvious artifacts (especially when those are small in size). This is clearly noticeable for VideoSeal-0.0 and TrustMark-Q which exhibit strong performance on computational metrics, but do poorly on the human evaluation. It is worth noting that in side-by-side evaluations where trained raters are asked to tell which one between two almost identical images is watermarked, all methods (except SynthID) do poorly.
<details>
<summary>x2.png Details</summary>
### Visual Description
## Bar Charts: Comparative Performance of Image/Video Watermarking Methods
### Overview
The image displays a composite of six bar charts arranged in a 2x3 grid. Each chart compares the performance of different watermarking or content-protection methods across various quantitative metrics. One method, "SynthID-Q," is consistently highlighted in blue across all charts for easy identification. The charts collectively assess trade-offs between robustness, imperceptibility, and human-perceived quality.
### Components/Axes
**General Layout:**
- Six individual bar charts in a grid.
- Each chart has a title indicating the metric being measured.
- The y-axis represents the metric value. An arrow (â or â) next to the title indicates whether a lower (â) or higher (â) value is desirable.
- The x-axis lists the names of the methods being compared. The order of methods varies slightly between charts.
- Error bars are present on all bars, indicating variability or confidence intervals.
- A single bar in each chart is colored blue; this bar corresponds to the method "SynthID-Q."
**Chart-Specific Details:**
1. **Top-Left: Human evaluation study**
* **Title:** `Human evaluation study (percent point increase in artifact rates) â`
* **Y-axis:** Scale from 0 to 40, increments of 5.
* **X-axis (Methods, left to right):** SynthID-Q, VideoSeal 1.0, InvMark, WAM, TrustMark-Ω, VideoSeal 1.5, TrustMark-Q, StegaStamp.
* **Legend/Highlight:** The first bar (SynthID-Q) is blue.
2. **Top-Center: FID**
* **Title:** `FID â`
* **Y-axis:** Scale from 0 to 35, increments of 5.
* **X-axis (Methods, left to right):** InvMark, SynthID-Q, TrustMark-Ω, VideoSeal 1.0, VideoSeal 1.5, TrustMark-Q, WAM, StegaStamp.
* **Legend/Highlight:** The second bar (SynthID-Q) is blue.
3. **Top-Right: CMMD**
* **Title:** `CMMD â`
* **Y-axis:** Scale from 0.00 to 0.40, increments of 0.05.
* **X-axis (Methods, left to right):** SynthID-Q, TrustMark-Q, VideoSeal 1.5, InvMark, TrustMark-Ω, VideoSeal 1.0, WAM, StegaStamp.
* **Legend/Highlight:** The first bar (SynthID-Q) is blue.
4. **Bottom-Left: LPIPS VGG**
* **Title:** `LPIPS VGG â`
* **Y-axis:** Scale from 0.00 to 0.30, increments of 0.05.
* **X-axis (Methods, left to right):** VideoSeal 1.0, InvMark, SynthID-Q, TrustMark-Ω, VideoSeal 1.5, TrustMark-Q, WAM, StegaStamp.
* **Legend/Highlight:** The third bar (SynthID-Q) is blue.
5. **Bottom-Center: PSNR**
* **Title:** `PSNR â`
* **Y-axis:** Scale from 0 to 50, increments of 10.
* **X-axis (Methods, left to right):** VideoSeal 1.0, InvMark, TrustMark-Ω, SynthID-Q, VideoSeal 1.5, TrustMark-Q, WAM, StegaStamp.
* **Legend/Highlight:** The fourth bar (SynthID-Q) is blue.
6. **Bottom-Right: SSIM**
* **Title:** `SSIM â`
* **Y-axis:** Scale from 0.0 to 1.0, increments of 0.2.
* **X-axis (Methods, left to right):** VideoSeal 1.0, InvMark, VideoSeal 1.5, TrustMark-Ω, SynthID-Q, WAM, StegaStamp.
* **Legend/Highlight:** The fifth bar (SynthID-Q) is blue.
### Detailed Analysis
**1. Human Evaluation (Artifact Rates - Lower is Better):**
* **Trend:** A clear upward trend from left to right. SynthID-Q has the lowest increase in artifact rates, while StegaStamp has the highest.
* **Approximate Values (Percent Point Increase):**
* SynthID-Q (Blue): ~1.5
* VideoSeal 1.0: ~5
* InvMark: ~7
* WAM: ~9
* TrustMark-Ω: ~10
* VideoSeal 1.5: ~27
* TrustMark-Q: ~27
* StegaStamp: ~38
**2. FID (Fréchet Inception Distance - Lower is Better):**
* **Trend:** Most methods have low FID scores (<5). There is a sharp increase for the last two methods, with StegaStamp being a significant outlier.
* **Approximate Values:**
* InvMark: ~0.5
* SynthID-Q (Blue): ~1.0
* TrustMark-Ω: ~2.0
* VideoSeal 1.0: ~2.5
* VideoSeal 1.5: ~3.0
* TrustMark-Q: ~7.0
* WAM: ~7.5
* StegaStamp: ~37.0
**3. CMMD (Lower is Better):**
* **Trend:** Similar to FID, most methods score very low (<0.05). StegaStamp is again a major outlier with a much higher value.
* **Approximate Values:**
* SynthID-Q (Blue): ~0.005
* TrustMark-Q: ~0.01
* VideoSeal 1.5: ~0.015
* InvMark: ~0.02
* TrustMark-Ω: ~0.02
* VideoSeal 1.0: ~0.025
* WAM: ~0.04
* StegaStamp: ~0.40
**4. LPIPS VGG (Learned Perceptual Image Patch Similarity - Lower is Better):**
* **Trend:** Very low values for the first six methods, with a slight increase for WAM and a larger jump for StegaStamp.
* **Approximate Values:**
* VideoSeal 1.0: ~0.002
* InvMark: ~0.003
* SynthID-Q (Blue): ~0.005
* TrustMark-Ω: ~0.007
* VideoSeal 1.5: ~0.008
* TrustMark-Q: ~0.01
* WAM: ~0.025
* StegaStamp: ~0.12
**5. PSNR (Peak Signal-to-Noise Ratio - Higher is Better):**
* **Trend:** A gradual downward trend from left to right. VideoSeal 1.0 has the highest PSNR, while StegaStamp has the lowest.
* **Approximate Values (dB):**
* VideoSeal 1.0: ~53
* InvMark: ~51
* TrustMark-Ω: ~49
* SynthID-Q (Blue): ~48
* VideoSeal 1.5: ~47
* TrustMark-Q: ~45
* WAM: ~43
* StegaStamp: ~30
**6. SSIM (Structural Similarity Index - Higher is Better):**
* **Trend:** All methods achieve very high SSIM scores (close to 1.0), indicating high structural similarity to the original. There is a very slight downward trend, with StegaStamp showing the most noticeable drop.
* **Approximate Values:**
* VideoSeal 1.0: ~0.995
* InvMark: ~0.995
* VideoSeal 1.5: ~0.995
* TrustMark-Ω: ~0.995
* SynthID-Q (Blue): ~0.995
* WAM: ~0.99
* StegaStamp: ~0.97
### Key Observations
1. **Consistent Outlier:** The method "StegaStamp" is a consistent outlier, performing worst (highest value) on all "lower-is-better" metrics (FID, CMMD, LPIPS, Human Artifacts) and worst (lowest value) on the "higher-is-better" PSNR metric. It also shows the largest drop in SSIM.
2. **SynthID-Q Performance:** The highlighted method, SynthID-Q, generally performs very well. It has the lowest human-perceived artifact rate, competitive (low) scores on FID, CMMD, and LPIPS, and strong (high) scores on PSNR and SSIM.
3. **Metric Clustering:** Methods like VideoSeal (1.0 & 1.5), InvMark, and TrustMark variants often cluster together in performance, forming a middle tier between the top-performing SynthID-Q and the poorly-performing StegaStamp/WAM on several metrics.
4. **Human vs. Computational Metrics:** There is a notable discrepancy between the human evaluation chart and the computational metrics (FID, CMMD, LPIPS). While SynthID-Q is best in human evaluation, it is not always the absolute best on computational metrics (e.g., VideoSeal 1.0 has a slightly better PSNR). This suggests computational metrics may not perfectly correlate with human perception of artifacts.
### Interpretation
This composite chart provides a multi-faceted evaluation of watermarking techniques, revealing critical trade-offs. The data suggests that **SynthID-Q achieves a superior balance between imperceptibility and robustness**. It introduces the fewest visible artifacts according to human evaluators while maintaining strong performance on objective image quality metrics (FID, PSNR, SSIM).
The dramatic underperformance of **StegaStamp** across all metrics indicates it may be a less sophisticated or older method, causing significant distortion. The clustering of other methods suggests incremental improvements among them.
The key investigative insight is the **divergence between human evaluation and some computational metrics**. A method can score well on PSNR or SSIM (which measure pixel-level or structural fidelity) but still introduce artifacts noticeable to humans (as seen with VideoSeal 1.5 having a high artifact rate despite decent PSNR/SSIM). This underscores the importance of including human-in-the-loop evaluation for tasks where perceptual quality is paramount. The charts collectively argue that the optimal watermarking method is not necessarily the one that maximizes a single computational score, but the one that best minimizes human-perceived degradation while maintaining sufficient signal strength for detection.
</details>
Figure 4: Quality metrics for all methods computed on 1,000 watermarked and unwatermarked images. The top-left panel shows the average difference in identified artifact rate between watermarked and non-watermarked samples (lower is better). A 5 percent point increase indicates that the watermarking method creates visible artifacts in 5% of the images. For FID, CMMD and LPIPS the lower is better and for PSNR and SSIM the higher is better. Overall, we see little correlation between computational metrics and human ratings.
### 8.2 Robustness evaluation
#### Setup.
We evaluate all models on a consistent set of 10,000 images from ImageNet âs validation set. We focus this evaluation on photo-realistic content which is the most likely source of misinformation. The robustness of detection and payload recovery is assessed under 30 common image transformations (shown on Figure 3). In this section, we evaluate images at a resolution of 512 $Ă$ 512. Evaluations at native image resolutions are shown in Appendix A.1. Evaluations where images are transformed after being resized at each methodâs preferred resolution are shown in Appendix A.2.
We group image transformations into six categories: Color, Combination, Noise, Overlay, Quality and Spatial. Aggregated includes all transformations across categories, and Identity applies no transformation. Additionally, we use two evaluation settings: Random:
The strength of each transformation is sampled uniformly from within a specified range. Worst:
The strength of each transformation is set to its most severe value, typically an extreme within the range described above.
While the random setting simulates real-world variability, worst setting allows us to discern the limits of model performance under extreme image manipulation. Unless stated otherwise, the models are calibrated to obtain a 0.1% false positive rate in average on worst -case transformations.
#### Detection results.
In this section, we focus exclusively on the detection performance under various image transformations. For models that offer a detection score independent of payload recovery (i.e., SynthID-O and WAM), we use this score. For models that solely perform payload recovery, we compute the detection score by counting the number of correctly decoded bits (as done by Fernandez et al., 2023). This implies that, for all methods except SynthID-O and WAM, we assume that there is a single, unique known payload we can match. We do not observe any significant difference when using a weighted sum over payload logits. We also include the performance of SynthID-O and WAM using payload matching in a separate table.
Table 1 summarizes the results. It reports the true positive rate (TPR) at a fixed false positive rate of 0.1% in average on worst -case transformations. We observe that over aggregated random and aggregated worst transformations, SynthID-O outperforms all other methods by significant margins (i.e., $+9.36$ and $+16.35$ percentage points respectively). In the worst setting, SynthID-O is the only models surpassing 99% TPR in aggregate and it reaches more than 98% TPR on combinations of transformations (which is the most challenging setting). Overall, it is worth noting that not a single model outperforms all others across all categories. Comparisons are made more difficult when we start considering quality: StegaStamp is the worst offender in terms of quality and, as a result, it is regularly topping individual categories in terms of robustness.
| Identity (excl. resizing) Aggregated Color | 100.00% 99.98% 100.00% | 100.00% [70.06%, 70.41%] [75.34%, 75.49%] | 100.00% [72.41%, 72.57%] 99.95% | [99.95%, 99.96%] [66.25%, 66.74%] [79.22%, 79.82%] | 100.00% [78.61%, 79.02%] [99.65%, 99.69%] | 100.00% [78.35%, 78.91%] [75.97%, 76.22%] | 99.98% [88.75%, 88.87%] [99.86%, 99.87%] | 100.00% 90.62% 81.29% |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Combination | 99.96% | [22.28%, 23.19%] | [49.62%, 49.75%] | [23.06%, 24.46%] | [64.00%, 66.25%] | [77.67%, 79.65%] | [84.59%, 84.91%] | 96.08% |
| Noise | 99.98% | [92.10%, 92.73%] | 100.00% | [95.10%, 95.49%] | 100.00% | [96.93%, 97.37%] | [99.11%, 99.18%] | 100.00% |
| Overlay | 100.00% | 100.00% | 100.00% | [99.71%, 99.73%] | 99.99% | [99.93%, 99.95%] | 99.99% | 100.00% |
| Quality | 99.99% | [89.51%, 89.78%] | 100.00% | [97.56%, 97.72%] | 100.00% | [98.60%, 98.76%] | 99.99% | 99.97% |
| Spatial | 99.98% | [61.09%, 61.32%] | [25.23%, 25.73%] | [38.05%, 38.32%] | [40.75%, 41.04%] | [55.95%, 56.46%] | [67.47%, 67.72%] | 84.34% |
| Aggregated Worst | 99.72% | [60.11%, 60.53%] | [66.51%, 66.79%] | [54.15%, 54.68%] | [68.73%, 68.87%] | [61.22%, 61.91%] | [76.74%, 76.89%] | 83.37% |
| Color Worst | 100.00% | [69.58%, 69.67%] | [99.79%, 99.82%] | [66.32%, 67.16%] | [98.93%, 99.05%] | [66.19%, 66.49%] | [99.69%, 99.71%] | 84.00% |
| Combination Worst | 98.06% | [6.83%, 7.89%] | [27.07%, 27.83%] | [4.16%, 4.76%] | [21.22%, 21.72%] | [27.71%, 29.05%] | [31.00%, 31.38%] | 55.96% |
| Noise Worst | 99.96% | [80.86%, 82.05%] | 100.00% | [88.13%, 89.02%] | 99.99% | [90.77%, 91.93%] | [97.73%, 97.89%] | 100.00% |
| Overlay Worst | 100.00% | 100.00% | 100.00% | [98.76%, 98.85%] | [99.87%, 99.89%] | [99.79%, 99.83%] | 99.97% | 100.00% |
| Quality Worst | 99.99% | [76.24%, 76.73%] | 100.00% | [84.65%, 85.36%] | 99.99% | [81.84%, 82.97%] | 99.98% | 99.38% |
| Spatial Worst | 99.97% | [50.06%, 50.13%] | [15.25%, 15.87%] | [25.09%, 25.17%] | [25.76%, 25.92%] | [38.89%, 39.32%] | [51.62%, 51.86%] | 76.04% |
Table 1: TPR at 0.1% FPR aggregated across each transformation category when images are resized to 512 $Ă$ 512. For each model, the detection threshold is calibrated to reach 0.1% FPR across all worst transformations in average. When brackets are used, we display the range of TPR values that can accommodate the target FPR (this number is not unique when there are ties).
For completeness, we also report the detection performance of SynthID-O and WAM if they were to use their payload for detection in Table 2.
| Identity (excl. resizing) Aggregated Color | 100.00% [99.96%, 99.97%] [99.99%, 100.00%] | [99.96%, 99.98%] [81.26%, 81.85%] [77.16%, 77.31%] |
| --- | --- | --- |
| Combination | [99.92%, 99.93%] | [89.68%, 92.56%] |
| Noise | [99.89%, 99.90%] | [99.95%, 99.98%] |
| Overlay | 100.00% | [99.96%, 99.98%] |
| Quality | [99.96%, 99.97%] | [99.70%, 99.79%] |
| Spatial | 99.98% | [57.39%, 57.91%] |
| Aggregated Worst | [98.84%, 98.90%] | [69.11%, 69.91%] |
| Color Worst | 99.99% | [69.78%, 69.82%] |
| Combination Worst | [91.96%, 92.36%] | [36.25%, 39.52%] |
| Noise Worst | [99.74%, 99.77%] | [99.95%, 99.98%] |
| Overlay Worst | 100.00% | [99.97%, 99.98%] |
| Quality Worst | 99.97% | [95.21%, 96.29%] |
| Spatial Worst | 99.97% | [48.76%, 49.45%] |
Table 2: TPR at 0.1% FPR aggregated across each transform category when images are resized to 512 $Ă$ 512. The detection is performed by computing the number of correct bits in the payload. For each model, the detection threshold is calibrated to reach 0.1% FPR across all worst transformations. Models that are not listed here are already listed in Table 1. When brackets are used, we display the range of TPR values that can accommodate the target FPR (this number is not unique when there are ties).
#### Payload recovery results.
We report the bit accuracy measured on watermarked images (as detailed in Section 5). Table 3 compares SynthID-O to other models. We observe that SynthID-O often surpasses other models despite having a larger payload (except VideoSeal-1.0) and better quality. WAM which tops a few categories only has a 32-bit payload.
| Identity (excl. resizing) Aggregated Color | 100.00% 99.59% 99.97% | 95.04% 79.28% 82.47% | 99.84% 85.03% 99.14% | 99.23% 79.46% 84.53% | 99.91% 87.27% 98.16% | 96.94% 82.67% 83.24% | 99.61% 89.60% 97.60% | 99.96% 90.62% 88.42% |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Combination | 99.12% | 56.57% | 70.45% | 58.64% | 74.10% | 74.75% | 79.31% | 93.04% |
| Noise | 98.57% | 85.21% | 99.61% | 89.26% | 98.85% | 87.98% | 90.13% | 99.94% |
| Overlay | 100.00% | 94.62% | 99.66% | 97.55% | 98.93% | 96.24% | 99.27% | 99.96% |
| Quality | 99.73% | 88.56% | 99.75% | 95.30% | 99.62% | 93.13% | 98.75% | 99.65% |
| Spatial | 99.82% | 76.40% | 61.68% | 68.09% | 69.43% | 74.85% | 80.47% | 79.83% |
| Aggregated Worst | 98.07% | 75.26% | 82.58% | 73.29% | 82.47% | 76.03% | 83.53% | 85.03% |
| Color Worst | 99.93% | 78.31% | 98.19% | 76.86% | 96.03% | 78.16% | 95.10% | 84.29% |
| Combination Worst | 89.46% | 52.60% | 62.62% | 51.69% | 58.16% | 59.06% | 60.49% | 71.86% |
| Noise Worst | 97.63% | 80.61% | 99.44% | 84.64% | 98.07% | 84.34% | 86.10% | 99.91% |
| Overlay Worst | 99.99% | 93.91% | 99.04% | 93.69% | 94.80% | 95.55% | 98.52% | 99.96% |
| Quality Worst | 99.49% | 84.17% | 99.57% | 89.02% | 99.05% | 87.00% | 97.34% | 97.30% |
| Spatial Worst | 99.77% | 72.13% | 57.87% | 62.33% | 63.04% | 68.14% | 73.00% | 74.97% |
Table 3: Payload bit accuracy aggregated across each transformation category when images are resized to 512 $Ă$ 512. Note that numbers are not directly comparable as the payload size is different for different methods.
## 9 Related work
While this paper is not meant to be a comprehensive survey on watermarking â we refer to Wan et al. (2022) for relatively a recent overview â we want to provide a short overview of watermarking in general and recent developments in deploying watermarking alongside generative AI. Besides presenting results for SynthID-Image, this paper also touches on robustness, threat models and practical concerns. In this spirit, our paper is similar to this recent State-of-Knowledge on watermarking (Zhao et al., 2025).
To the best of our knowledge, Van Schyndel et al. (1994); Braudaway (1997) presented the first invisible watermarks. As visible watermarks were not yet automatically removable as easily as today (Dekel et al., 2017), early work in invisible watermarking was closely entangled with steganography (Cox, 2007). Irrespective of the use, early methods either used the spatial or frequency domain to hide information. For example, Wang et al. (2001) hide information in less important bits across pixels. This is not very robust to common image transformations (Fazli and Khodaverdi, 2009). Instead, a long line of work uses DCT (Ahmed et al., 1974) or DWT (Shensa, 1992) transforms. Well-received examples include Barni et al. (1998); Chen and Wornell (1998, 2001); Parah et al. (2016) and Barni et al. (2001); Kashyap and Sinha (2012), respectively. Approaches working in the spatial and frequency domains have also been combined, e.g., using DCT-SVD approaches (Singh and Singh, 2017; Su and Chen, 2018). Recently, DCT and DWT based approaches have been re-popularized due to their usage by Stability AI https://github.com/Stability-AI/invisible-watermark-gpu despite their flaws compared to recent deep learning based methods.
While there are early experiments using neural networks for watermark detection in (Yu et al., 2001), proper deep learning based approaches gained traction around 2017 (Mun et al., 2017; Zhu et al., 2018). Usually, encoder-decoder architectures (i.e., content-based, post-hoc methods) tried to improve on quality and robustness (Wen and Aydöre, 2019; Hayes and Danezis, 2017; Hayes et al., 2020; Tancik et al., 2020; Luo et al., 2020; Bui et al., 2025, 2023; Pan et al., 2023, 2024; Xu et al., 2025). Recently, these approaches have been extended to allow localization of (multiple) watermarks (Sander et al., 2025). Only more recently, with the rise of diffusion models (Ho et al., 2020; Sohl-Dickstein et al., 2015) such as stable diffusion (Rombach et al., 2022), watermarking has been integrated into the generation process; see Fernandez et al. (2023); Wen et al. (2023); Yang et al. (2024). The pros and cons have been discussed at length in Section 2. These works usually exclusively focus on image watermarking; literature on video (Zhang et al., 2020a; Fernandez et al., 2024) or audio watermarking (Hua et al., 2016; Roman et al., 2024) tends to be sparse, possibly because methods are usually general enough across these modalities.
The importance of appropriate benchmarks for watermarking has been highlighted as early as 2000 (Petitcolas, 2000) with a thorough account of possible attacks some years later by Cox (2007). More recently, An et al. (2024) introduced an up-to-date and relatively comprehensive benchmark of deep learning based watermarking techniques. Proper benchmarking also includes considering a variety of (adaptive) attacks (Saberi et al., 2024a; Lukas et al., 2024; Zhao et al., 2024), many of which have been discussed at length in Section 6. Meta also maintains a benchmark for image watermarking, called Omni Seal Bench. https://github.com/facebookresearch/omnisealbench
In terms of deployed systems at the time of writing, the solution used by Stability AI https://github.com/ShieldMnt/invisible-watermark has received considerable attention, but usage is difficult to track. Imatag released several models on Hugging Face, including a StableSignature (Fernandez et al., 2023) based one. https://huggingface.co/imatag/stable-signature-bzh-sdxl-vae-weak However, the corresponding proprietary service seems to focus on content tracking and information leak identification. https://www.imatag.com/ For DALL-E 2 https://openai.com/index/dall-e-2/, OpenAI experimented with a visible watermark referred to as âcoloured squaresâ by users. While there was no official communication on this, and OpenAIâs FAQs state that users are allowed to remove it, https://web.archive.org/web/*/https://help.openai.com/en/articles/6468065-dall-e-2-faq this can be understood as an early approach to watermarking. More recently, for DALL-E 3, https://openai.com/index/dall-e-3/ OpenAI announced both C2PA metadata as well as proper invisible watermarking (for image and audio). To date, however, this has not been released. Instead, OpenAI showcased early results for AI-generated image detection, https://openai.com/index/understanding-the-source-of-what-we-see-and-hear-online/ which seems rather fragile under certain transformations. Meta recently announced to follow C2PA specifications and reveal this information to users on Facebook, Instragram and Threads. https://about.fb.com/news/2024/02/labeling-ai-generated-images-on-facebook-instagram-and-threads/ Meta seems to pursue invisible watermarking across modalities, as well, with some open-source models (Sander et al., 2025; Roman et al., 2024; Fernandez et al., 2024) but no concrete public-facing verification system yet. Similarly, Microsoft published and open-sourced an image watermarking model (Xu et al., 2025). https://github.com/microsoft/InvisMark Overall, this makes SynthID-Image the first system deployed at large-scale that consistently watermarks large amounts of generated content and discloses this information to end users.
## 10 Limitations and future work
Despite its success, SynthID-Image alone will not solve many of the problems we set out to alleviate, including misinformation, impersonation or copyright tracking. This is because watermarking in itself does not solve the provenance problem. Instead, SynthID-Image needs to become part of an ecosystem of tools, including C2PA, that are adopted by all major players across industry and governments to watermark AI-generated content and surface this information to the public. This requires both technological advancements as well as work on policy and cooperation across companies and governments. Besides, there are various technical challenges that we believe will become important with advancements in generative AI. For example, SynthID operates on text, images, audio or video. Thus, work on detection and handling fractional watermarks becomes important. Similarly, we likely need larger payloads as use cases of generative AI grow and we want to enable public detectability using cryptographic signatures (Fairoze et al., 2025). For open models, we need to work on improving security, particularly considering white-box threat models and we generally need to figure out proper versioning of watermarks, assuming that multiple watermarking solutions will be available soon.
## Acknowledgments
We would like to thank Demis Hassabis for starting, sponsoring and advising throughout this project. We thank Clement Wolf, Ian Goodfellow, Chris Bregler and Oriol Vinyals for their advice. We also thank many others who contributed across Google DeepMind and Google, including our partners at Google Research, Google Cloud, Google Global Business & Operations and YouTube.
## References
- Adi et al. (2018) Y. Adi, C. Baum, M. CissĂ©, B. Pinkas, and J. Keshet. Turning your weakness into a strength: Watermarking deep neural networks by backdooring. In USENIX Security Symposium, pages 1615â1631, 2018.
- Agarwal et al. (2019) S. Agarwal, H. Farid, Y. Gu, M. He, K. Nagano, and H. Li. Protecting world leaders against deep fakes. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
- Ahmed et al. (1974) N. Ahmed, T. R. Natarajan, and K. R. Rao. Discrete cosine transform. IEEE Transactions on Computers, 23(1):90â93, 1974.
- Alaifari et al. (2019) R. Alaifari, G. S. Alberti, and T. Gauksson. ADef: An iterative algorithm to construct adversarial deformations. In Proc. of the International Conference on Learning Representations (ICLR), 2019.
- An et al. (2024) B. An, M. Ding, T. Rabbani, A. Agrawal, Y. Xu, C. Deng, S. Zhu, A. Mohamed, Y. Wen, T. Goldstein, and F. Huang. WAVES: Benchmarking the robustness of image watermarks. In Proc. of the International Conference on Machine Learning (ICML), 2024.
- Anderson (1996) R. J. Anderson, editor. Information Hiding, First International Workshop, Cambridge, UK, May 30 - June 1, 1996, Proceedings, volume 1174, 1996.
- Balan et al. (2023) K. Balan, S. Agarwal, S. Jenni, A. Parsons, A. Gilbert, and J. P. Collomosse. EKILA: Synthetic media provenance and attribution for generative art. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2023.
- Barni et al. (1998) M. Barni, F. Bartolini, V. Cappellini, and A. Piva. A DCT-domain system for robust image watermarking. Signal Processing, 66(3):357â372, 1998.
- Barni et al. (2001) M. Barni, F. Bartolini, and A. Piva. Improved wavelet-based watermarking through pixel-wise masking. IEEE Trans. on Image Processing (TIP), 10(5):783â791, 2001.
- Bates et al. (2021) S. Bates, E. Candes, L. Lei, Y. Romano, and M. Sesia. Calibrated out-of-distribution detection with conformal p-values. In Proc. of the International Conference on Machine Learning (ICML) Workshops, 2021.
- Bates et al. (2023) S. Bates, J. CandĂšs, L. Lei, Y. Romano, and M. Sesia. Testing for outliers with conformal p-values. Ann. of Statistics, 51(1):149â178, 2023.
- Bharati et al. (2021) A. Bharati, D. Moreira, P. J. Flynn, A. de Rezende Rocha, K. W. Bowyer, and W. J. Scheirer. Transformation-aware embeddings for image provenance. Proc. of the IEEE Transactions on Information Forensics and Security, 16, 2021.
- Biggio et al. (2013) B. Biggio, I. Corona, D. Maiorca, B. Nelson, N. Srndic, P. Laskov, G. Giacinto, and F. Roli. Evasion attacks against machine learning at test time. In Proc. of the European Conference on Machine Learning and Knowledge Discovery in Databases (ECML PKDD), 2013.
- Braudaway (1997) G. W. Braudaway. Protecting publicly-available images with an invisible image watermark. In Proc. of the IEEE International Conference on Image Processing (ICIP), 1997.
- Brown et al. (2017) T. B. Brown, D. Mané, A. Roy, M. Abadi, and J. Gilmer. Adversarial patch. arXiv.org, abs/1712.09665, 2017.
- Bryniarski et al. (2022) O. Bryniarski, N. Hingun, P. Pachuca, V. Wang, and N. Carlini. Evading adversarial example detection defenses with orthogonal projected gradient descent. In Proc. of the International Conference on Learning Representations (ICLR), 2022.
- Bui et al. (2023) T. Bui, S. Agarwal, N. Yu, and J. P. Collomosse. RoSteALS: Robust steganography using autoencoder latent space. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
- Bui et al. (2025) T. Bui, S. Agarwal, and J. P. Collomosse. TrustMark: Universal watermarking for arbitrary resolution images. In Proc. of the IEEE International Conference on Computer Vision (ICCV), 2025.
- Carlini and Farid (2020) N. Carlini and H. Farid. Evading deepfake-image detectors with white- and black-box attacks. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
- Carlini and Wagner (2017a) N. Carlini and D. Wagner. Adversarial examples are not easily detected: Bypassing ten detection methods. In Proc. of the ACM Workshop on Artificial Intelligence and Security, 2017a.
- Carlini and Wagner (2017b) N. Carlini and D. Wagner. Towards evaluating the robustness of neural networks. In Proc. of the IEEE Symposium on Security and Privacy, 2017b.
- Chen and Wornell (1998) B. Chen and G. W. Wornell. Digital watermarking and information embedding using dither modulation. In IEEE Workshop on Multimedia Signal Processing (MMSP), 1998.
- Chen and Wornell (2001) B. Chen and G. W. Wornell. Quantization index modulation: A class of provably good methods for digital watermarking and information embedding. IEEE Transactions on Information Theory, 47(4):1423â1443, 2001.
- Christ et al. (2024) M. Christ, S. Gunn, and O. Zamir. Undetectable watermarks for language models. In Proc. of the Conference on Learning Theory (COLT), 2024.
- Coalition for Content Provenance and Authenticity (2023) Coalition for Content Provenance and Authenticity. C2PA technical specifications version 1.3. https://c2pa.org/specifications/specifications/1.3/specs/C2PA_Specification.html, 2023. Online; accessed Mar 15 2024.
- Collomosse and Parsons (2024) J. Collomosse and A. Parsons. To authenticity, and beyond! Building safe and fair generative AI upon the three pillars of provenance. IEEE Computer Graphics and Applications, 44(3):82â90, 2024.
- Cox (2007) I. Cox. Digital watermarking and steganography. The Morgan Kaufmann Series in Multimedia Information and Systems, 2:893â914, 2007.
- Croce and Hein (2020) F. Croce and M. Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In Proc. of the International Conference on Machine Learning (ICML), 2020.
- Dathathri et al. (2024) S. Dathathri, A. See, S. Ghaisas, P.-S. Huang, R. McAdam, J. Welbl, V. Bachani, A. Kaskasoli, R. Stanforth, T. Matejovicova, et al. Scalable watermarking for identifying large language model outputs. Nature, 634(8035):818â823, 2024.
- Dekel et al. (2017) T. Dekel, M. Rubinstein, C. Liu, and W. T. Freeman. On the effectiveness of visible watermarks. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
- Dumont et al. (2018) B. Dumont, S. Maggio, and P. Montalvo. Robustness of rotation-equivariant networks to adversarial perturbations. arXiv.org, abs/1802.06627, 2018.
- Engstrom et al. (2017) L. Engstrom, D. Tsipras, L. Schmidt, and A. Madry. A rotation and a translation suffice: Fooling CNNs with simple transformations. arXiv.org, abs/1712.02779, 2017.
- Fairoze et al. (2025) J. Fairoze, G. Ortiz-Jimenez, M. Vecerik, S. Jha, and S. Gowal. On the difficulty of constructing a robust and publicly-detectable watermark. In Conference on Artificial Intelligence and Statistics (AISTATS), 2025.
- Fazli and Khodaverdi (2009) S. Fazli and G. Khodaverdi. Trade-off between imperceptibility and robustness of LSB watermarking using SSIM quality metrics. In Proc. of the International Conference on Machine Vision, 2009.
- Fernandez et al. (2023) P. Fernandez, G. Couairon, H. JĂ©gou, M. Douze, and T. Furon. The stable signature: Rooting watermarks in latent diffusion models. In Proc. of the IEEE International Conference on Computer Vision (ICCV), 2023.
- Fernandez et al. (2024) P. Fernandez, H. Elsahar, I. Z. Yalniz, and A. Mourachko. Video Seal: Open and efficient video watermarking. arXiv preprint arXiv:2412.09492, 2024.
- Gowal et al. (2021) S. Gowal, P.-S. Huang, A. van den Oord, T. Mann, and P. Kohli. Self-supervised adversarial robustness for the low-label, high-data regime. In Proc. of the International Conference on Learning Representations (ICLR), 2021.
- Gunn et al. (2025) S. Gunn, X. Zhao, and D. Song. An undetectable watermark for generative image models. In Proc. of the International Conference on Learning Representations (ICLR), 2025.
- Hayes (2020) J. Hayes. Provable trade-offs between private & robust machine learning. arXiv.org, abs/2006.04622, 2020.
- Hayes and Danezis (2017) J. Hayes and G. Danezis. Generating steganographic images via adversarial training. In Advances in Neural Information Processing Systems (NeurIPS), 2017.
- Hayes et al. (2020) J. Hayes, K. Dvijotham, Y. Chen, S. Dieleman, P. Kohli, and N. Casagrande. Towards transformation-resilient provenance detection of digital media. arXiv.org, abs/2011.07355, 2020.
- Hendrycks and Dietterich (2019) D. Hendrycks and T. G. Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. In Proc. of the International Conference on Learning Representations (ICLR), 2019.
- Hendrycks et al. (2019) D. Hendrycks, M. Mazeika, and T. G. Dietterich. Deep anomaly detection with outlier exposure. In Proc. of the International Conference on Learning Representations (ICLR), 2019.
- Heusel et al. (2017) M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter. GANs trained by a two time-scale update rule converge to a local Nash equilibrium. In Advances in Neural Information Processing Systems (NeurIPS), 2017.
- Ho et al. (2020) J. Ho, A. Jain, and P. Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
- Hochberg (1988) Y. Hochberg. A sharper Bonferroni procedure for multiple tests of significance. Biometrika, 75(4):800â802, 1988.
- Holm (1979) S. Holm. A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, pages 65â70, 1979.
- Hua et al. (2016) G. Hua, J. Huang, Y. Q. Shi, J. Goh, and V. L. L. Thing. Twenty years of digital audio watermarking - a comprehensive review. Signal Processing, 128:222â242, 2016.
- Jayasumana et al. (2024) S. Jayasumana, S. Ramalingam, A. Veit, D. Glasner, A. Chakrabarti, and S. Kumar. Rethinking FID: Towards a better evaluation metric for image generation. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
- Jovanovic et al. (2024) N. Jovanovic, R. Staab, and M. T. Vechev. Watermark stealing in large language models. In Proc. of the International Conference on Machine Learning (ICML), 2024.
- Kadian et al. (2021) P. Kadian, S. M. Arora, and N. Arora. Robust digital watermarking techniques for copyright protection of digital data: A survey. Wireless Personal Communications, 118(4), 2021.
- Kashyap and Sinha (2012) N. Kashyap and G. Sinha. Image watermarking using 3-level discrete wavelet transform (DWT). International Journal of Modern Education and Computer Science, 4(3):50, 2012.
- Kirchenbauer et al. (2023) J. Kirchenbauer, J. Geiping, Y. Wen, J. Katz, I. Miers, and T. Goldstein. A watermark for large language models. In Proc. of the International Conference on Machine Learning (ICML), 2023.
- Kurakin et al. (2017) A. Kurakin, I. J. Goodfellow, and S. Bengio. Adversarial examples in the physical world. In Proc. of the International Conference on Learning Representations (ICLR), 2017.
- Lukas and Kerschbaum (2023) N. Lukas and F. Kerschbaum. PTW: Pivotal tuning watermarking for pre-trained image generators. In USENIX Security Symposium, 2023.
- Lukas et al. (2024) N. Lukas, A. Diaa, L. Fenaux, and F. Kerschbaum. Leveraging optimization for adaptive attacks on image watermarks. In Proc. of the International Conference on Learning Representations (ICLR), 2024.
- Luo et al. (2020) X. Luo, R. Zhan, H. Chang, F. Yang, and P. Milanfar. Distortion agnostic deep watermarking. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
- Madry et al. (2018) A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu. Towards deep learning models resistant to adversarial attacks. Proc. of the International Conference on Learning Representations (ICLR), 2018.
- Maini et al. (2020) P. Maini, E. Wong, and J. Z. Kolter. Adversarial robustness against the union of multiple perturbation models. In Proc. of the International Conference on Machine Learning (ICML), 2020.
- Mun et al. (2017) S. Mun, S. Nam, H. Jang, D. Kim, and H. Lee. A robust blind watermarking using convolutional neural network. arXiv.org, abs/1704.03248, 2017.
- Nguyen et al. (2021) E. Nguyen, T. Bui, V. V. Swaminathan, and J. P. Collomosse. OSCAR-Net: Object-centric scene graph attention for image attribution. In Proc. of the IEEE International Conference on Computer Vision (ICCV), 2021.
- Oh et al. (2019) S. J. Oh, B. Schiele, and M. Fritz. Towards Reverse-Engineering Black-Box Neural Networks, pages 121â144. 2019. In book: Explainable AI: Interpreting, Explaining and Visualizing Deep Learning.
- Orekondy et al. (2019) T. Orekondy, B. Schiele, and M. Fritz. Knockoff nets: Stealing functionality of black-box models. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
- Orekondy et al. (2020) T. Orekondy, B. Schiele, and M. Fritz. Prediction poisoning: Towards defenses against DNN model stealing attacks. In Proc. of the International Conference on Learning Representations (ICLR), 2020.
- Pan et al. (2023) M. Pan, Y. Zeng, X. Lin, N. Yu, C.-J. Hsieh, and R. Jia. AnchMark: Anchor-contrastive watermarking vs GenAI-based image modifications. In Advances in Neural Information Processing Systems (NeurIPS) Workshops, 2023.
- Pan et al. (2024) M. Pan, Y. Zeng, X. Lin, N. Yu, C.-J. Hsieh, P. Henderson, and R. Jia. JIGMARK: A black-box approach for enhancing image watermarks against diffusion model edits. arXiv.org, abs/406.03720, 2024.
- Papernot et al. (2017) N. Papernot, P. McDaniel, I. Goodfellow, S. Jha, Z. B. Celik, and A. Swami. Practical black-box attacks against machine learning. In Proc. of the ACM on Asia Conference on Computer and Communications Security (AsiaCCS). ACM, 2017.
- Papernot et al. (2018) N. Papernot, P. D. McDaniel, A. Sinha, and M. P. Wellman. SoK: Security and privacy in machine learning. In Proc. of the IEEE European Symposium on Security and Privacy (EuroS&P), 2018.
- Parah et al. (2016) S. A. Parah, J. A. Sheikh, N. A. Loan, and G. M. Bhat. Robust and blind watermarking technique in DCT domain using inter-block coefficient differencing. Digital Signal Processing, 53:11â24, 2016.
- Petitcolas (2000) F. A. P. Petitcolas. Watermarking schemes evaluation. Signal Processing Magazine, 17(5), 2000.
- Piet et al. (2025) J. Piet, C. Sitawarin, V. Fang, N. Mu, and D. A. Wagner. MarkMyWords: Analyzing and evaluating language model watermarks. In IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), 2025.
- Radford et al. (2021) A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. Learning transferable visual models from natural language supervision. In Proc. of the International Conference on Machine Learning (ICML), 2021.
- Roman et al. (2024) R. S. Roman, P. Fernandez, A. DĂ©fossez, T. Furon, T. Tran, and H. Elsahar. Proactive detection of voice cloning with localized watermarking. In Proc. of the International Conference on Machine Learning (ICML), 2024.
- Rombach et al. (2022) R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer. High-resolution image synthesis with latent diffusion models. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
- Saberi et al. (2024a) M. Saberi, V. S. Sadasivan, K. Rezaei, A. Kumar, A. M. Chegini, W. Wang, and S. Feizi. Robustness of AI-image detectors: Fundamental limits and practical attacks. In Proc. of the International Conference on Learning Representations (ICLR), 2024a.
- Saberi et al. (2024b) M. Saberi, V. S. Sadasivan, A. Zarei, H. Mahdavifar, and S. Feizi. DREW: Towards robust data provenance by leveraging error-controlled watermarking. arXiv.org, abs/2406.02836, 2024b.
- Sander et al. (2025) T. Sander, P. Fernandez, A. Durmus, T. Furon, and M. Douze. Watermark anything with localized messages. In Proc. of the International Conference on Learning Representations (ICLR), 2025.
- Scao et al. (2022) T. L. Scao, A. Fan, C. Akiki, E. Pavlick, S. Ilic, D. Hesslow, R. Castagné, A. S. Luccioni, F. Yvon, M. Gallé, J. Tow, A. M. Rush, S. Biderman, A. Webson, P. S. Ammanamanchi, T. Wang, B. Sagot, N. Muennighoff, A. V. del Moral, O. Ruwase, R. Bawden, S. Bekman, A. McMillan-Major, I. Beltagy, H. Nguyen, L. Saulnier, S. Tan, P. O. Suarez, V. Sanh, H. Laurençon, Y. Jernite, J. Launay, M. Mitchell, C. Raffel, A. Gokaslan, A. Simhi, A. Soroa, A. F. Aji, A. Alfassy, A. Rogers, A. K. Nitzav, C. Xu, C. Mou, C. Emezue, C. Klamm, C. Leong, D. van Strien, D. I. Adelani, and et al. BLOOM: A 176b-parameter open-access multilingual language model. arXiv.org, abs/2211.05100, 2022.
- Shensa (1992) M. J. Shensa. The discrete wavelet transform: Wedding the Ă trous and Mallat algorithms. IEEE Trans. on Signal Processing (TSP), 40(10):2464â2482, 1992.
- Simes (1986) R. J. Simes. An improved Bonferroni procedure for multiple tests of significance. Biometrika, 73(3):751â754, 1986.
- Singh and Singh (2017) D. Singh and S. K. Singh. DWT-SVD and DCT based robust and blind watermarking scheme for copyright protection. Multimedia Tools and Applications, 76(11):13001â13024, 2017.
- Sohl-Dickstein et al. (2015) J. Sohl-Dickstein, E. A. Weiss, N. Maheswaranathan, and S. Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In Proc. of the International Conference on Machine Learning (ICML), 2015.
- Stutz et al. (2020) D. Stutz, M. Hein, and B. Schiele. Confidence-calibrated adversarial training: Generalizing to unseen attacks. In Proc. of the International Conference on Machine Learning (ICML), 2020.
- Stutz et al. (2023) D. Stutz, A. G. Roy, T. Matejovicova, P. Strachan, A. T. Cemgil, and A. Doucet. Conformal prediction under ambiguous ground truth. Transactions on Machine Learning Research, 2023.
- Su and Chen (2018) Q. Su and B. Chen. Robust color image watermarking technique in the spatial domain. Soft Computing, 22(1):91â106, 2018.
- Szegedy et al. (2014) C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. J. Goodfellow, and R. Fergus. Intriguing properties of neural networks. In Proc. of the International Conference on Learning Representations (ICLR), 2014.
- Tancik et al. (2020) M. Tancik, B. Mildenhall, and R. Ng. StegaStamp: Invisible hyperlinks in physical photographs. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
- Touvron et al. (2023) H. Touvron, T. Lavril, G. Izacard, X. Martinet, M. Lachaux, T. Lacroix, B. RoziĂšre, N. Goyal, E. Hambro, F. Azhar, A. Rodriguez, A. Joulin, E. Grave, and G. Lample. LLaMA: Open and efficient foundation language models. arXiv.org, abs/2302.13971, 2023.
- TramĂšr and Boneh (2019) F. TramĂšr and D. Boneh. Adversarial training and robustness for multiple perturbations. In Advances in Neural Information Processing Systems (NeurIPS), 2019.
- TramĂšr et al. (2016) F. TramĂšr, F. Zhang, A. Juels, M. K. Reiter, and T. Ristenpart. Stealing machine learning models via prediction APIs. In USENIX Security Symposium, 2016.
- Vaccari and Chadwick (2020) C. Vaccari and A. Chadwick. Deepfakes and disinformation: Exploring the impact of synthetic political video on deception, uncertainty, and trust in news. Social Media + Society, 6(1), 2020.
- Van Schyndel et al. (1994) R. G. Van Schyndel, A. Z. Tirkel, and C. F. Osborne. A digital watermark. In Proc. of the IEEE International Conference on Image Processing (ICIP), 1994.
- Wan et al. (2022) W. Wan, J. Wang, Y. Zhang, J. Li, H. Yu, and J. Sun. A comprehensive survey on robust image watermarking. Neurocomputing, 488, 2022.
- Wang et al. (2001) R. Wang, C. Lin, and J. Lin. Image hiding by optimal LSB substitution and genetic algorithm. Pattern Recognition, 34(3):671â683, 2001.
- Wang et al. (2004) Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. Image quality assessment: From error visibility to structural similarity. IEEE Trans. on Image Processing (TIP), 13(4):600â612, 2004.
- Weidinger et al. (2021) L. Weidinger, J. Mellor, M. Rauh, C. Griffin, J. Uesato, P. Huang, M. Cheng, M. Glaese, B. Balle, A. Kasirzadeh, Z. Kenton, S. Brown, W. Hawkins, T. Stepleton, C. Biles, A. Birhane, J. Haas, L. Rimell, L. A. Hendricks, W. Isaac, S. Legassick, G. Irving, and I. Gabriel. Ethical and social risks of harm from language models. arXiv.org, abs/2112.04359, 2021.
- Wen and Aydöre (2019) B. Wen and S. Aydöre. ROMark: A robust watermarking system using adversarial training. arXiv.org, abs/1910.01221, 2019.
- Wen et al. (2023) Y. Wen, J. Kirchenbauer, J. Geiping, and T. Goldstein. Tree-ring watermarks: Fingerprints for diffusion images that are invisible and robust. arXiv.org, abs/2305.20030, 2023.
- Xu et al. (2025) R. Xu, M. Hu, D. Lei, Y. Li, D. Lowe, A. Gorevski, M. Wang, E. Ching, A. Deng, et al. InvisMark: invisible and robust watermarking for AI-generated image provenance. In Proc. of the IEEE Winter Conference on Applications of Computer Vision (WACV), 2025.
- Xuehua (2010) J. Xuehua. Digital watermarking and its application in image copyright protection. In Proc. of the International Conference on Intelligent Computation Technology and Automation, volume 2, pages 114â117. IEEE, 2010.
- Yang et al. (2024) Z. Yang, K. Zeng, K. Chen, H. Fang, W. Zhang, and N. H. Yu. Gaussian shading: Provable performance-lossless image watermarking for diffusion models. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
- Yu et al. (2021) N. Yu, V. Skripniuk, S. Abdelnabi, and M. Fritz. Artificial fingerprinting for generative models: Rooting deepfake attribution in training data. In Proc. of the IEEE International Conference on Computer Vision (ICCV), 2021.
- Yu et al. (2001) P. Yu, H. Tsai, and J. Lin. Digital watermarking based on neural networks for color images. Signal Processing, 81(3):663â671, 2001.
- Zellers et al. (2019) R. Zellers, A. Holtzman, H. Rashkin, Y. Bisk, A. Farhadi, F. Roesner, and Y. Choi. Defending against neural fake news. In Advances in Neural Information Processing Systems (NeurIPS), 2019.
- Zhang et al. (2020a) H. Zhang, H. Chen, C. Xiao, B. Li, D. S. Boning, and C. Hsieh. Towards stable and efficient training of verifiably robust neural networks. In Proc. of the International Conference on Learning Representations (ICLR), 2020a.
- Zhang et al. (2023) H. Zhang, B. L. Edelman, D. Francati, D. Venturi, G. Ateniese, and B. Barak. Watermarks in the sand: Impossibility of strong watermarking for generative models. In Proc. of the International Conference on Learning Representations (ICLR) Workshops, 2023.
- Zhang et al. (2019) K. A. Zhang, L. Xu, A. Cuesta-Infante, and K. Veeramachaneni. Robust invisible video watermarking with attention. arXiv.org, abs/1909.01285, 2019.
- Zhang et al. (2024) L. Zhang, X. Liu, A. V. Martin, C. X. Bearfield, Y. Brun, and H. Guan. Robust image watermarking using stable diffusion. arXiv.org, abs/2401.04247, 2024.
- Zhang et al. (2018) R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
- Zhang et al. (2020b) X. Zhang, Z. H. Sun, S. Karaman, and S. Chang. Discovering image manipulation history by pairwise relation and forensics tools. IEEE Journal of Selected Topics in Signal Processing, 14(5):1012â1023, 2020b.
- Zhao et al. (2024) X. Zhao, K. Zhang, Z. Su, S. Vasan, I. Grishchenko, C. Kruegel, G. Vigna, Y.-X. Wang, and L. Li. Invisible image watermarks are provably removable using generative AI. In Advances in Neural Information Processing Systems (NeurIPS), 2024.
- Zhao et al. (2025) X. Zhao, S. Gunn, M. Christ, J. Fairoze, A. Fabrega, N. Carlini, S. Garg, S. Hong, M. Nasr, F. Tramer, et al. SoK: Watermarking for AI-generated content. In Proc. of the IEEE Symposium on Security and Privacy, 2025.
- Zhu et al. (2018) J. Zhu, R. Kaplan, J. Johnson, and L. Fei-Fei. HiDDeN: Hiding data with deep networks. In Proc. of the European Conference on Computer Vision (ECCV), 2018.
## Appendix A Additional robustness evaluations
### A.1 Native image resolution evaluation
While Section 8 evaluated all models at a 512 $Ă$ 512 resolution, this section evaluates all baselines as well as SynthID-O using the native resolution of images in ImageNet. The average resolution of ImageNet images is 469 $Ă$ 387, which means that methods that operate at larger resolutions may come at a slight disadvantage since images will generally be downscaled compared to the resolution at which they are watermarked (due to the intrinsic resizing stemming from variable image resolutions).
Table 4 summarizes the results. It reports the true positive rate (TPR) at a fixed false positive rate of 0.1% in average on worst -case transformations. We observe that over aggregated random and aggregated worst transformations, SynthID-O continues to outperform all other methods by significant margins (i.e., $+10.64$ and $+17.17$ percentage points respectively). In the worst setting, SynthID-O is the only models surpassing 97% TPR in worst-case aggregate (which is the most challenging setting).
| Aggregated Aggregated Worst | 98.72% 97.22% | [65.60%, 65.88%] [56.72%, 56.98%] | [72.24%, 72.43%] [66.21%, 66.48%] | [64.81%, 65.31%] [52.78%, 53.36%] | [77.24%, 77.70%] [68.06%, 68.21%] | [76.60%, 77.30%] [58.69%, 59.38%] | [87.93%, 88.08%] [76.31%, 76.49%] | 86.79% 80.05% |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
Table 4: TPR at 0.1% FPR aggregated across random and worst-case categories. For each model, the detection threshold is calibrated to reach 0.1% FPR across all worst transformations in average. When brackets are used, we display the range of TPR values that can accommodate the target FPR target(this number is not unique when there are ties). We highlight the best model in bold and the second best with an underline.
### A.2 Native model resolution evaluation
While the previous section evaluated all models using the native resolution of images in ImageNet, this section evaluates all baselines as well as SynthID-O at their preferred resolution. Table 5 reports the true positive rate (TPR) at a fixed false positive rate of 0.1% in average on worst -case transformations.
| Aggregated Aggregated Worst | 99.98% 99.72% | [61.45%, 61.77%] [51.11%, 51.40%] | [72.42%, 72.58%] [66.49%, 66.78%] | [58.91%, 59.54%] [47.89%, 48.38%] | [77.69%, 78.11%] [67.61%, 67.78%] | [64.53%, 65.45%] [52.08%, 52.60%] | [86.56%, 86.78%] [78.73%, 78.95%] | 82.32% 79.38% |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
Table 5: TPR at 0.1% FPR aggregated across each transformation category when images are resized to the model input size. For each model, the detection threshold is calibrated to reach 0.1% FPR across all worst transformations in average. When brackets are used, we display the range of TPR values that can accommodate the target FPR target(this number is not unique when there are ties). We highlight the best model in bold and the second best with an underline.
## Appendix B Human evaluation prompts
Below is the list of prompts used to generated images for our quality evaluations. We used Imagen 2 to generate 1000 images (i.e., 4 images per prompt).
| Realistic 3d render of a happy, furry and cute baby tiger smiling with big eyes looking straight at you, Pixar style, full body shot with a light blue background | 3D Rendering |
| --- | --- |
| Detailed robot portrait, shading details | 3D Rendering |
| A 3D rendering of a spaceship. | 3D Rendering |
| 3d rendering of a sleek, silver futuristic car with a glass roof and a glowing blue light on the front. | 3D Rendering |
| Isometric 3D render of a small Japanese house with a thatched roof, surrounded by bamboo and cherry blossoms. | 3D Rendering |
| 3D render of 3 planets orbiting around themselves in space. The planets are made of different colors and textures. The background is black. | 3D Rendering |
| 3D pixelated render of a panda in the style of Doom, standing on a rocky platform, with a lava background. | 3D Rendering |
| Ray-traced 3D rendering of a crystal ball on a table in a dark room. The ball is reflecting the light from a nearby candle. | 3D Rendering |
| 3D render of Gandalf fighting Balrog in a dark fantasy setting. The scene is rendered in Unity with realistic graphics. | 3D Rendering |
| Exciting car race from a 3D computer game, rendered in Unity, | 3D Rendering |
| Abstract art, very regular shapes and lines. | Abstract Art |
| Cubist painting, digital art, colourful, patchwork | Abstract Art |
| An abstract expressionist painting using bold brush strokes, dripping paint, and vibrant colors, reflecting the inner emotions and turmoil | Abstract Art |
| A regular black and white triangle pattern on a white background. The pattern is made of equilateral triangles, and the triangles are arranged in a grid. | Abstract Art |
| Abstract painting with all the colors of the rainbow, made with acrylic paint on canvas. | Abstract Art |
| Abstract art in greyscale. | Abstract Art |
| Abstract art piece of several overlapping circles in bright colors | Abstract Art |
| An art piece consisting of a lot of eyes painting in different style, abstract | Abstract Art |
| Drawing of different traffic signs chaotically arranged, abstract art | Abstract Art |
| Abstract art painting made with oil, different shades of green in a triangle shapes | Abstract Art |
| Painting of bear hiking in the mountains | Animal Painting |
| Ukiyo-e style cat | Animal Painting |
| Blacklight painting style cat, painting, blacklight painting | Animal Painting |
| A water color painting of a flamingo, Kilimanjaro in the background | Animal Painting |
| A panda in the style of Miro | Animal Painting |
| Panda in the style of Picasso, Cubism | Animal Painting |
| Abstract painting of a horse with broad brush strokes using only 2 colors | Animal Painting |
| Oil painting of a close-up of a cat eye | Animal Painting |
| A photorealistic painting of a family of African elephants with long tusks | Animal Painting |
| An impressionist painting of a butterfly | Animal Painting |
| Black and white picture of a swan swimming | Animal Photo |
| A picture of a cow on an alpine meadow. | Animal Photo |
| A herd of zebras standing on the surface of mars. The sun is shining. High quality photograph. 4k. | Animal Photo |
| Photograph of a coral reef with lots of colorful fishes swimming around. The reef is lit by sunlight coming through the water. | Animal Photo |
| Close up photo of a dog with the pop rocket instagram color filter. The dog is a golden retriever, and it is sitting on a red blanket. The background is a blur of green grass. | Animal Photo |
| Polaroid photo of a horse, taken in a park on a sunny day. 70s photo. Polaroid photo with yellowed edges. | Animal Photo |
| Close-up of a cat eye | Animal Photo |
| A full-body photograph of a Sumatran tiger roaring, with blurred background | Animal Photo |
| A photograph of a Humpback whale and its calf playing underwater | Animal Photo |
| Close up photograph of a group of ants. | Animal Photo |
| Anime drawing, portrait sunny weather, wearing sun hat, shadow hiding part of the face | Cartoon |
| A still from an 80s cartoon show about a butterfly with superpowers. The butterfly is wearing a cape and has a glowing red heart on its chest. It is flying through the air, surrounded by stars. | Cartoon |
| A male anime character with black hair runs through a deep forest. He is wearing a red ninja suit and has a sword in his hand. The sun is shining through the trees, creating a dappled pattern on the ground. anime, cartoon | Cartoon |
| A black and white cartoon of a giraffe driving a red car. The giraffe is wearing a top hat and a bow tie, and he is smiling. The car is driving down a road, and there is a palm tree in the background. | Cartoon |
| A cartoony, colorful icon of a mobile game app. The icon is of a smiling cat wearing a hat and sunglasses, and it is sitting on a pile of gold coins. The background is a bright blue sky with clouds. | Cartoon |
| A close-up portrait of a cute character in the style of Studio Ghibli, with big expressive eyes and a bright smile. | Cartoon |
| A dark and atmospheric scene from a seinen anime, with a lone character walking through a rain-soaked city, lit only by the neon lights of storefronts. | Cartoon |
| A whimsical and imaginative scene from a childrenâs cartoon, with a group of animal friends exploring a magical forest filled with talking trees and playful fairies. | Cartoon |
| A humorous and satirical scene from an cartoon, with a family sitting around the dinner table, engaging in witty banter and absurd situations. | Cartoon |
| A 1970s-style cartoon of a futuristic car flying through the sky. The car is bright yellow and has a big rocket engine on the back. The sky is a deep blue and there are clouds floating by. | Cartoon |
| Defocused specks of blurry swaying coral in a landscape colorful | Defocused Blurry |
| Close-up of out-of-focus grass | Defocused Blurry |
| street lights, beautiful bokeh | Defocused Blurry |
| a wasp in front of out-of-focus meadow | Defocused Blurry |
| A black and white photo of a microphone stand with the band members standing out of focus in the back. The band members are wearing black and white clothes, and the background is a dark blue. The photo is taken in a studio | Defocused Blurry |
| Underwater photo of a sunken ship. Out of focus. Very blurry. | Defocused Blurry |
| A supermarket selection of fruits. Out of focus and very blurry. | Defocused Blurry |
| A wall of text, out of focus, unreadable, blurry | Defocused Blurry |
| Sunset at a beach, out of focus, blurry | Defocused Blurry |
| A large group of people in a street, out of focus, blurry | Defocused Blurry |
| A portrait of a mysterious person sitting in an ornate, gothic throne room, with subtle elements of fantasy and magical realism, painted with the dark and dramatic style of the Baroque period | Fantasy Painting |
| A dynamic action scene of a knight battling a dragon on a cliffside, with roaring waves below, inspired by the Romanticism movement, reminiscent of the works of EugĂšne Delacroix | Fantasy Painting |
| A haunting gothic horror scene of an abandoned mansion on a hill, with fog, full moon, and shadowy figures, in a digital painting style reminiscent of the Castlevania game series. | Fantasy Painting |
| Dramatic fantasy painting of a Balrog facing a wizard. | Fantasy Painting |
| Fantasy painting of a pirate ship resting inside a canyon, with a waterfall cascading down the rocks. The ship is lit by a full moon, and the water is reflecting the stars. | Fantasy Painting |
| A group of orcs with axes in a forest, oil painting | Fantasy Painting |
| Detailed painting of a wizard casting a spell, she has a magic wand and sparkles fly out of it | Fantasy Painting |
| A beautiful painting of a unicorn grazing in a fairy tale forest | Fantasy Painting |
| A fantasy dwarf with a long beard sitting at a camp fire, painting | Fantasy Painting |
| Dramatic and rich painting of a medieval city with fireballs falling from the sky | Fantasy Painting |
| A bear dancing disco. Flat art style. | Flat art |
| Hedgehog in the grass and a tree in flat art style | Flat art |
| A large cruise ship with lots of flags on it. Flat art style. | Flat art |
| Flat art style postcard of a sailor. | Flat art |
| All planets of our solar system in flat art style. | Flat art |
| Picture of a flag of a fictional country. 5 diagonal blue and green stripes and a silver spoon in the top right corner. Made in flat art style. | Flat art |
| A steak with potatoes, carrots and green beans. Flat art style. | Flat art |
| Abstract flat art style image of a red radio tower with yellow lines emitting from it. White background. | Flat art |
| Lightning coming from a cloud and hitting a tree. Abstract. Minimalist. Flat art style. Cartoon in flat art style. | Flat art |
| Minimalist flat art style depiction of a woman sitting on a bench. The woman is black and wearing red clothes. Flat art style. Cartoon. Abstract. | Flat art |
| Black and white photo of a train conductor in a blue uniform standing on a train platform in 1875 | Historic Photo |
| A historic photograph of the Parthenon temple in Athens, Greece, taken before 1900 | Historic Photo |
| A historic photo of a family of 5. Mother, father, 2 sons and a daughter. | Historic Photo |
| A farmer in a field of sun flowers. Historic photo from 1912. | Historic Photo |
| Historic photo from 1874 of a personal computer. | Historic Photo |
| Cows in a barn. Black and white photo from 1930. | Historic Photo |
| Historic photo of a burning house. Taken in 1921. Grainy, bad quality. | Historic Photo |
| Black and white photo of a 1900s Model T Ford parked on a dirt road in front of a barn. The sky is cloudy and the sun is setting. | Historic Photo |
| Vintage photo of the Chicago skyline from the 1950s, with washed out colors and a sepia tone. | Historic Photo |
| Vintage photo of a steam boat on a large river, with a city in the background. | Historic Photo |
| A digital painting of a serene Japanese garden in autumn, with a tea house, koi pond, and maple trees with falling leaves, with an anime-inspired art style reminiscent of Studio Ghibli films. | Landscape Painting |
| A wide-angle realistic painting of mount Bromo volcano in Indonesia erupting, with visible lava and volcanic fumes coming out of the crater | Landscape Painting |
| A dreamy watercolour painting of many cherry blossom trees in front of Mount Fuji in Japan, mainly in pastel colours | Landscape Painting |
| A realistic painting of a stormy night in the mountains. Itâs raining and there are lightning visible in the background. | Landscape Painting |
| Oil painting of a savanna. The painting is in the Renaissance style, with a lot of detail and bright colors. | Landscape Painting |
| Impressionist painting of the ice cold plains of Antarctica. The sky is a deep blue, and the sun is setting. There is a sense of loneliness and isolation. | Landscape Painting |
| Expressionist painting of the Grand Canyon in shades of red, orange, and yellow. The painting is done on canvas and is 3 feet by 4 feet. | Landscape Painting |
| Baroque painting of a camel caravan crossing the dunes in the desert of Tunisia. The painting is done in a realistic style, with the dunes depicted in shades of yellow and brown. The sky is a deep blue, and the sun is setting, casting long shadows across the sand. | Landscape Painting |
| A painting of the city of London in the early medieval period, with a castle in the foreground and a river in the background. The painting is done in a realistic style, with the colors muted and the details precise. | Landscape Painting |
| A modern painting of a deserted beach in the Caribbean. The painting is done in a vibrant, fauvist style, and the colors are saturated and bright. The beach is deserted, with only a few palm trees and a few rocks. The sky is a deep blue, and the water is a clear turquoise. | Landscape Painting |
| High quality photo of a sun set on the beach. | Landscape Photo |
| A serene autumn landscape with a reflective lake surrounded by trees with colorful fall foliage, in the style of the Hudson River School, capturing the romantic essence and fine details. | Landscape Photo |
| A black and white street photography of a busy New York City intersection with the Empire State Building in the background, capturing the energy and chaos of city life | Landscape Photo |
| A realistic landscape shot of the Northern Lights dancing over a snowy mountain range in Iceland, with long exposure to capture the motion and vibrant colors. | Landscape Photo |
| A realistic photo of the Milky Way with diagonal orientation, next to the Tre Cime di Lavaredo mountain peaks in Italy | Landscape Photo |
| A photo of a Baroque chapel on a green valley catching rays of the rising sun, with defined mountain peaks in the background | Landscape Photo |
| A photograph of the Golden Gate bridge on a foggy sunset, with its pillars and the water underneath being illuminated by red light | Landscape Photo |
| A dramatic black and white photo of a stone bridge leading to a lighthouse on a rocky cliff, with ocean waves crashing onto the cliff | Landscape Photo |
| A photograph capturing the movement of black horses running on the side of an empty road which leads towards a large snowy mountain range | Landscape Photo |
| A beautiful photo of a medieval castle in the mountains at sunset. The castle is made of gray stone and has turrets and towers. The mountains are covered in snow and the sky is a deep blue. | Landscape Photo |
| Figurative line drawing | Line Drawing |
| Blueprint for a rocket ship heading to Mars, blueprint, pencil drawing, drawing | Line Drawing |
| City skyline picasso pencil | Line Drawing |
| Sketch of the ying yang | Line Drawing |
| Architectural drawing of bank floor plan, line drawing | Line Drawing |
| A line drawing of a womanâs face, with geometric patterns used to define facial features, line drawing | Line Drawing |
| Line drawing of two people arguing, pencil. | Line Drawing |
| A black and white line drawing of a skyscraper driving with wheels on a road. | Line Drawing |
| Colorful line drawing of a rooster, line drawing | Line Drawing |
| Pencil drawing of a complex machine. Line drawing, pencil drawing | Line Drawing |
| A mascot logo of a robot, simple, vector | Logo |
| A minimalist logo of a research company, consisting of a magnifying glass and a DNA strand. The logo is black and white, and the background is a light blue. Just the logo. | Logo |
| A logo of a fashion company. The logo is a stylized letter "Y" in bright colors. The logo is on a white background. | Logo |
| A complex logo made of geometric shapes and lines in a dark blue color. The logo is on a white background. | Logo |
| A red and white logo of a games company. The logo is a wolf with sunglasses on black background. | Logo |
| A playful logo of a AI startup making shoes. The logo is a cartoon of a shoe with a big smile on its face. The shoe is wearing a hard hat and a tool belt. Logo on a white background. | Logo |
| A simple, modern logo for a company called Logo. The logo is a blue circle with a white letter L in the center. The logo is on a white background. | Logo |
| Logo for an early 2000s dot-com bubble company that sells books online. The logo is a stylized book with a rocket ship on top. | Logo |
| A modern, minimalist logo for a new supermarket chain. The logo is a green leaf with a wavy black line circling around it. On a pink background. | Logo |
| Logo for a new online bank. The logo has several currency symbols in triangle pattern with the word bank above it. The background is black. | Logo |
| A cubist painting of a still life with geometric shapes, fragmented forms, and multiple perspectives, inspired by the works of Pablo Picasso and Georges Braque, using a muted color palette | Object Painting |
| Minimalist painting of a flower on black background. Very minimalistic. Extremely minimalistic. Minimalism. | Object Painting |
| A painting of a knight in the style of fauvism, with bold colors and simplified shapes. | Object Painting |
| A Horse Drawn Carriage. Simplistic woodcut from 1568. | Object Painting |
| Still life painting of a calculator. Oil on canvas | Object Painting |
| Surrealist painting of a toaster melting on a white background. The toaster is made of blue and yellow plastic, and the melting is done in a realistic style. | Object Painting |
| A baroque painting of a chest full with treasure. | Object Painting |
| A painting of a pineapple in the cubist style. | Object Painting |
| An impressionist painting of a shoe. Impressionist style. | Object Painting |
| Oil painting of an very intrinsically decorated vase | Object Painting |
| Chess board, high contrast, top view | Object Photo |
| A close-up macro shot of morning dew on a spiderâs web, with the focus on the intricate patterns and water droplets, giving it an ethereal quality, high resolution. | Object Photo |
| Large fireworks in the night sky. | Object Photo |
| A close up photograph of snowflakes | Object Photo |
| Food photo of a steak with potatoes, carrots and green beans. | Object Photo |
| A photo of a house built of match sticks. | Object Photo |
| A table covered in various tools, high quality photograph | Object Photo |
| Photo of an astronauts helmet, you can see the surface of the moon in the reflection | Object Photo |
| A photograph of a broken glass lying across a marble floor. | Object Photo |
| Very close up photo of a purple light diode | Object Photo |
| Pencil drawing of a cloud | Pencil Drawing |
| 1950s cat illustration | Pencil Drawing |
| Architectural pencil drawing of a residential home. | Pencil Drawing |
| Pencil drawing of steam train. | Pencil Drawing |
| A pencil drawing of a street in Berlin. There is a döner kebap shop next to a bar. | Pencil Drawing |
| A pencil drawing of a cityscape, with shadows on the glass and concrete of tall buildings | Pencil Drawing |
| An extremely detailed pencil portrait of a black womanâs face | Pencil Drawing |
| Pencil drawing of a rainbow | Pencil Drawing |
| Extremely realistic pencil drawing of an eye | Pencil Drawing |
| An inkjet printer drawn with pencil, pencil drawing | Pencil Drawing |
| A man with a blue hat jumping on a two legged turtle. Pixel Art. | Pixel art |
| A female doctor jumping over an abyss. Pixel art game screenshot. | Pixel art |
| A pixelated screenshot of a 8-bit game, showing a spaceship shooting at asteroids. Pixel Art. | Pixel art |
| Pixel art of a steampunk machine with gears, cogs, and valves. Pixel art. Pixelated. Super Nintendo graphics. | Pixel art |
| A pirate ship on the sea, pixel art, 16-bit | Pixel art |
| 16-bit pixel art of a supermarket | Pixel art |
| A knight on a horse riding over a draw bridge, pixel art | Pixel art |
| A group of crocodiles in a river, pixel art | Pixel art |
| Pixel art drawing of cars on a highway, top view, pixel art | Pixel art |
| A desert scene with cacti and tumble weeds, 32-bit pixel art | Pixel art |
| A portrait of a man in glasses, psychedelic painting, primary colors, water color | Portrait Painting |
| Renaissance painting of a king wearing a crown, reneaissance painting | Portrait Painting |
| Abstract painting showing the portrait of women in her 20s on the left with bright colors and in her 80s on the right with darker colors | Portrait Painting |
| Painting with thick lines and bold colors of a doctorâs face | Portrait Painting |
| Portrait of a female african-american professor using pointillism | Portrait Painting |
| Official painted portrait of the first Indian-American President of the United States | Portrait Painting |
| Painted portrait of a happy child, oil painting | Portrait Painting |
| Portrait painting of a female baker in traditional white clothing | Portrait Painting |
| Baroque portrait of a blacksmith in his workshop, baroque painting, painting | Portrait Painting |
| Very detailed painting of a female Chinese bank manager | Portrait Painting |
| Close-up portrait of a man, Sepia | Portrait Photo |
| Colorful vivid portrait of a woman, bokeh, kodachrome | Portrait Photo |
| A powerful portrait of an elderly person, capturing the wrinkles, texture of the skin, and expressive eyes that tell a story of a lifetime, in high resolution and natural lighting. | Portrait Photo |
| A black and white portrait of a middle-aged Vietnamese lady smiling, and wearing a traditional outfit and bamboo hat | Portrait Photo |
| A portrait of a kid, balloons floating in the background. | Portrait Photo |
| Bokeh portrait shot of a group of 3 adult brothers, in a garden | Portrait Photo |
| Portrait photo of a man in his 50s with sun glasses on, he has green colored hair and smiles | Portrait Photo |
| A airplane pilot in her full uniform, bokeh photo | Portrait Photo |
| Wedding photo of a couple | Portrait Photo |
| College graduation photo of a native american man | Portrait Photo |
| Movie Poster of a 1980s comedy movie about racoons living on a college campus | Poster Art |
| A poster of a Maneki-Neko (Japanese lucky cat) and a bowl of ramen noodles on a grey background, with the cat holding a few noodles in its raised paw | Poster Art |
| A coffee guide poster with colourful icons of different kinds of coffees and brewing methods | Poster Art |
| A poster showing the dramatic cliffs and colourful houses in the Amalfi Cost, Italy, with text rendered on top âITALY, AMALFI COASTâ | Poster Art |
| Abstract poster of a pattern of thick black lines looking like a maze | Poster Art |
| Poster of an animated movie about a funny bear with title | Poster Art |
| A poster for an action movie featuring a a panda in sunglasses, holding a bamboo stick. The background is dark and stormy, and the panda is standing in front of a burning building. | Poster Art |
| An advertisement poster for a new detergent | Poster Art |
| A scientific poster explaining neural networks to children | Poster Art |
| Advertisement poster of a new smartphone from the 1960s | Poster Art |
| A futuristic cyberpunk cityscape at night with towering neon-lit skyscrapers, flying cars, and a diverse crowd of humans and androids, in a highly detailed digital painting reminiscent of Blade Runner. | SciFi Photo |
| A portrait of a post-apocalyptic warrior with battle scars, armor, and weapons, standing in a desolate wasteland, in a detailed digital painting with gritty textures, inspired by the Mad Max franchise. | SciFi Photo |
| Empty spaceship interior. Dramatic light. | SciFi Photo |
| A futuristic hover craft with cyan and orange tones | SciFi Photo |
| A spacesuit-clad astronaut walking through a forest, surrounded by trees and ferns. The astronaut is carrying a backpack. | SciFi Photo |
| Photo of a green alien in Paris dressed like a tourist | SciFi Photo |
| Photo of a woman in a futuristic spacesuit floating in space | SciFi Photo |
| Dramatic photo of a UFO hovering over a fast food restaurant | SciFi Photo |
| Photo of a man in a futuristic jumpsuit riding a gigantic grasshopper | SciFi Photo |
| Close up photo of a futuristic laser gun | SciFi Photo |
| Black matte cube in the middle of room of a poorly lit room | Shapes |
| A dark but slightly transparent sphere, fre-floating in front of clear sky | Shapes |
| A solid prism on a table in front of a forest | Shapes |
| An abstract sculpture out of dark metal in the middle of a traffic circle | Shapes |
| A closeup photo of a honeycomb with hexagonal prismatic cells of varying size | Shapes |
| A page of a notebook with a pencil drawing of a triangle. The notebook is open to a blank page, and the triangle is drawn in the center of the page. The pencil is resting on the page next to the triangle. | Shapes |
| Two circles overlapping (Venn diagram) | Shapes |
| An image filled with concentric circles | Shapes |
| Multiple irregular polygons in black and white | Shapes |
| A smooth white pyramid on a white floor | Shapes |
| A scientist writing an equation on a blackboard | Stock Photo |
| A trader looking at stock price graphs on the screen | Stock Photo |
| A group of business professionals with different nationalities having a meeting | Stock Photo |
| A female hacker sitting in front of a computer, she is wearing a dark hoody and the room is badly lit | Stock Photo |
| Stock photo of a man eating a salad | Stock Photo |
| Two men holding hands smiling at the camera | Stock Photo |
| An Asian man in a cafe ordering, looking pensive, stock photo | Stock Photo |
| Two women high fiving. One is African American and one is Indian American | Stock Photo |
| Stock photo of a person having a headache | Stock Photo |
| A woman having an idea, stock photo | Stock Photo |
| The word provenance in a funny font | Text |
| Restaurant menu written on a yellow page, written menu, full of text | Text |
| A large blackboard with lecture notes from a computer science lecture | Text |
| A small whiteboard with Maxwellâs equations | Text |
| 3D rendering of letters forming "HELLO", with the front surface of each 3D letter having a different inclination | Text |
| Logo of well-known web search company. White background and colorful letters. | Text |
| A white letter with the words "I love you" written on them in red | Text |
| All the letters of the alphabet | Text |
| A soup with letters in it forming the word "Soup" | Text |
| A page in a very old book | Text |
| A vertical wallpaper that is painted to look like a birdâs wing, with the wing in different shades of gray and streaks of light in the sky. | Wallpaper |
| Golden tree branches densely packed against a dark background | Wallpaper |
| Clouds in a blue sky, wallpaper photo | Wallpaper |
| Matte colors in triangle shapes, wallpaper | Wallpaper |
| Regular pattern of stick figures holding hands | Wallpaper |
| Wallpaper of stars on a black background | Wallpaper |
| Stones of different colors and shapes, wallpaper | Wallpaper |
| An empty beach from above, perfect photography, clear blue water | Wallpaper |
| Wooden texture | Wallpaper |
| Image of a seastar, made as a tile mosaic, blue and white, phone wallpaper | Wallpaper |