SAFE Synthetic Video Detection Challenge 2025

Register your team to get started. The principal investigator fills out the registration form -- organizers manually approve and issue your access token.

Registration Closed

• Competition tasks are closed. Thanks to all the participants.
• Private Leaderboard is now visible for every task: SAFE Video Challenge Collection
• Competition Results will be presented at the APAI @ ICCV 2025 October 19, 2025.
• Look out for our paper with competition details.

• Competitions Close at 23:59 October 10th UTC. The task spaces will not accept any more submissions after that.
• Final Rankings. Final rankings will be w.r.t. to balanced accuracy on the private set. Remember the private set is a superset of the public. It contains examples from additional real and synthetic sources not present in the public set. Results on the private set will be shared after the competition closes.
• (Optional) Select Your Submissions For Private Leaderboard. You are allowed to select up to 2 submissions for the private leaderboard. If you do not make any selections then your submission with the highest public score will be chosen. If you do make any selections then the submission with the highest private score (out of the 2 selected) will be used for the private leaderboard. You can do this by checking the "selected" box next to a submission from the "My Submissions" section for each task space on huggingface. For example, this allows you to choose an additional submission with a lower public score for the private leaderboard. In this competition, the private set is a superset of the public, so we don't expect this option to make much difference but we wanted to clarify it just in case.

• Task 2 Detection of Synthetic Video Content after post-processing is now open.
• This task will measure impact of common post-processing techniques on model performance.
• In this task, we apply several common post-processing techniques (resizing, compression, etc.) to a subset of Task 1 data.
• Please submit your best model from Task 1 to this task so that we can have a meaningful comparison.
• We'll update the detailed leaderboard to show model performance conditioned on each (anonymized) post-processing operation.

• Automatic Submission Log Summaries.It can be accessed under "My submission" -> "Admin Comment" in each task space.
• Upgraded Compute to have 2x CPU & GPU memory: Nvidia 1xL40S, 8 vCPU, 62 GB RAM. 48 GB VRAM .
• Updated Sample Submission. Some of the videos can get large (4k and 60fps): a few tips to better manage memory in the updated submission repo: (1) moving to GPU after every frame, (2) checking memory and (3) center cropping. See sample repo

Detailed Leaderboard is Live. It shows detailed performance: TPR, TNR and balanced accuracy conditioned on an anonymized source.

• Added pointers to some publicly available training datasets. While we do not provide training data, you are free to use anything you want for training.
• The challenge duration is extended until the workshop (Oct 19th). We'll announce a cut-off date (early October) at which the leading teams will be determined.
• There will also be a remote option to participate in the competition session at the APAI workshop.

Pilot task is live SAFE Video Challenge Collection

Task 1 is live SAFE Video Challenge Collection

This challenge uses novel, private datasets for evaluation. To maximize the validity of the performance measurements:

• No training data will be provided. Participants may train on any data they have rights to use.
• Evaluation data will not be published. Organizers will publish general descriptions of the collection methodology only.
• Small data samples are provided only to validate submission flow and are not representative of the actual evaluation data content.

Detection of Synthetic Video Content:

• The objective is to detect synthetically generated video clips.
• Your model must submit a binary decision for every example.
• The submissions will be ranked by balanced accuracy.
• The data consists of real and synthetic videos. The latter are generated with several older generative models. The former (reals) will be pulled from various sources. The videos will cover a wide range of content such as natural scenes, human activities, animals, etc. There is no audio in the videos. Please see the representative grid below.
• The video data will consist of various common compression formats, varying resolution, bitrate, FPS and length. The length of each clip will vary, averaging around 5 seconds but no longer than 60 seconds. The resolution of the video will also vary but will not exceed 2000 x 2000 pixels.
• The purpose of the pilot task is to allow participants to test initial submission logistics, to support early experimentation and understand task dynamics before later tasks open.
• For this task, the data will be randomly divided into a private and public split.

Detection of Synthetic Video Content:

• The objective is to detect synthetically generated video clips.
• The task is similar in structure to the pilot task, but it contains newer generative models of higher quality and realism. The real videos also cover a wider range of content and sources.
• The synthetic videos are generated by a range of state-of-the-art techniques, including image/text-to-video and other generative video models.
• The real videos are pulled from various sources and cover a wide range of content such as natural scenes, human activities, animals, etc.
• There is no audio in the videos.
• The video data will consist of various common compression formats, varying resolution, bitrate, FPS and length. The length of each clip will vary averaging around 5-10 seconds. The resolution of the video will also vary but will not exceed 2000 x 2000 pixels.
• The focus will be on generalization to unseen generators, robustness to visual variability, and applicability to real-world forensics workflows.
• For this task, the public set will consist of a subset of generator models and sources. The private set will include additional model and sources not present in the public set. Performance breakdown by source/model will be available in the detailed leaderboard.

Detection of Synthetic Video Content after Post-processing:

• The objective is to detect synthetically generated video clips.
• The task is similar in structure to the Task 1, but the videos are post-processed with several methods.
• Post-processing includes common operations seen on the internet such as compression, resizing, cropping and others.
• The videos come from the same models and sources as in Task 1.
• Please submit your bset model from Task 1 to this task so that we can measure how much post-processing impact model performance.

• The competition page will maintain a public leaderboard and a private leaderboard. The data will be devided differently between public and public depending on the task.
• The leaderboards will also show true positive rate, true negative rate, AUC and fail rate.
• A detailed public leaderboard will also show error rates for every source, and perhaps additiona breakdowns. However, the specific source name will be anonymized.

1. The competition will maintain both a public and a private leaderboard.
2. The public leaderboard will show error rates for each anonymized source.
3. The private leaderboard will be used for the final evaluation and will include non-overlapping data from the public leaderboard.

Participants will be limited in submissions per day.

1. Participants agree not to publicly compare their results with those of other participants until the other participant’s results are published outside of the conference venue.
2. Participants are free to use and publish their own results independently.

1. Participants must comply with all rules and guidelines provided by the organizers.
2. Failure to comply with the rules may result in disqualification from the competition and exclusion from future evaluations.

[email protected]

Discord Server Invite

https://huggingface.co/safe-challenge-2025

Team GRIP-UNINA:
PI Davide Cozzolino, Image Processing Research Group, University Federico II Naples, Italy

Teams DASH:
PI Simon Woo, DASH Lab, Sungkyunkwan University, South Korea

Team ISPLynx:
Paolo Bestagini PI, Image and Sound Processing Lab (ISPL), Politecnico Milano