Case study · Human-centered AI
Human detection of AI-generated video
Generative video has moved from uncanny to convincing within a few years. This controlled study measures how accurately untrained viewers detect it, and under which conditions detection fails.
across 3,132 real-or-AI judgments by 87 participants, against a 50% chance level.
Overview
Designer and researcher, first author
With Prof. R. M. Karim (NUST) and Prof. KwanMyung Kim (UNIST)
IASDR 2025, pictorial track, Taipei
87 adults aged 18 to 70, none with professional AI, graphics or video experience
Within-subjects online study, binary forced choice, randomized order
Logistic regression with Holm–Bonferroni correction
Research on synthetic media has concentrated on face and voice deepfakes. Designers and media artists, however, now use AI video in three distinct ways: generating clips from text, animating still images, and extending real footage beyond its frame. These techniques had not been compared side by side.
How accurately can untrained viewers detect each generation technique?
How does content theme moderate detection across techniques?
A balanced stimulus set of 36 clips
Half real and half AI-generated, distributed evenly across three themes. Pilot tests showed that AI renders some subjects, such as static objects, far better than people or motion; varying the themes kept the comparison balanced.
Stimulus design
Clips per theme and source
Stimulus generation
A single pipeline, so that only the technique varied
All AI clips were produced in ComfyUI, whose node graphs allowed diffusion steps, guidance and prompt weights to be replicated across methods. Five designers generated candidates for each condition, and one was selected for visual fidelity, temporal coherence and thematic relevance.
Real clips were drawn from publicly available stock and archival footage matched to the same themes. All 36 clips were trimmed to 8–12 seconds, standardized to 1080p and presented without sound.
Procedure
Binary forced choice across 36 trials
Each clip is shown individually, in randomized order, and can be replayed without limit.
“Do you believe this video is real or AI-generated?” Response options: real or AI.
A short break after 18 clips to limit fatigue.
Participants were not told the proportion of AI clips and were instructed to judge each clip on its visual content alone.
Detection accuracy depended on technique and theme
Accuracy by source
Share of clips classified correctly. For real footage, correct means “real.”
Text-to-video accuracy fell below chance, and outpainting accuracy was at chance. Image-to-video retained motion cues that viewers could detect.
Real nature footage was often judged synthetic. Real environmental clips were classified correctly only about 48% of the time; pristine natural scenery frequently read as AI-generated.
Theme strongly moderated outpainting detection. Outpainting was detected in 61% of environment trials but only 32% of historical trials.
Image-to-video cues held across themes. Image-to-video clips were identified consistently in the film, history and environment themes.
| Term, compared with real film footage | Odds ratio | p |
|---|---|---|
| Image-to-video | 1.13 | n.s. |
| Text-to-video | 0.50 | .001 |
| Outpainting | 0.45 | .001 |
| Environment theme | 0.37 | < .001 |
| Image-to-video × Environment | 8.22 | < .001 |
| Outpainting × Environment | 6.89 | < .001 |
| Text-to-video × History | 0.44 | .003 |
| Outpainting × History | 0.41 | < .001 |
| Technique, theme and their interaction were all significant (p < .001). p-values Holm–Bonferroni adjusted. 87 participants × 36 clips = 3,132 observations. | ||
Implications for media literacy and creative practice
Awareness of AI undermines trust in real footage
Real clips were classified correctly only 64% of the time. When viewers expect AI content, authentic footage also begins to appear synthetic.
Perceived realism depends on content, not only the model
Perceived realism is not a fixed property of a model; it depends on what a clip depicts. Creators can use this deliberately to calibrate how real a piece appears.
Detection training should address context as well as artifacts
Media-literacy programs and detection tools should cover contextual cues as well as model-specific artifacts; intuition alone is insufficient.
Limitations and next steps. Short, silent clips viewed online limit ecological validity. Future work should include techniques such as inpainting, ControlNet and depth-aware video, collect confidence ratings, broaden the demographic sample, and test whether training improves detection over time.
