---
title: "Bootstrapping Video Interaction Generation with Synthetic State Transitions"
authors:
  - "Jiho Jang"
  - "Jin-Young Kim"
  - "Nojun Kwak"
  - "Kyungjune Baek"
doi: "10.24963/ijcai.2026/139"
url: "https://ijcai.org/proceedings/2026/139"
pdf: "https://ijcai.org/proceedings/2026/0139.pdf"
venue: "Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence"
year: 2026
pages: "1243-1251"
track: "Main Track"
keywords:
  - "Computer Vision: Image and video synthesis and generation"
  - "Computer Vision: Video analysis and understanding"
---

# Bootstrapping Video Interaction Generation with Synthetic State Transitions

Jiho Jang, Jin-Young Kim, Nojun Kwak, Kyungjune Baek

*Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence*, pages 1243-1251. DOI: [10.24963/ijcai.2026/139](https://doi.org/10.24963/ijcai.2026/139)

## Abstract

While recent video generative models can synthesize high-fidelity videos, they struggle to portray plausible physical interactions and the resulting state transitions, a critical bottleneck for applications in robotics and VR/AR. To address this, we introduce a framework to generate a scalable synthetic dataset of controllable interactions.
Our pipeline leverages a structured taxonomy and state-of-the-art image editing models to create explicit `start' and `end' state images, which serve as visual anchors for the interaction. To generate a seamless video utilizing these anchors, we propose State-Guided Sampling (SGS), a novel sampling technique that mitigates artifacts common in naive conditional generation. Furthermore, we develop and validate a new automated evaluation system that aligns with human judgments to ensure data quality. 
Experiments show that fine-tuning a base model on our dataset significantly enhances its ability to generate plausible interactions.