{"type":"ScholarlyArticle","title":"Bootstrapping Video Interaction Generation with Synthetic State Transitions","authors":[{"name":"Jiho Jang","slug":"jiho-jang","given":"Jiho","family":"Jang","affiliation":""},{"name":"Jin-Young Kim","slug":"jin-young-kim","given":"Jin-Young","family":"Kim","affiliation":""},{"name":"Nojun Kwak","slug":"nojun-kwak","given":"Nojun","family":"Kwak","affiliation":""},{"name":"Kyungjune Baek","slug":"kyungjune-baek","given":"Kyungjune","family":"Baek","affiliation":""}],"abstract":"While recent video generative models can synthesize high-fidelity videos, they struggle to portray plausible physical interactions and the resulting state transitions, a critical bottleneck for applications in robotics and VR/AR. To address this, we introduce a framework to generate a scalable synthetic dataset of controllable interactions.\r\nOur pipeline leverages a structured taxonomy and state-of-the-art image editing models to create explicit `start' and `end' state images, which serve as visual anchors for the interaction. To generate a seamless video utilizing these anchors, we propose State-Guided Sampling (SGS), a novel sampling technique that mitigates artifacts common in naive conditional generation. Furthermore, we develop and validate a new automated evaluation system that aligns with human judgments to ensure data quality. \r\nExperiments show that fine-tuning a base model on our dataset significantly enhances its ability to generate plausible interactions.","doi":"10.24963/ijcai.2026/139","url":"https://ijcai.org/proceedings/2026/139","pdf":"https://ijcai.org/proceedings/2026/0139.pdf","markdown":"https://ijcai.org/proceedings/2026/139.md","bibtex":"https://ijcai.org/proceedings/2026/bibtex/139","volume":{"slug":"2026","title":"Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2026","url":"https://ijcai.org/proceedings/2026/","year":2026,"location":"Bremen, Germany","dates":"15-21 August 2026","editors":"Diego Calvanese","isbn":"978-1-956792-09-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2026"},"section":"Computer Vision","track":"Main Track","number":139,"pages":{"first":1243,"last":1251},"keywords":["Computer Vision: Image and video synthesis and generation","Computer Vision: Video analysis and understanding"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}