M. Kalinski et al., “Addressing Character Consistency Challenges in AI Filmmaking,” 2025 International Conference on Computing, Networking and Communications (ICNC), Honolulu, HI, USA, 2025, pp. 642-648, doi: 10.1109/ICNC64010.2025.10993630. https://ieeexplore-ieee-org.ezproxy.emich.edu/document/10993630
When watching longer form AI generated video content, viewers may notice that the characters don’t look the same between scenes. So how can we fix that? This article covers some of the different models of AI, and how they do with character consistency. There is a bit of math in this article, but to summarize, closed diffusion models create high quality images, but struggle with consistency because closed diffusion models don’t build on prior work (this is, of course, a massive simplification). Open diffusion models do build on prior work, and using a technique called low-rank adaptation (LoRA), users can “fine-tune” what they generate because the model uses something like an anchor point to base the content on. You can use the closed models to create something of a data set, and then feed that to other models to use for video generation.
This article is massive, and I can’t help but think that this is more work than just hiring an actor or learning how to 3D model. These datasets also remind me a lot of what textures look like in the files of video games. Although this is very interesting, it’s very obviously using techniques that already existed in animation and video game design, in a much lazier way.
Leave a Reply