CamDirector:面向长期一致的视频轨迹编辑

📄 中文摘要

视频(摄像机)轨迹编辑旨在合成遵循用户定义的摄像机路径的新视频,同时保持场景内容并合理地填补之前未见区域,从而将业余视频升级为专业风格的视频。现有的视频轨迹编辑方法在精确的摄像机控制和长距离一致性方面存在困难,因为它们要么通过有限容量的嵌入注入目标姿态,要么依赖于仅具有隐式跨帧聚合的单帧扭曲。为了解决这些问题,提出了一种新的视频轨迹编辑框架,该框架通过混合扭曲方案显式聚合整个源视频的信息。具体而言,静态区域逐步融合到一个世界缓存中,然后进行渲染。

📄 English Summary

CamDirector: Towards Long-Term Coherent Video Trajectory Editing

Video trajectory editing aims to synthesize new videos that follow user-defined camera paths while preserving scene content and plausibly inpainting previously unseen regions, effectively upgrading amateur footage into professionally styled videos. Existing video trajectory editing methods struggle with precise camera control and long-range consistency, as they either inject target poses through a limited-capacity embedding or rely on single-frame warping with only implicit cross-frame aggregation in video diffusion models. To address these challenges, a new video trajectory editing framework is introduced that explicitly aggregates information across the entire source video via a hybrid warping scheme. Specifically, static regions are progressively fused into a world cache and then rendered.

Powered by Cloudflare Workers + Payload CMS + Claude 3.5

数据源: OpenAI, Google AI, DeepMind, AWS ML Blog, HuggingFace 等