This work aims to achieve more precise and versatile object control in image-to-video (I2V) generation. Current methods typically represent the spatial movement of target objects with 2D trajectories, which often fail to capture user attention and frequently produce unnatural results. To enhance control, this paper presents ObjCtrl-2.5D, a training-free object control approach that uses a 3D trajectory, extended from a 2D trajectory with depth information, as a control signal. By modeling object movement as camera movement, ObjCtrl-2.5D represents the 3D trajectory as a sequence of camera poses, enabling object motion control using an existing camera motion control I2V generation model (CMC-I2V) without training. To adapt the CMC-I2V model originally designed for global motion control to handle local object motion, we introduce a module to isolate the target object from the background, enabling independent local control. In addition, we devise an effective way to achieve more accurate object control by sharing low-frequency warped latent within the object's region across frames. Extensive experiments demonstrate that ObjCtrl-2.5D significantly improves object control accuracy compared to training-free methods and offers more diverse control capabilities than training-based approaches using 2D trajectories, enabling complex effects like object rotation.

ObjCtrl-2.5D first extends the provided 2D trajectory to a 3D trajectory using depth information from the conditioning image. This 3D trajectory is then transformed into a camera pose via triangulation algorithm. To achieve object motion control within a frozen camera motion control module, ObjCtrl-2.5D integrates a Layer Control Module (LCM) that separates the object and background with distinct camera poses. After extracting camera pose features via a Camera Encoder, LCM spatially combines these features using a series of scale-wise masks. Additionally, ObjCtrl-2.5D introduces a Shared Warping Latent technique, which enhances control by sharing low-frequency initialized noise across frames within the warped areas of the object.

Results Control with Trajectory

Results Control with Camera Pose

Flexible Background Movements

Compare with Previous Methods

(A) Compare with Training-free Methods

(B) Compare with Training-based Methods

(C) More Comparisons

Ablation Studies

Failure Cases

Due to the limitations of SVD (the basic T2V generation model of ObjCtrl-2.5D) in handling large motions, ObjCtrl-2.5D with high-speed camera poses results in the object fading out of the scene, leaving only the background. Interestingly, this outcome reveals potential for image inpainting applications, as seen in the last frames of the generated videos.