Samuel Teodoro's Paper Accepted for IEEE TPAMI Journal

Samuel Teodoro's paper has been accepted for IEEE TPAMI Journal. Congratulations~

Title: PRIMEdit: Probability Redistribution for Instance-aware Multi-object Video Editing with a New Benchmark Dataset

Authors: Samuel Teodoro, Agus Gunawan, Soo Ye Kim, Jihyong Oh, and Munchurl Kim

Abstract:

Recent Artificial Intelligence (AI)-based video editing has enabled users to edit videos through simple text prompts, significantly simplifying the editing process. However, recent zero-shot video editing techniques focus on global or singleobject edits, which can lead to unintended changes in other parts of the video. When editing multiple objects only within their localized regions, existing methods face challenges, such as unfaithful editing, editing leakage, and lack of suitable evaluation datasets and metrics. To overcome these limitations, we propose Probability Redistribution for Instance-aware Multi-object Video Editing (PRIMEdit). PRIMEdit is a zero-shot framework that introduces two key modules: (i) Instance-centric Probability Redistribution (IPR) to ensure precise localization and faithful editing and (ii) Disentangled Multi-instance Sampling (DMS) to prevent editing leakage. Additionally, we present the MIVE dataset for multi-instance video editing, including its subsets categorized by instance counts and sizes for controlled analysis across different levels of scene complexity. We also introduce the Cross-Instance Accuracy (CIA) Score that is firstly proposed to evaluate the editing leakage in multi-instance video editing tasks, which exhibits strong correlation with human evaluation as measured by Spearman’s ρ = 0.86. Our extensive qualitative, quantitative, and user study evaluations demonstrate that PRIMEdit significantly outperforms recent state-of-the-art methods in terms of editing faithfulness, accuracy, and leakage prevention, setting a new benchmark for multi-instance video editing.