Interactive Image Segmentation with Temporal Information Augmented


Qiaoqiao Wei (School of Software, Tsinghua University), Hui Zhang (Tsinghua University), Jun-Hai Yong (Tsinghua University, Tsinghua University)
The 35th British Machine Vision Conference
PDFPosterVideo (Right click to download)Supplementary

Abstract

Interactive image segmentation aims to achieve pixel-wise localization of an object of interest in an image using minimal user annotations. Despite advances, existing methods suffer from accuracy fluctuations and notable constrained minimal errors due to annotation sparsity and neural network limitations. To improve segmentation quality and stability, this paper proposes the Temporal Information Augmentation (TIA) method. Informed by the concept of the proportional-integral-derivative (PID) controller, TIA integrates contextual information from multiple interaction rounds to enhance feature representations. Specifically, TIA strengthens the response of current feedback information through cosine feature similarities, fuses foreground and background instructive information from past interaction rounds with current features, and refines features in potential wrongly segmented areas by perceiving changes in the segmentation results. By incorporating current, past, and future contextual cues, TIA improves the discrimination ability of the segmentation model for target objects. Experimental results on the GrabCut, Berkeley, SBD, and DAVIS datasets with SegFormer- and ViT-based backbones have demonstrated state-of-the-art performance, highlighting generalization capability, efficiency, and effectiveness of TIA.

Citation

@inproceedings{Wei_2024_BMVC,
author    = {Qiaoqiao Wei and Hui Zhang and Jun-Hai Yong},
title     = {Interactive Image Segmentation with Temporal Information Augmented},
booktitle = {35th British Machine Vision Conference 2024, {BMVC} 2024, Glasgow, UK, November 25-28, 2024},
publisher = {BMVA},
year      = {2024},
url       = {https://papers.bmvc2024.org/0101.pdf}
}


Copyright © 2024 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection