Predictive Coding For Animation-Based Video Compression (2307.04187v1)
Abstract: We address the problem of efficiently compressing video for conferencing-type applications. We build on recent approaches based on image animation, which can achieve good reconstruction quality at very low bitrate by representing face motions with a compact set of sparse keypoints. However, these methods encode video in a frame-by-frame fashion, i.e. each frame is reconstructed from a reference frame, which limits the reconstruction quality when the bandwidth is larger. Instead, we propose a predictive coding scheme which uses image animation as a predictor, and codes the residual with respect to the actual target frame. The residuals can be in turn coded in a predictive manner, thus removing efficiently temporal dependencies. Our experiments indicate a significant bitrate gain, in excess of 70% compared to the HEVC video standard and over 30% compared to VVC, on a datasetof talking-head videos
- “Ultra-low bitrate video conferencing using deep image animation,” in IEEE ICASSP, 2020.
- “A hybrid animation codec for low bitrate video conferencing,” in IEEE ICIP, 2022.
- “One-shot free-view neural talking-head synthesis for video conferencing,” in CVPR, 2021.
- “Low bandwidth video-chat compression using deep generative models,” arXiv preprint arXiv:2012.00328, 2020.
- “Beyond keypoint coding: Temporal evolution inference with compact feature representation for talking face video compression,” in DCC, 2022.
- “Robust ultralow bitrate video conferencing with second order motion coherency,” in 2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP), 2022, pp. 1–6.
- “First order motion model for image animation,” in Neurips, 2019.
- “Real-time adaptive image compression,” arXiv preprint arXiv:1705.05823, 2017.
- “Variational image compression with a scale hyperprior,” arXiv preprint arXiv:1802.01436, 2018.
- “Generative adversarial networks for extreme learned image compression,” in IEEE ICCV, 2019.
- “Fvc: A new frame- work towards deep video compression in feature space,” in IEEE Conference on Computer Vision and Pattern Recognition(CVPR), 2021.
- “End-to-end optimized image compression,” Neurips, 2016.
- “Dvc: An end-to-end deep video compression framework,” in IEEE CVPR, 2019.
- “Vct: A video compression transformer,” in NeurIPS, 2022.
- J. Cleary and I. Witten, “Data compression using adaptive coding and partial string matching,” IEEE Transactions on Communications, vol. 32, no. 4, pp. 396–402, 1984.
- “The unreasonable effectiveness of deep features as a perceptual metric,” in CVPR, 2018.
- “Very deep convolutional networks for large scale image recognition,” in ICLR, 2015.
- “Image quality assessment: Unifying structure and texture similarity,” in IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, vol. 44, pp. 2567–2581.
Collections
Sign up for free to add this paper to one or more collections.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
