Image and Video Coding

Recent works on image and video coding (especially deep-based)

This list is maintained by:

Sifeng Xia, Yueyue Hu, Wenhan Yang, Dezhao Wang, Yuzhang Hu

[STRUCT] PKU (PI: Prof. Jiaying Liu)

Image Compression

Prediction

• CNN-based Prediction
  • CNN-based Prediction for Lossless Coding of Photographic Images (PCS 2018), Ionut Schiopu, Yu Liu, Adrian Munteanu.

Auto-Encoder

• Deep Convolutional AutoEncoder [Web] [PDF]
  • Deep Convolutional AutoEncoder-based Lossy Image Compression (PCS 2018), Zhengxue Cheng, Heming Sun, Masaru Takeuchi, Jiro Katto.
• Quantization Independent [Web] [PDF]
  • Autoencoder Based Image Compression: Can The Learning Be Quantization Independent? (ICASSP 2018), Thierry Dumas, Aline Roumy, Christine Guillemot.
• Compressive Autoencoders [Web] [PDF]
  • Lossy Image Compression with Compressive Autoencoders (ICLR 2017), Lucas Theis, Wenzhe Shi, Andrew Cunningham, Ferenc Huszár.
• End-to-end Compression [Web] [PDF]
  • End-to-end Optimized Image Compression (ICLR 2017), Ballé, Laparra & Simoncelli.
• CAS-CNN [Web] [PDF]
  • CAS-CNN: A Deep Convolutional Neural Network for Image Compression Artifact Suppression (Arxiv 2016), Lukas Cavigelli, Pascal Hager, Luca Benini.
• Auto-encoders compression [Web] [PDF]
  • Auto-encoders: reconstruction versus compression (Arxiv 2014), Yann Ollivier.
• Lightweight Lossy Compression [Web] [PDF]
  • Lightweight Lossy Compression of Biometric Patterns via Denoising Autoencoders (IEEE SPL 2015), Davide Del Testa, Michele Rossi.
• Entropy Control and Auto-encoders [Web] [PDF]
  • Autoencoder based image compression: can the learning be quantization independent? (ICASSP 2018), Thierry Dumas, Aline, Roumy, Christine Guillemot.
• End-to-end optimization of nonlinear transform codes [Web]
  • End-to-end optimization of nonlinear transform codes for perceptual quality (PCS 2016), J. Ballé, V. Laparra, E. P. Simoncelli.
• Autoencoder beyond pixels [Web] [PDF]
  • Autoencoding beyond pixels using a learned similarity metric (ICML 2016),A. B. L. Larsen, S. K. Sønderby, H. Larochelle, O. Winther.
• Simultaneous Compression and retrieval
  • Deep network-based image coding for simultaneous compression and retrieval (ICIP 2017), Q. Zhang, D. Liu, H. Li.
• Inpainting-based Coding [Web] [PDF]
  • Learning to inpaint for image compression (NIPS 2017), M. H. Baig, V. Koltun, L. Torresani.

Colorization

• Colorization-based Coding [Web]
  • Multiple hypothesis colorization and its application to image compression (CVIU 2017), M. H. Baig, L. Torresani.

Adaptive Bit Rates

• Recurrent Neural Networks [Web] [PDF]
  • Full Resolution Image Compression with Recurrent Neural Networks (CVPR 2017), George Toderici, Damien Vincent, Nick Johnston, Sung Jin Hwang, David Minnen, Joel Shor, Michele Covell.
• Variable Rate Image Compression [Web] [PDF]
  • Variable Rate Image Compression with Recurrent Neural Networks (ICLR 2016), George Toderici, Sean M. O'Malley, Sung Jin Hwang, Damien Vincent, David Minnen, Shumeet Baluja, Michele Covell, Rahul Sukthankar.
• Priming and Spatially Adaptive Bit Rates Compression[Web] [PDF]
  • Improved Lossy Image Compression with Priming and Spatially Adaptive Bit Rates for Recurrent Networks (Arxiv 2017), Nick Johnston, Damien Vincent, David Minnen, Michele Covell, Saurabh Singh, Troy Chinen, Sung Jin Hwang, Joel Shor, George Toderici.
• Content-weighted Image Compression[Web] [PDF]
  • Learning Convolutional Networks for Content-weighted Image Compression (Arxiv 2017), Mu Li, Wangmeng Zuo, Shuhang Gu, Debin Zhao, David Zhang.
• Real-time adaptive compression [Web] [PDF]
  • Real-time adaptive image compression (ICML 2017), O. Rippel, L. Bourdev.

Generative Model

• Generative Compression [Web] [PDF]
  • Generative Compression (PCS 2018), Shibani Santurkar, David Budden, Nir Shavit.
• Semantic Perceptual Image Compression [Web] [PDF]
  • Semantic Perceptual Image Compression (Arxiv 2016), A Prakash, N Moran, S Garber, A DiLillo, J Storer.
• Conceptual Compression [Web] [PDF]
  • Towards Conceptual Compression (NIPS 2016), Karol Gregor, Frederic Besse, Danilo Jimenez Rezende, Ivo Danihelka, Daan Wierstra.
• DNN-based Perceputual Similarity Metrics [Web] [PDF]
  • Generating images with perceptual similarity metrics based on deep networks. (NIPS 2016), A. Dosovitskiy, T. Brox.
• Perceputual Similarity Metrics [Web] [PDF]
  • Learning to generate images with perceptual similarity metrics (Arxiv 2017), J. Snell, K. Ridgeway, R. Liao, B. D. Roads, M. C. Mozer, R. S. Zemel.
• Spatial LSTMs [Web] [PDF]
  • Generative image modeling using spatial LSTMs (NIPS 2015), L. Theis, M. Bethge.
• Pixel RNN [Web] [PDF]
  • Pixel recurrent neural networks (ICML 2016), A. van den Oord, N. Kalchbrenner, K. Kavukcuoglu.
• Conditional image generation [Web] [PDF]
  • Conditional image generation with PixelCNN decoders (NIPS 2016), A. van den Oord, N. Kalchbrenner, O. Vinyals, L. Espeholt, A. Graves, K. Kavukcuoglu.

Transform

• Nonlinear Transform Codes [Web] [PDF]
  • End-to-end optimization of nonlinear transform codes for perceptual quality (PCS 2016), Johannes Ballé, Valero Laparra, Eero P. Simoncelli.
• Soft-to-Hard Vector Quantization [Web] [PDF]
  • Soft-to-Hard Vector Quantization for End-to-End Learned Compression of mages and Neural Networks (Arxiv 2017), Eirikur Agustsson, Fabian Mentzer, Michael Tschannen, Lukas Cavigelli, Radu Timofte, Luca Benini, Luc Van Gool.
• DC Coefficient Estimation [PDF]
  • DC Coefficient Estimation of Intra-Predicted Residuals in HEVC (TCSVT 2017), Chen Chen, Zexiang Miao, Xiandong Meng, Shuyuan Zhu, Bing Zeng.
• CNN-based DCT-like transform [Web]
  • CNN-based DCT-like transform for image compression (MMM 2018), D. Liu, H. Ma, Z. Xiong, F. Wu.

Post-Processing

• Codes + CNN [PDF]
  • An End-to-End Compression Framework Based on Convolutional Neural Networks (TCSVT 2017), Feng Jiang, Wen Tao, Shaohui Liu, Jie Ren, Xun Guo, Debin Zhao.
• Image Decoding via Edge-Preserving GAN
  • Enhanced Image Decoding via Edge-Preserving Generative Adversarial Network (ICME 2018), Qi Mao, Shiqi Wang, Shanshe Wang, Xinfeng Zhang, Siwei Ma.

Video Coding

Loop Filter and Post-Processing

• Deep HEVC CU Split Decision
  • Fully Connected Network for HEVC CU Split Decision equipped with Laplacian Transparent Composite Model (PCS 2018), Hossam Amer, Abdullah Rashwan, En-hui Yang.
• Deep In-Loop Filtering
  • Deep Learning Based HEVC In-Loop Filtering For Decoder Quality Enhancement (PCS 2018), Shiba Kuanar, Christopher Conly, Kamisetty Rao.
• CNN for HEVC Intra Coding [Web]
  • A convolutional neural network approach for post-processing in HEVC intra coding (MMM 2017), Y. Dai, D. Liu, F. Wu.
• CNN-based In-loop Filtering [Web]
  • CNN-based in-loop filtering for coding efficiency improvement (IVMSP 2016), W.-S. Park, M. Kim.
• Spatial-Temporal Residue Network Based In-Loop Filter [Web]
  • Spatial-Temporal Residue Network Based In-Loop Filter for Video Coding (VCIP 2017), Chuanmin Jia, Shiqi Wang, Xinfeng Zhang, Shanshe Wang, Siwei Ma.
• In-Loop Filter Based on Multi-channel Long-Short-term Dependency Residual Networks [Web]
  • A New HEVC In-Loop Filter Based on Multi-channel Long-Short-term Dependency Residual Networks (DCC 2018), Xiandong Meng, Chen Chen, Shuyuan Zhu, and Bing Zeng.
• Multi-Modal/Multi-Scale Convolutional Neural Network [Web]
  • Multi-Modal/Multi-Scale Convolutional Neural Network Based In-Loop Filter Design For Next Generation Video Codec (ICIP 2017), Jihong Kang, Sungjei Kim, Kyoung Mu Lee.
• Residual Highway Convolutional Neural Networks [Web]
  • Residual Highway Convolutional Neural Networks for in-loop Filtering in HEVC (TIP 2018), Yongbing Zhang, Tao Shen, Xiangyang Ji, Yun Zhang, Ruiqin Xiong, and Qionghai Dai.
• Decoder-Side Hevc Quality Enhancement [Web]
  • Decoder-Side Hevc Quality Enhancement With Scalable Convolutional Neural Network (ICME 2017), Ren Yang, Mai Xu and Zulin Wang.
• Enhancing Quality for HEVC Compressed Videos [Web]
  • Enhancing Quality for HEVC Compressed Videos (Arxiv 2017), Ren Yang, Mai Xu, Tie Liu, Zulin Wang, and Zhenyu Guan.
• Deep Learning-Based Method of Improving Coding Efficiency [Web]
  • A Novel Deep Learning-Based Method of Improving Coding Efficiency from the Decoder-end for HEVC (DCC 2017), Tingting Wang, Mingjin Chen and Hongyang Chao.
• Cnn Based Post-Processing [Web]
  • Cnn Based Post-Processing To Improve Hevc (ICIP 2017), Chen Li, Li Song, Rong Xie and Wenjun Zhang.
• Multi-Scale Deep Decoder [Web]
  • The Multi-Scale Deep Decoder for the Standard HEVC Bitstreams (DCC 2018), Tingting Wang, Wenhui Xiao, Minglin Chen and Hongyang Chao.
• Multi-Frame Quality Enhancement [Web]
  • Multi-Frame Quality Enhancement for Compressed Video (CVPR 2018), Ren Yang, Mai Xu, Zulin Wang and Tianyi Li.
• Convolutional Neural Network As Loop Filter For Intra Frame [Web]
  • A Practical Convolutional Neural Network As Loop Filter For Intra Frame (ICIP 2018), Xiaodan Song, Jiabao Yao, Lulu Zhou, Li Wang, Xiaoyang Wu, Di Xie and Shiliang Pu.
• Enhancing Hevc Compressed Videos With Convolutional Neural Network [Web]
  • Enhancing Hevc Compressed Videos With A Partition-Masked Convolutional Neural Network (ICIP 2018), Xiaoyi He, Qiang Hu, Xintong Han, Xia.

Intra-Prediction Related

• Progressive Spatial Recurrent Neural Network for Intra Prediction[Web] [PDF]
  • Progressive Spatial Recurrent Neural Network for Intra Prediction (Arxiv 2018), Yueyu Hu, Wenhan Yang, Mading Li, Jiaying Liu.
• Fully Connected Network [PDF]
  • Intra prediction using fully connected network for video coding (ICIP 2017), Jiahao Li, Bin Li, Jizheng Xu, Ruiqin Xiong.
• Convolutional Neural Networks [PDF]
  • Convolutional Neural Networks based Intra Prediction for HEVC (Arxiv 2018), Wenxue Cui, Tao Zhang, Shengping Zhang, Feng Jiang, Wangmeng Zuo, Debin Zhao.
• Entropy coding [Web] [PDF]
  • Neural network-based arithmetic coding of intra prediction modes in HEVC (VCIP 2017), R. Song, D. Liu, H. Li, F. Wu.
• Convolutional Neural Network Approach for Post-Processing in HEVC Intra Coding [Web] [PDF]
  • A Convolutional Neural Network Approach for Post-Processing in HEVC Intra Coding (MMM 2017), Yuanying Dai, Dong Liu, and Feng Wu.

Inter-Prediction Related

• Neural Network Based Inter Prediction for HEVC
  • Neural Network Based Inter Prediction for HEVC (ICME 2018), Yang Wang, Xiaopeng Fan, Chuanmin Jia, Debin Zhao, Wen Gao.
• CNN Motion Compensation Refinement[Web] [PDF]
  • Convolutional Neural Network-Based Motion Compensation Refinement for Video Coding (ISCAS 2018), Zhenghui Zhao, Shiqi Wang, Shanshe Wang, Xinfeng Zhang, Siwei Ma and Jiansheng Yang.
• CNN-Based Bi-Directional Motion Compensation[Web] [PDF]
  • CNN-Based Bi-Directional Motion Compensation for High Efficiency Video Coding (ISCAS 2018), Zhenghui Zhao, Shiqi Wang, Shanshe Wang, Xinfeng Zhang, Siwei Ma and Jiansheng Yang.
• Group Variational Transformer for Fractional Interpolation[Web] [PDF]
  • A Group Variational Transformation Neural Network for Fractional Interpolation of Video Coding (DCC 2018), Sifeng Xia, Wenhan Yang, Yueyu Hu, Siwei Ma, Jiaying Liu.
• CNN for Half-Pel Interpolation[Web] [PDF]
  • A Convolutional Neural Network Approach for Half-Pel Interpolation in Video Coding (ISCAS 2017), Ning Yan, Dong Liu, Houqiang Li, and Feng Wu.

Rate Control

• RL for Rate Control[Web] [PDF]
  • Reinforcement Learning for HEVC/H.265 Intra-Frame Rate Control (ISCAS 2018), Jun-Hao Hu, Wen-Hsiao Peng, and Chia-Hua Chung.
• CNN-based Approach
  • A convolutional neural network-based approach to rate control in HEVC intra coding (VCIP 2017), Y. Li, B. Li, D. Liu, Z. Chen.

Transform

• Variable Block-Sized Signal Dependent Transform [PDF]
  • Variable Block-Sized Signal Dependent Transform for Video Coding (TCSVT 2017), Cuiling Lan, Jizheng Xu, Wenjun Zeng, Guangming Shi, Feng Wu.

Coding + Image Processing

• CNN-Based Block Up-sampling[Web] [PDF]
  • Convolutional Neural Network-Based Block Up-sampling for Intra Frame Coding (TCSVT 2017), Yue Li, Dong Liu, Houqiang Li, Li Li, and Feng Wu.
• Image interpolation [Web] [PDF]
  • Video compression through image interpolation (Arxiv 2018), C.-Y. Wu, N. Singhal, P. Krähenbühl.

Auto-encoder

• Learning Binary Residual Representations [Web] [PDF]
  • Learning Binary Residual Representations for Domain-specific Video Streaming (AAAI 2018), Y.-H. Tsai, M.-Y. Liu, D. Sun, M.-H. Yang, J. Kautz.

Mode Decision

• CU partition [Web]
  • CU partition mode decision for HEVC hardwired intra encoder using convolution neural network (TIP 2016), Z. Liu, X. Yu, Y. Gao, S. Chen, X. Ji, D. Wan.