Publication | Chen Change Loy

Low-Level Vision

Let's enhance! Our team has been working on various image/video restoration and enhancement problems such as super-resolution, denoising, low-light enhancement etc. Some notable methods developed by us include SRCNN, ESRGAN, EDVR, BasicVSR, and Zero-DCE.

Super-Resolution

Arbitrary-steps Image Super-resolution via Diffusion Inversion
Z. Yue, K. Liao, C. C. Loy
Technical report, arXiv:2412.09013, 2024
[arXiv] [Project Page] [Demo]
Efficient Diffusion Model for Image Restoration by Residual Shifting
Z. Yue, J. Wang, C. C. Loy
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024 (TPAMI)
[arXiv] [Project Page]
DifFace: Blind Face Restoration with Diffused Error Contraction
Z. Yue, C. C. Loy
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024 (TPAMI)
[arXiv] [Project Page] [Demo]
Kalman-Inspired Feature Propagation for Video Face Super-Resolution
R. Feng, C. Li, C. C. Loy
European Conference on Computer Vision, 2024 (ECCV)
[arXiv] [Project Page]
Exploiting Diffusion Prior for Real-World Image Super-Resolution
J. Wang, Z. Yue, S. Zhou, K. C. K. Chan, C. C. Loy
International Journal of Computer Vision, 2024 (IJCV)
[PDF] [DOI] [arXiv] [Project Page]
Upscale-A-Video: Temporal-Consistent Diffusion Model for Real-World Video Super-Resolution
S. Zhou, P. Yang, J. Wang, Y. Luo, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2024 (CVPR, Highlight)
[PDF] [arXiv] [Supplementary Material] [Project Page] [YouTube]
PGDiff: Guiding Diffusion Models for Versatile Face Restoration via Partial Guidance
P. Yang, S. Zhou, Q. Tao, C. C. Loy
in Proceedings of Neural Information Processing Systems, 2023 (NeurIPS)
[PDF] [arXiv] [Project Page]
ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shifting
Z. Yue, J. Wang, C. C. Loy
in Proceedings of Neural Information Processing Systems, 2023 (NeurIPS, Spotlight)
[PDF] [arXiv] [Project Page]
Learning Generative Structure Prior for Blind Text Image Super-resolution
X. Li, W. Zuo, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2023 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Reference-based Image and Video Super-Resolution via C²-Matching
Y. Jiang, K. C. K. Chan, X. Wang, C. C. Loy, Z. Liu
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023 (TPAMI)
[DOI] [arXiv] [Project Page]
Towards Robust Blind Face Restoration with Codebook Lookup Transformer
S. Zhou, K. C. K. Chan, C. Li, C. C. Loy
in Proceedings of Neural Information Processing Systems, 2022 (NeurIPS)
[arXiv] [Project Page] [Demo]
GLEAN: Generative Latent Bank for Image Super-Resolution and Beyond
K. C. K. Chan, X. Wang, X. Xu, J. Gu, C. C. Loy
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022 (TPAMI)
[DOI] [arXiv] [Project Page]
Investigating Trade-offs in Real-World Video Super-Resolution
K. C. K. Chan, S. Zhou, X. Xu, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2022 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page]
BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment
K. C. K. Chan, S. Zhou, X. Xu, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2022 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page]
GLEAN: Generative Latent Bank for Large-Factor Image Super-Resolution
K. C. K. Chan, X. Wang, X. Xu, J. Gu, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2021 (CVPR, Oral)
[PDF] [arXiv] [Supplementary Material] [Project Page]
BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond
K. C. K. Chan, X. Wang, K. Yu, C. Dong, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2021 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Robust Reference-based Super-Resolution via C²-Matching
Y. Jiang, K. C. K. Chan, X. Wang, C. C. Loy, Z. Liu
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2021 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Understanding Deformable Alignment in Video Super-Resolution
K. C. K. Chan, X. Wang, K. Yu, C. Dong, C. C. Loy
in Proceedings of AAAI Conference on Artificial Intelligence, 2021 (AAAI)
[arXiv] [Project Page]
Cross-Scale Internal Graph Neural Network for Image Super-Resolution
S. Zhou, J. Zhang, W. Zuo, C. C. Loy
in Proceedings of Neural Information Processing Systems, 2020 (NeurIPS)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Non-Local Recurrent Network for Image Restoration
D. Liu, B. Wen, Y. Fan, C. C. Loy, T. S. Huang
in Proceedings of Neural Information Processing Systems, 2018 (NeurIPS)
[PDF] [arXiv] [Project Page]
ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks
X. Wang, K. Yu, S. Wu, J. Gu, Y. Liu, C. Dong, Y. Qiao, C. C. Loy
in Workshop Proceedings of European Conference on Computer Vision, 2018 (ECCVW)
[PDF] [arXiv] [Project Page]
Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform
X. Wang, K. Yu, C. Dong, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2018 (CVPR)
[PDF] [arXiv] [Project Page]
Deep Cascaded Bi-Network for Face Hallucination
S. Zhu, S. Liu, C. C. Loy, X. Tang
in Proceedings of European Conference on Computer Vision, 2016 (ECCV)
[PDF] [arXiv] [Project Page]
Accelerating the Super-Resolution Convolutional Neural Network
C. Dong, C. C. Loy, X. Tang
in Proceedings of European Conference on Computer Vision, 2016 (ECCV)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Depth Map Super Resolution by Deep Multi-Scale Guidance
T.-W. Hui, C. C. Loy, X. Tang
in Proceedings of European Conference on Computer Vision, 2016 (ECCV)
[PDF] [arXiv]
Image Super-Resolution Using Deep Convolutional Networks
C. Dong, C. C. Loy, K. He, X. Tang
IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 2, pp. 295-307, 2015 (TPAMI)
[DOI] [arXiv] [Supplementary Material] [Project Page]
Learning a Deep Convolutional Network for Image Super-Resolution
C. Dong, C. C. Loy, K. He, X. Tang
in Proceedings of European Conference on Computer Vision, pp 184-199, 2014 (ECCV)
[PDF] [Supplementary Material] [Project Page]
Boosting Optical Character Recognition: A Super-Resolution Approach
C. Dong, X. Zhu, Y. Deng, C. C. Loy, Y. Qiao
Technical report, arXiv:1506.02211, 2015
[arXiv]

Restoration | Enhancement

Paint Bucket Colorization Using Anime Character Color Design Sheets
Y. Dai, Q. Li, S. Zhou, Y. Luo, C. Li, C. C. Loy
Technical report, arXiv:2410.19424, 2024
[arXiv] [Project Page]
Generalizable Implicit Motion Modeling for Video Frame Interpolation
Z. Guo, W. Li, C. C. Loy
in Proceedings of Neural Information Processing Systems, 2024 (NeurIPS)
[arXiv] [Project Page]
Denoising as Adaptation: Noise-Space Domain Adaptation for Image Restoration
K. Liao, Z. Yue, Z. Wang, C. C. Loy
Technical report, arXiv:2406.18516, 2024
[arXiv] [Project Page]
MOWA: Multiple-in-One Image Warping Model
K. Liao, Z. Yue, Z. Wu, C. C. Loy
Technical report, arXiv:2404.10716, 2024
[arXiv] [Project Page]
Flare7K++: Mixing Synthetic and Real Datasets for Nighttime Flare Removal and Beyond
Y. Dai, C. Li, S. Zhou, R. Feng, Y. Luo, C. C. Loy
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024 (TPAMI)
[arXiv] [Project Page]
Learning Inclusion Matching for Animation Paint Bucket Colorization
Y. Dai, S. Zhou, Q. Li, C. Li, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2024 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page] [YouTube]
Towards Language-Driven Video Inpainting via Multimodal Large Language Models
J. Wu, X. Li, C. Si, S. Zhou, J. Yang, J. Zhang, Y. Li, K. Chen, Y. Tong, Z. Liu, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2024 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Adaptive Window Pruning for Efficient Local Motion Deblurring
H. Li, J. Zhao, S. Zhou, H. Feng, C. Li, C. C. Loy
International Conference on Learning Representations, 2024 (ICLR)
[arXiv] [Project Page]
Control Color: Multimodal Diffusion-based Interactive Image Colorization
Z. Liang, Z. Li, S. Zhou, C. Li, C. C. Loy
Technical report, arXiv:2402.10855, 2024
[arXiv] [Project Page]
Temporally Consistent Video Colorization with Deep Feature Propagation and Self-regularization Learning
Y. Liu, H. Zhao, K. C. K. Chan, X. Wang, C. C. Loy, Y. Qiao, C. Dong
Computational Visual Media , 2024 (CVM)
[DOI] [arXiv] [Project Page]
Rubik's Cube: High-Order Channel Interactions with a Hierarchical Receptive Field
N. Zheng, M. Zhou, C. Zhou, C. C. Loy
in Proceedings of Neural Information Processing Systems, 2023 (NeurIPS)
[PDF] [Project Page]
Video Infilling with Rich Motion Prior
X. Hou, L. Jiang, R. Shao, C. C. Loy
in Proceedings of British Machine Vision Conference, 2023 (BMVC)
[PDF] [Supplementary Material]
Iterative Prompt Learning for Unsupervised Backlit Image Enhancement
Z. Liang, C. Li, S. Zhou, R. Feng, C. C. Loy
in Proceedings of IEEE/CVF International Conference on Computer Vision, 2023 (ICCV, Oral)
[PDF] [arXiv] [Project Page]
ProPainter: Improving Propagation and Transformer for Video Inpainting
S. Zhou, C. Li, K. C. K. Chan, C. C. Loy
in Proceedings of IEEE/CVF International Conference on Computer Vision, 2023 (ICCV)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Deep Geometrized Cartoon Line Inbetweening
L. Siyao, T. Gu, W. Xiao, H. Ding, Z. Liu, C. C. Loy
in Proceedings of IEEE/CVF International Conference on Computer Vision, 2023 (ICCV)
[PDF] [Supplementary Material] [Project Page]
Nighttime Smartphone Reflective Flare Removal using Optical Center Symmetry Prior
Y. Dai, Y. Luo, S. Zhou, C. Li, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2023 (CVPR, Highlight)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Generating Aligned Pseudo-Supervision from Non-Aligned Data for Image Restoration in Under-Display Camera
R. Feng, C. Li, H. Chen, S. Li, J. Gu, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2023 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Flexible Piecewise Curves Estimation for Photo Enhancement
C. Li, C. Guo, Q. Ai, S. Zhou, C. C. Loy
in Workshop Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, NTIRE, 2023 (CVPRW)
[arXiv]
Embedding Fourier for Ultra-High-Definition Low-Light Image Enhancement
C. Li, C. Guo, M. Zhou, Z. Liang, S. Zhou, R. Feng, C. C. Loy
International Conference on Learning Representations, 2023 (ICLR, Oral)
[arXiv] [Project Page]
Exploring CLIP for Assessing the Look and Feel of Images
J. Wang, K. C. K. Chan, C. C. Loy
in Proceedings of AAAI Conference on Artificial Intelligence, 2023 (AAAI)
[arXiv] [Project Page]
AnimeRun: 2D Animation Visual Correspondence from Open Source 3D Movies
L. Siyao, Y. Li, B. Li, C. Dong, Z. Liu, C. C. Loy
in Proceedings of Neural Information Processing Systems (Datasets and Benchmarks Track), 2022 (NeurIPS)
[arXiv] [Project Page]
Flare7K: A Phenomenological Nighttime Flare Removal Dataset
Y. Dai, C. Li, S. Zhou, R. Feng, C. C. Loy
in Proceedings of Neural Information Processing Systems (Datasets and Benchmarks Track), 2022 (NeurIPS)
[arXiv] [Project Page]
Deep Fourier Up-Sampling
M. Zhou, H. Yu, J. Huang, F. Zhao, J. Gu, C. C. Loy, D. Meng, C. Li
in Proceedings of Neural Information Processing Systems, 2022 (NeurIPS)
[arXiv] [Project Page]
CuDi: Curve Distillation for Efficient and Controllable Exposure Adjustment
C. Li, C. Guo, R. Feng, S. Zhou, C. C. Loy
Technical report, arXiv:2207.14273, 2022
[arXiv] [Project Page]
LEDNet: Joint Low-light Enhancement and Deblurring in the Dark
S. Zhou, C. Li, C. C. Loy
European Conference on Computer Vision, 2022 (ECCV)
[PDF] [arXiv] [Supplementary Material] [Project Page]
On the Generalization of BasicVSR++ to Video Deblurring and Denoising
K. C. K. Chan, S. Zhou, X. Xu, C. C. Loy
Technical report, arXiv:2204.05308, 2022
[arXiv] [Project Page]
Low-Light Image and Video Enhancement Using Deep Learning: A Survey
C. Li, C. Guo, L. Han, J. Jiang, M. Cheng, J. Gu, C. C. Loy
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021 (TPAMI)
[DOI] [arXiv] [Project Page]
Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation
X. Pan, X. Zhan, B. Dai, D. Lin, C. C. Loy, P. Luo
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021 (TPAMI)
[DOI] [Project Page]
ReconfigISP: Reconfigurable Camera Image Processing Pipeline
K. Yu, Z. Li, Y. Peng, C. C. Loy, J. Gu
in Proceedings of IEEE/CVF International Conference on Computer Vision, 2021 (ICCV)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Path-Restore: Learning Network Path Selection for Image Restoration
K. Yu, X. Wang, C. Dong, X. Tang, C. C. Loy
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021 (TPAMI)
[DOI] [arXiv] [Project Page]
Learning to Enhance Low-Light Image via Zero-Reference Deep Curve Estimation
C. Li, C. Guo, C. C. Loy
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021 (TPAMI)
[DOI] [arXiv] [Project Page]
Removing Diffraction Image Artifacts in Under-Display Camera via Dynamic Skip Connection Network
R. Feng, C. Li, H. Chen, S. Li, C. C. Loy, J. Gu
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2021 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Deep Animation Video Interpolation in the Wild
L. Siyao, S. Zhao, W. Yu, W. Sun, D. Metaxas, C. C. Loy, Z. Liu
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2021 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation
X. Pan, X. Zhan, B. Dai, D. Lin, C. C. Loy, P. Luo
European Conference on Computer Vision, 2020 (ECCV, Oral)
[PDF] [arXiv] [Project Page]
Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement
C. Guo, C. Li, J. Guo, C. C. Loy, J. Hou, S. Kwong, R. Cong
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2020 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page]
EDVR: Video Restoration with Enhanced Deformable Convolutional Networks
X. Wang, C. K. Chan, K. Yu, C. Dong, X. Tang, C. C. Loy
in Workshop Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, NTIRE, 2019 (CVPRW)
[PDF] [arXiv] [Project Page]
Deep Network Interpolation for Continuous Imagery Effect Transition
X. Wang, K. Yu, C. Dong, X. Tang, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2019 (CVPR)
[PDF] [arXiv] [Project Page]
Aesthetic-Driven Image Enhancement by Adversarial Learning
Y. Deng, C. C. Loy, X. Tang
ACM Multimedia, 2018 (ACM MM)
[PDF] [arXiv] [Project Page]
Crafting a Toolchain for Image Restoration by Deep Reinforcement Learning
K. Yu, C. Dong, L. Lin, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2018 (CVPR, Spotlight)
[PDF] [arXiv] [Project Page]
Image Aesthetic Assessment: An Experimental Survey
Y. Deng, C. C. Loy, X. Tang
IEEE Signal Processing Magazine, vol. 34, no. 4, pp. 80–106, 2017 (SPM)
[DOI] [arXiv]
Deep Specialized Network for Illuminant Estimation
W. Shi, C. C. Loy, X. Tang
in Proceedings of European Conference on Computer Vision, 2016 (ECCV)
[PDF] [Project Page]
Compression Artifacts Reduction by a Deep Convolutional Network
C. Dong, Y. Deng, C. C. Loy, X. Tang
in Proceedings of International Conference on Computer Vision, pp. 576-584, 2015 (ICCV)
[PDF] [arXiv] [Project Page]

Optical Flow Estimation

LiteFlowNet3: Resolving Correspondence Ambiguity for More Accurate Optical Flow Estimation
T.-W. Hui, C. C. Loy
European Conference on Computer Vision, 2020 (ECCV)
[PDF] [arXiv] [Supplementary Material] [Project Page]
A Lightweight Optical Flow CNN - Revisiting Data Fidelity and Regularization
T.-W. Hui, X. Tang, C. C. Loy
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020 (TPAMI)
[DOI] [arXiv] [Project Page]
LiteFlowNet: A Lightweight Convolutional Neural Network for Optical Flow Estimation
T.-W. Hui, X. Tang, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2018 (CVPR, Spotlight)
[PDF] [arXiv] [Project Page] [YouTube]

Editing and Generation

We like algorithms that could generate new visual contents, e.g., face generation, face reenactment, image inpainting, scene de-occlusion, etc.

Face Editing

Talk-to-Edit: Fine-Grained 2D and 3D Facial Editing via Dialog
Y. Jiang, Z. Huang, T. Wu, X. Pan, C. C. Loy, Z. Liu
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024 (TPAMI)
[DOI] [Project Page]
StyleGANEX: StyleGAN-Based Manipulation Beyond Cropped Aligned Faces
S. Yang, L. Jiang, Z. Liu, C. C. Loy
in Proceedings of IEEE/CVF International Conference on Computer Vision, 2023 (ICCV)
[PDF] [arXiv] [Supplementary Material] [Project Page] [Demo]
CelebV-Text: A Large-Scale Facial Text-Video Dataset
J. Yu, H. Zhu, L. Jiang, C. C. Loy, W. Cai, W. Wu
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2023 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page]
BeautyREC: Robust, Efficient, and Content-preserving Makeup Transfer
Q. Yan, C. Guo, J. Zhao, Y. Dai, C. C. Loy, C. Li
in Workshop Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, NTIRE, 2023 (CVPRW)
[arXiv] [Project Page]
VToonify: Controllable High-Resolution Portrait Video Style Transfer
S. Yang, L. Jiang, Z. Liu, C. C. Loy
ACM Transactions on Graphics, 2022 (SIGGRAPH ASIA - TOG)
[DOI] [arXiv] [Project Page] [YouTube]
Audio-driven Dubbing for User Generated Contents via Style-aware Semi-parametric Synthesis
L. Song, W. Wu, C. Fu, C. C. Loy, R. He
IEEE Transactions on Circuits and Systems for Video Technology, 2022 (TCVST)
[DOI] [arXiv]
CelebV-HQ: A Large-Scale Video Facial Attributes Dataset
H. Zhu, W. Wu, W. Zhu, L. Jiang, S. Tang, L. Zhang, Z. Liu, C. C. Loy
European Conference on Computer Vision, 2022 (ECCV)
[PDF] [arXiv] [Supplementary Material] [Project Page]
TransEditor: Transformer-Based Dual-Space GAN for Highly Controllable Facial Editing
Y. Xu, Y. Yin, L. Jiang, Q. Wu, C. Zheng, C. C. Loy, B. Dai, W. Wu
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2022 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Talking Faces: Audio-to-Video Face Generation
Y. Wang, L. Song, W. Wu, C. Qian, R. He, C. C. Loy
In C. Rathgeb, R. Tolosana, R. Vera-Rodriguez, C. Busch (Eds.), Handbook of Digital Face Manipulation and Detection, Springer, 2022
[Book Link]
Everybody’s Talkin’: Let Me Talk as You Want
L. Song, W. Wu, C. Qian, R. He, C. C. Loy
IEEE Transactions on Information Forensics and Security, 2022 (TIFS)
[DOI] [arXiv] [Project Page]
Talk-to-Edit: Fine-Grained Facial Editing via Dialog
Y. Jiang, Z. Huang, X. Pan, C. C. Loy, Z. Liu
in Proceedings of IEEE/CVF International Conference on Computer Vision, 2021 (ICCV)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation
H. Zhou, Y. Sun, W. Wu, C. C. Loy, X. Wang, Z. Liu
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2021 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Pareidolia Face Reenactment
L. Song, W. Wu, C. Fu, C. Qian, C. C. Loy, R. He
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2021 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page] [YouTube]
Audio-Driven Emotional Video Portraits
X. Ji, H. Zhou, K. Wang, W. Wu, X. Cao, C. C. Loy, F. Xu
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2021 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page]
MEAD: A Large-scale Audio-visual Dataset for Emotional Talking Face Generation
K. Wang, Q. Wu, L. Song, Z. Yang, W. Wu, C. Qian, R. He, Y. Qiao, C. C. Loy
European Conference on Computer Vision, 2020 (ECCV)
[PDF] [Supplementary Material] [Project Page]
One-shot Face Reenactment
Y. Zhang, S. Zhang, Y. He, C. Li, C. C. Loy, Z. Liu
in Proceedings of British Machine Vision Conference, 2019 (BMVC, Spotlight)
[PDF] [arXiv] [Project Page]
Instance-level Facial Attributes Transfer with Geometry-aware Flow
W. Yin, Z. Liu, C. C. Loy
in Proceedings of AAAI Conference on Artificial Intelligence, 2019 (AAAI, Spotlight)
[PDF] [arXiv] [Project Page]
ReenactGAN: Learning to Reenact Faces via Boundary Transfer
W. Wu, Y. Zhang, C. Li, C. Qian, C. C. Loy
in Proceedings of European Conference on Computer Vision, 2018 (ECCV)
[PDF] [arXiv] [Project Page] [YouTube]

Image and Video Generation

ObjCtrl-2.5D: Training-free Object Control with Camera Poses
Z. Wang, Y. Lan, S. Zhou, C. C. Loy
Technical report, arXiv:2412.07721, 2024
[arXiv] [Project Page] [Demo]
Omegance: A Single Parameter for Various Granularities in Diffusion-Based Synthesis
X. Hou, Z. Yue, X. Li, C. C. Loy
Technical report, arXiv:2411.17769, 2024
[arXiv] [Project Page]
GroupDiff: Diffusion-based Group Portrait Editing
Y. Jiang, N. Zhao, Q. Liu, K. K. Singh, S. Yang, C. C. Loy, Z. Liu
European Conference on Computer Vision, 2024 (ECCV)
[arXiv] [Project Page]
AITTI: Learning Adaptive Inclusive Token for Text-to-Image Generation
X. Hou, X. Li, C. C. Loy
Technical report, arXiv:2406.12805, 2024
[arXiv] [Project Page]
VideoBooth: Diffusion-based Video Generation with Image Prompts
Y. Jiang, T. Wu, S. Yang, C. Si, D. Lin, Y. Qiao, C. C. Loy, Z. Liu
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2024 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page] [YouTube]
FRESCO: Spatial-Temporal Correspondence for Zero-Shot Video Translation
S. Yang, Y. Zhou, Z. Liu, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2024 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page]
When StyleGAN Meets Stable Diffusion: a W+ Adapter for Personalized Image Generation
X. Li, X. Hou, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2024 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page]
LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Models
Y. Wang, X. Chen, X. Ma, S. Zhou, Z. Huang, Y. Wang, C. Yang, Y. He, J. Yu, P. Yang, Y. Guo, T. Wu, C. Si, Y. Jiang, C. Chen, C. C. Loy, B. Dai, D. Lin, Y. Qiao, Z. Liu
International Journal of Computer Vision, 2024 (IJCV)
[arXiv] [Project Page]
Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation
S. Yang, Y. Zhou, Z. Liu, C. C. Loy
SIGGRAPH ASIA, 2023 (SIGGRAPH ASIA)
[arXiv] [Project Page]
Bailando++: 3D Dance GPT with Choreographic Memory
L. Siyao. W. Yu, T. Gu, C. Lin, Q. Wang, C. Qian, C. C. Loy, Z. Liu
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023 (TPAMI)
[DOI] [Project Page]
GP-UNIT: Generative Prior for Versatile Unsupervised Image-to-Image Translation
S. Yang, L. Jiang, Z. Liu, C. C. Loy
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023 (TPAMI)
[DOI] [arXiv] [Project Page]
Text2Performer: Text-Driven Human Video Generation
Y. Jiang, S. Yang, T. L. Koh, W. Wu, C. C. Loy, Z. Liu
in Proceedings of IEEE/CVF International Conference on Computer Vision, 2023 (ICCV)
[PDF] [arXiv] [Supplementary Material] [Project Page]
StyleInV: A Temporal Style Modulated Inversion Network for Unconditional Video Generation
Y. Wang, L. Jiang, C. C. Loy
in Proceedings of IEEE/CVF International Conference on Computer Vision, 2023 (ICCV)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Scenimefy: Learning to Craft Anime Scene via Semi-Supervised Image-to-Image Translation
Y. Jiang, L. Jiang, S. Yang, C. C. Loy
in Proceedings of IEEE/CVF International Conference on Computer Vision, 2023 (ICCV)
[PDF] [arXiv] [Supplementary Material] [Project Page] [Demo]
The Nuts and Bolts of Adopting Transformer in GANs
R. Xu and X. Xu and K. Chen and B. Zhou and C. C. Loy
in Workshop Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, AI4CC, 2023 (CVPRW)
[arXiv] [Project Page]
BRACE: The Breakdancing Competition Dataset for Dance Motion Synthesis
D. Moltisanti, J. Wu, B. Dai, C. C. Loy
European Conference on Computer Vision, 2022 (ECCV)
[PDF] [arXiv] [Supplementary Material] [Project Page]
StyleLight: HDR Panorama Generation for Lighting Estimation and Editing
G. Wang, Y. Yang, C. C. Loy, Z. Liu
European Conference on Computer Vision, 2022 (ECCV)
[PDF] [arXiv] [Supplementary Material] [Project Page]
StyleGAN-Human: A Data-Centric Odyssey of Human Generation
J. Fu, S. Li, Y. Jiang, K.-Y. Lin, C. Qian, C. C. Loy, W. Wu, Z. Liu
European Conference on Computer Vision, 2022 (ECCV)
[PDF] [arXiv] [Supplementary Material] [Project Page] [YouTube]
Text2Human: Text-Driven Controllable Human Image Generation
Y. Jiang, S. Yang, H. Qiu, W. Wu, C. C. Loy, Z. Liu
ACM Transactions on Graphics, 2022 (SIGGRAPH - TOG)
[DOI] [arXiv] [Project Page] [YouTube]
Pastiche Master: Exemplar-Based High-Resolution Portrait Style Transfer
S. Yang, L. Jiang, Z. Liu, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2022 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page] [YouTube]
Unsupervised Image-to-Image Translation with Generative Prior
S. Yang, L. Jiang, Z. Liu, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2022 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Bailando: 3D Dance Generation by Actor-Critic GPT with Choreographic Memory
L. Siyao, W. Yu, T. Gu, C. Lin, Q. Wang, C. Qian, C. C. Loy, Z. Liu
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2022 (CVPR, Oral)
[PDF] [arXiv] [Supplementary Material] [Project Page] [YouTube]
MoCaNet: Motion Retargeting in-the-wild via Canonicalization Networks
W. Zhu, Z. Yang, Z. Di, W. Wu, Y. Wang, C. C. Loy
in Proceedings of AAAI Conference on Artificial Intelligence, 2022 (AAAI)
[arXiv] [Project Page]
Deceive D: Adaptive Pseudo Augmentation for GAN Training with Limited Data
L. Jiang, B. Dai, W. Wu, C. C. Loy
in Proceedings of Neural Information Processing Systems, 2021 (NeurIPS)
[PDF] [arXiv] [Project Page] [YouTube]
Focal Frequency Loss for Image Reconstruction and Synthesis
L. Jiang, B. Dai, W. Wu, C. C. Loy
in Proceedings of IEEE/CVF International Conference on Computer Vision, 2021 (ICCV)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Positional Encoding as Spatial Inductive Bias in GANs
R. Xu, X. Wang, K. Chen, B. Zhou, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2021 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page] [YouTube]
Do 2D GANs Know 3D Shape? Unsupervised 3D Shape Reconstruction from 2D Image GANs
X. Pan, B. Dai, Z. Liu, C. C. Loy, P. Luo
International Conference on Learning Representations, 2021 (ICLR, Oral)
[PDF] [arXiv] [Project Page]
Texture Memory-Augmented Deep Patch-Based Image Inpainting
R. Xu, M. Guo, J. Wang, X. Li, B. Zhou, C. C. Loy
IEEE Transactions on Image Processing, 2021 (TIP)
[DOI] [arXiv]
Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation
X. Pan, X. Zhan, B. Dai, D. Lin, C. C. Loy, P. Luo
European Conference on Computer Vision, 2020 (ECCV, Oral)
[PDF] [arXiv] [Project Page]
TSIT: A Simple and Versatile Framework for Image-to-Image Translation
L. Jiang, C. Zhang, M. Huang, C. Liu, J. Shi, C. C. Loy
European Conference on Computer Vision, 2020 (ECCV, Spotlight)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Self-Supervised Scene De-occlusion
X. Zhan, X. Pan, B. Dai, Z. Liu, D. Lin, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2020 (CVPR, Oral)
[PDF] [arXiv] [Supplementary Material] [Project Page] [YouTube]
TransMoMo: Invariance-Driven Unsupervised Video Motion Retargeting
Z. Yang, W. Zhu, W. Wu, C. Qian, Q. Zhou, B. Zhou, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2020 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Real or Not Real, That is the Question
Y. Xiangli, Y. Deng, B. Dai, C. C. Loy, D. Lin
International Conference on Learning Representations, 2020 (ICLR, Spotlight)
[PDF] [Project Page]
High-Quality Video Generation from Static Structural Annotations
L. Sheng, J. Pan, J. Guo, J. Shao, C. C. Loy
International Journal of Computer Vision, 2020 (IJCV)
[DOI]
TransGaGa: Geometry-Aware Unsupervised Image-to-Image Translation
W. Wu, K. Cao, C. Li, C. Qian, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2019 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Deep Flow-Guided Video Inpainting
R. Xu, X. Li, B. Zhou, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2019 (CVPR)
[PDF] [arXiv] [Project Page]
Dense Intrinsic Appearance Flow for Human Pose Transfer
Y. Li, C. Huang, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2019 (CVPR)
[PDF] [Supplementary Material] [Project Page]
Disentangling Content and Style via Unsupervised Geometry Distillation
W. Wu, K. Cao, C. Li, C. Qian, C. C. Loy
International Conference on Learning Representations Workshop, 2019 (ICLRW)
[PDF]
Be Your Own Prada: Fashion Synthesis with Structural Coherence
S. Zhu, S. Fidler, R. Urtasun, D. Lin, C. C. Loy
in Proceedings of International Conference on Computer Vision, 2017 (ICCV)
[PDF] [Project Page]

Simulation

Learning 3D Garment Animation from Trajectories of A Piece of Cloth
Y. Shao, C. C. Loy, B. Dai
in Proceedings of Neural Information Processing Systems, 2024 (NeurIPS)
[Coming Soon]
Towards Multi-Layered 3D Garments Animation
Y. Shao, C. C. Loy, B. Dai
in Proceedings of IEEE/CVF International Conference on Computer Vision, 2023 (ICCV)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Transformer with Implicit Edges for Particle-based Physics Simulation
Y. Shao, C. C. Loy, B. Dai
European Conference on Computer Vision, 2022 (ECCV)
[PDF] [arXiv] [Project Page]

Image and Video Understanding

We explore effective and efficient methods to detect, segment and recognize objects in complex scenes.

Object Detection

DST-Det: Simple Dynamic Self-Training for Open-Vocabulary Object Detection
S. Xu, X. Li, S. Wu, W. Zhang, Y. Li, G. Cheng, Y. Tong, K. Chen, C. C. Loy
IEEE Transactions on Circuits and Systems for Video Technology, 2025 (TCVST)
[arXiv] [Project Page]
CLIPSelf: Vision Transformer Distills Itself for Open-Vocabulary Dense Prediction
S. Wu, W. Zhang, L. Xu, S. Jin, X. Li, W. Liu, C. C. Loy
International Conference on Learning Representations, 2024 (ICLR, Spotlight)
[PDF] [arXiv] [Project Page]
Aligning Bag of Regions for Open-Vocabulary Object Detection
S. Wu, W. Zhang, S. Jin, W. Liu, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2023 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Siamese DETR
Z. Chen, G. Huang, W. Li, J. Teng, K. Wang, J. Shao, C. C. Loy, L. Sheng
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2023 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Semi-Supervised and Long-Tailed Object Detection with CascadeMatch
Y. Zang, K. Zhou, C. Huang, C. C. Loy
International Journal of Computer Vision, 2022 (IJCV)
[PDF] [DOI] [arXiv] [Project Page]
Open-Vocabulary DETR with Conditional Matching
Y. Zang, W. Li, K. Zhou, C. Huang, C. C. Loy
European Conference on Computer Vision, 2022 (ECCV, Oral)
[PDF] [arXiv] [Supplementary Material] [Project Page]
CARAFE++: Unified Content-Aware ReAssembly of FEatures
J. Wang, K. Chen, R. Xu, Z. Liu, C. C. Loy, D. Lin
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021 (TPAMI)
[DOI] [arXiv]
Side-Aware Boundary Localization for More Precise Object Detection
J. Wang, W. Zhang, Y. Cao, K. Chen, J. Pang, T. Gong, J. Shi, C. C. Loy, D. Lin
European Conference on Computer Vision, 2020 (ECCV)
[PDF] [arXiv] [Supplementary Material]
RGB-D Salient Object Detection with Cross-Modality Modulation and Selection
C. Li, R. Cong, Y. Piao, Q. Xu, C. C. Loy
European Conference on Computer Vision, 2020 (ECCV)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Feature Pyramid Grids
K. Chen, Y. Cao, C. C. Loy, D. Lin, C. Feichtenhofer
Technical report, arXiv:2004.03580, 2020
[arXiv]
Prime Sample Attention in Object Detection
Y. Cao, K. Chen, C. C. Loy, D. Lin
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2020 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page]
CARAFE: Content-Aware ReAssembly of FEatures
J. Wang, K. Chen, R. Xu, Z. Liu, C. C. Loy, D. Lin
in Proceedings of International Conference on Computer Vision, 2019 (ICCV, Oral)
[PDF] [arXiv] [Supplementary Material]
Region Proposal by Guided Anchoring
J. Wang, K. Chen, S. Yang, C. C. Loy, D. Lin
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2019 (CVPR)
[PDF] [arXiv] [Project Page]
MMDetection: OpenMMLab Detection Toolbox and Benchmark
K. Chen, J. Wang, J. Pang, Y. Cao, Y. Xiong, X. Li, S. Sun, W. Feng, Z. Liu, J. Xu, Z. Zhang, D. Cheng, C. Zhu, T. Cheng, Q. Zhao, B. Li, X. Lu, R. Zhu, Y. Wu, J. Dai, J. Wang, J. Shi, W. Ouyang, C. C. Loy, D. Lin
Technical report, arXiv:1906.07155, 2019
[arXiv] [Project Page]
Optimizing Video Object Detection via a Scale-Time Lattice
K. Chen, J. Wang, S. Yang, X. Zhang, Y. Xiong, C. C. Loy, D. Lin
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2018 (CVPR)
[PDF] [arXiv] [Project Page] [YouTube]
Fusing Object Context to Detect Functional Area for Cognitive Robots
H. Cheng, J. Cai, Q. Liu, Z. Zhang, K. Yang, C. C. Loy, L. Lin
in Proceedings of IEEE International Conference on Robotics and Automation, 2018 (ICRA)
[PDF]
Discover and Learn New Objects from Documentaries
K. Chen, H. Song, C. C. Loy, D. Lin
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2017 (CVPR, Spotlight)
[PDF] [arXiv] [Project Page]
DeepID-Net: Object Detection with Deformable Part Based Convolutional Neural Networks
W. Ouyang, X. Zeng, X. Wang, S. Qiu, P. Luo, Y. Tian, H. Li, S. Yang, Z. Wang, H. Li, K. Wang, J. Yan, C. C. Loy, X. Tang
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016 (TPAMI)
[DOI]
DeepID-Net: Deformable Deep Convolutional Neural Networks for Object Detection
W. Ouyang, X. Wang, X. Zeng, S. Qiu, P. Luo, Y. Tian, H. Li, S. Yang, Z. Wang, C. C. Loy, X. Tang
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2403-2412, 2015 (CVPR)
[PDF] [Project Page]

Segmentation

Explore In-Context Segmentation via Latent Diffusion Models
C. Wang, X. Li, H. Ding, L. Qi, J. Zhang, Y. Tong, C. C. Loy, S. Yan
in Proceedings of AAAI Conference on Artificial Intelligence, 2025 (AAAI)
[arXiv]
OMG-LLaVA: Bridging Image-level, Object-level, Pixel-level Reasoning and Understanding
T. Zhang, X. Li, H. Fei, H. Yuan, S. Wu, S. Ji, C. C. Loy, S. Yan
in Proceedings of Neural Information Processing Systems, 2024 (NeurIPS)
[arXiv] [Project Page]
MosaicFusion: Diffusion Models as Data Augmenters for Large Vocabulary Instance Segmentation
J. Xie, W. Li, X. Li, Z. Liu, Y. S. Ong, C. C. Loy
International Journal of Computer Vision, 2024 (IJCV)
[DOI] [arXiv] [Project Page]
Transformer-Based Visual Segmentation: A Survey
X. Li, H. Ding, W. Zhang, H. Yuan, J. Pang, G. Cheng, K. Chen, Z. Liu, C. C. Loy
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024 (TPAMI)
[arXiv] [Project Page]
Open-Vocabulary SAM: Segment and Recognize Twenty-thousand Classes Interactively
H. Yuan, X. Li, C. Zhou, Y. Li, K. Chen, C. C. Loy
European Conference on Computer Vision, 2024 (ECCV)
[arXiv] [Project Page]
Eliminating Feature Ambiguity for Few-Shot Segmentation
Q. Xu, G. Lin, C. C. Loy, C. Long, Z. Li, R. Zhao
European Conference on Computer Vision, 2024 (ECCV)
[arXiv] [Project Page]
Mamba or RWKV: Exploring High-Quality and High-Efficiency Segment Anything Model
H. Yuan, X. Li, L. Qi, T. Zhang, M.-H. Yang, S. Yan, C. C. Loy
Technical report, arXiv:2406.19369, 2024
[arXiv]
OMG-Seg: Is One Model Good Enough For All Segmentation?
X. Li, H. Yuan, W. Li, H. Ding, S. Wu, W. Zhang, Y. Li, K. Chen, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2024 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page]
CLIM: Contrastive Language-Image Mosaic for Region Representation
S. Wu, W. Zhang, L. Xu, S. Jin, W. Liu, C. C. Loy
in Proceedings of AAAI Conference on Artificial Intelligence, 2024 (AAAI)
[arXiv] [Project Page]
EdgeSAM: Prompt-In-the-Loop Distillation for On-Device Deployment of SAM
C. Zhou, X. Li, C. C. Loy, B. Dai
Technical report, arXiv:2312.06660, 2023
[arXiv] [Project Page] [Demo]
Tube-Link: A Flexible Cross Tube Framework for Universal Video Segmentation
X. Li, H. Yuan, W. Zhang, G. Cheng, J. Pang, C. C. Loy
in Proceedings of IEEE/CVF International Conference on Computer Vision, 2023 (ICCV)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Betrayed by Captions: Joint Caption Grounding and Generation for Open Vocabulary Instance Segmentation
J. Wu, X. Li, H. Ding, X. Li, G. Cheng, Y. Tong, C. C. Loy
in Proceedings of IEEE/CVF International Conference on Computer Vision, 2023 (ICCV)
[PDF] [arXiv] [Supplementary Material] [Project Page]
MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressions
H. Ding, C. Liu, S. He, X. Jiang, C. C. Loy
in Proceedings of IEEE/CVF International Conference on Computer Vision, 2023 (ICCV)
[PDF] [arXiv] [Project Page]
Video K-Net: A Simple, Strong, and Unified Baseline For End-to-End Dense Video Segmentation
X. Li, W. Zhang, J. Pang, K. Chen, G. Cheng, Y. Tong, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2022 (CVPR, Oral)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Delving into High-Quality Synthetic Face Occlusion Segmentation Datasets
K. T. R. Voo, L. Jiang, C. C. Loy
in Workshop Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, VDU, 2022 (CVPRW)
[PDF] [arXiv] [Supplementary Material] [Project Page]
K-Net: Towards Unified Image Segmentation
W. Zhang, J. Pang, K. Chen, C. C. Loy
in Proceedings of Neural Information Processing Systems, 2021 (NeurIPS)
[PDF] [arXiv] [Project Page]
FASA: Feature Augmentation and Sampling Adaptation for Long-Tailed Instance Segmentation
Y. Zang, C. Huang, C. C. Loy
in Proceedings of IEEE/CVF International Conference on Computer Vision, 2021 (ICCV)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Seesaw Loss for Long-Tailed Instance Segmentation
J. Wang, W. Zhang, Y. Zang, Y. Cao, J. Pang, T. Gong, K. Chen, Z. Liu, C. C. Loy, D. Lin
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2021 (CVPR)
[PDF] [arXiv] [Supplementary Material]
Hybrid Task Cascade for Instance Segmentation
K. Chen, J. Pang, J. Wang, Y. Xiong, X. Li, S. Sun, W. Feng, Z. Liu, J. Shi, W. Ouyang, C. C. Loy, D. Lin
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2019 (CVPR)
[PDF] [arXiv] [Project Page]
Video Object Segmentation with Joint Re-identification and Attention-Aware Mask Propagation
X. Li, C. C. Loy
in Proceedings of European Conference on Computer Vision, 2018 (ECCV)
[PDF] [arXiv]
PSANet: Point-wise Spatial Attention Network for Scene Parsing
H. Zhao, Y. Zhang, S. Liu, J. Shi, C. C. Loy, D. Lin, J. Jia
in Proceedings of European Conference on Computer Vision, 2018 (ECCV)
[PDF] [Project Page]
Mix-and-Match Tuning for Self-supervised Semantic Segmentation
X. Zhan, Z. Liu, P. Luo, X. Tang, C. C. Loy
in Proceedings of AAAI Conference on Artificial Intelligence, 2018 (AAAI, Spotlight)
[PDF] [arXiv] [Project Page]
Video Object Segmentation with Re-identification
X. Li, Y. Qi, Z. Wang. K. Chen, Z. Liu, J. Shi, P. Luo, X. Tang, C. C. Loy
in Workshop of DAVIS Challenge on Video Object Segmentation, IEEE Conference on Computer Vision and Pattern Recognition, 2017 (CVPRW)
[PDF] [arXiv] [Project Page]
Not All Pixels Are Equal: Difficulty-Aware Semantic Segmentation via Deep Layer Cascade
X. Li, Z. Liu, P. Luo, C. C. Loy, X. Tang
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2017 (CVPR, Spotlight)
[PDF] [arXiv] [Project Page]
Deep Learning Markov Random Field for Semantic Segmentation
Z. Liu, X. Li, P. Luo, C. C. Loy, X. Tang
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017 (TPAMI)
[arXiv] [Project Page]
Semantic Image Segmentation via Deep Parsing Network
Z. Liu, X. Li, P. Luo, C. C. Loy, X. Tang
in Proceedings of International Conference on Computer Vision, pp. 1377-1385, 2015 (ICCV, Oral)
[PDF] [arXiv] [Project Page]

Tracking and Association

MessyTable: Instance Association in Multiple Camera Views
Z. Cai, J. Zhang, D. Ren, C. Yu, H. Zhao, S. Yi, C. K. Yeo, C. C. Loy
European Conference on Computer Vision, 2020 (ECCV)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Robust Multi-Modality Multi-Object Tracking
W. Zhang, H. Zhou, S. Sun, Z. Wang, J. Shi, C. C. Loy
in Proceedings of International Conference on Computer Vision, 2019 (ICCV)
[PDF] [arXiv] [Project Page]

Recognition

Interpret Vision Transformers as ConvNets with Dynamic Convolutions
C. Zhou, C. C. Loy, B. Dai
Technical report, arXiv:2309.10713, 2023
[arXiv]
Panoptic Video Scene Graph Generation
J. Yang, W. Peng, X. Li, Z. Guo, L. Chen, Bo Li, Z. Ma, W. Zhang, K. Zhou, C. C. Loy, Z. Liu
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2023 (CVPR)
[PDF]
Zoom-Net: Mining Deep Feature Interactions for Visual Relationship Recognition
G. Yin, L. Sheng, B. Liu, N. Yu, X. Wang, J. Shao, C. C. Loy
in Proceedings of European Conference on Computer Vision, 2018 (ECCV)
[PDF] [arXiv]
PolyNet: A Pursuit of Structural Diversity in Very Deep Networks
X. Zhang, Z. Li, C. C. Loy, D. Lin
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2017 (CVPR)
[PDF] [arXiv] [Project Page]
Learning to Disambiguate by Asking Discriminative Questions
Y. Li, C. Huang, X. Tang, C. C. Loy
in Proceedings of International Conference on Computer Vision, 2017 (ICCV)
[PDF] [arXiv] [Project Page]
Local Similarity-Aware Deep Feature Embedding
C. Huang, C. C. Loy, X. Tang
in Proceedings of Neural Information Processing Systems, 2016 (NIPS)
[PDF] [arXiv]
Unsupervised Learning of Discriminative Attributes and Visual Representations
C. Huang, C. C. Loy, X. Tang
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2016 (CVPR)
[PDF] [Project Page]
Learning Deep Representation for Imbalanced Classification
C. Huang, Y. Li, C. C. Loy, X. Tang
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2016 (CVPR, Spotlight)
[PDF] [Project Page]
A Large-Scale Car Dataset for Fine-Grained Categorization and Verification
L. Yang, P. Luo, C. C. Loy, X. Tang
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 3973-3981, 2015 (CVPR)
[PDF] [arXiv] [Project Page]
Deep Representation Learning with Target Coding
S. Yang, P. Luo, C. C. Loy, K. W. Shum, X. Tang
in Proceedings of AAAI Conference on Artificial Intelligence, 2015 (AAAI, Oral, nominated as easily accessible paper)
[PDF] [Supplementary Material] [Project Page]

3D Content Creation and Understanding

Our team has been working on various tasks related to 3D content creation and perception, e.g, 3D shape generation and 3D human recovery

3D Generation | Reconstruction

GaussianAnything: Interactive Point Cloud Latent Diffusion for 3D Generation
Y. Lan, S. Zhou, Z. Lyu, F. Hong, S. Yang, B. Dai, X. Pan, C. C. Loy
Technical report, arXiv:2411.08033, 2024
[arXiv] [Project Page] [Demo]
Playing for 3D Human Recovery
Z. Cai, M. Zhang, J. Ren, C. Wei, D. Ren, J. Li, Z. Lin, H. Zhao, S. Yi, L. Yang, C. C. Loy, Z. Liu
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024 (TPAMI)
[arXiv] [Project Page]
Gaussian3Diff: 3D Gaussian Diffusion for 3D Full Head Synthesis and Editing
Y. Lan, F. Tan, D. Qiu, Q. Xu, K. Genova, Z. Huang, R. Pandey, S. Fanello, T. Funkhouser, C. C. Loy, Y. Zhang
European Conference on Computer Vision, 2024 (ECCV)
[arXiv] [Project Page]
LN3Diff: Scalable Latent Neural Fields Diffusion for Speedy 3D Generation
Y. Lan, F. Hong, S. Yang, S. Zhou, X. Meng, B. Dai, X. Pan, C. C. Loy
European Conference on Computer Vision, 2024 (ECCV)
[arXiv] [Project Page]
PERF: Panoramic Neural Radiance Field from a Single Panorama
G. Wang, P. Wang, Z. Chen, W. Wang, C. C. Loy, Z. Liu
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024 (TPAMI)
[arXiv] [Project Page]
Digital Life Project: Autonomous 3D Characters with Social Intelligence
Z. Cai, J. Jiang, Z. Qing, X. Guo, M. Zhang, Z. Lin, H. Mei, C. Wei, R. Wang, W. Yin, X. Fan, H. Du, L. Pan, P. Gao, Z. Yang, Y. Gao, J. Li, T. Ren, Y. Wei, X. Wang, C. C. Loy, L. Yang, Z. Liu
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2024 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page] [YouTube]
MVIP-NeRF: Multi-view 3D Inpainting on NeRF Scenes via Diffusion Prior
H. Chen, C. C. Loy, X. Pan
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2024 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Duolando: Follower GPT with Off-Policy Reinforcement Learning for Dance Accompaniment
L. Siyao, T. Gu, Z. Yang, Z. Lin, Z. Liu, H. Ding, L. Yang, C. C. Loy
International Conference on Learning Representations, 2024 (ICLR)
[PDF] [arXiv] [Project Page]
PaintHuman: Towards High-fidelity Text-to-3D Human Texturing via Denoised Score Distillation
J. Yu, H. Zhu, L. Jiang, C. C. Loy, W. Cai, W. Wu
in Proceedings of AAAI Conference on Artificial Intelligence, 2024 (AAAI)
[arXiv]
Gaussian3Diff: 3D Gaussian Diffusion for 3D Full Head Synthesis and Editing
Y. Lan, F. Tan, D. Qiu, Q. Xu, K. Genova, Z. Huang, S. Fanello, R. Pandey, T. Funkhouser, C. C. Loy, Y. Zhang
Technical report, arXiv:2312.03763, 2023
[arXiv] [Project Page]
Task-Oriented Human-Object Interactions Generation with Implicit Neural Representations
Q. Li, J. Wang, C. C. Loy, B. Dai
in Proceedings of IEEE/CVF Winter Conference on Applications of Computer Vision, 2024 (WACV)
[arXiv]
RenderMe-360: A Large Digital Asset Library and Benchmarks Towards High-fidelity Head Avatars
D. Pan, L. Zhuo, J. Piao, H. Luo, W. Cheng, Y. Wang, S. Fan, S. Liu, L. Yang, B. Dai, Z. Liu, C. C. Loy, C. Qian, W. Wu, D. Lin, K.-Y. Lin
in Proceedings of Neural Information Processing Systems (Datasets and Benchmarks Track), 2023 (NeurIPS)
[PDF] [arXiv] [Project Page]
SMPLer-X: Scaling Up Expressive Human Pose and Shape Estimation
Z. Cai, W. Yin, A. Zeng, C. Wei, Q. Sun, Y. Wang, H. E. Pang, H. Mei, M. Zhang, L. Zhang, C. C. Loy, L. Yang, Z. Liu
in Proceedings of Neural Information Processing Systems (Datasets and Benchmarks Track), 2023 (NeurIPS)
[PDF] [arXiv] [Project Page] [YouTube]
DeformToon3D: Deformable 3D Toonification from Neural Radiance Fields
J. Zhang, Y. Lan, S. Yang, F. Hong, Q. Wang, C. K. Yeo, Z. Liu, C. C. Loy
in Proceedings of IEEE/CVF International Conference on Computer Vision, 2023 (ICCV)
[PDF] [arXiv] [Supplementary Material] [Project Page]
SparseNeRF: Distilling Depth Ranking for Few-shot Novel View Synthesis
G. Wang, Z. Chen, C. C. Loy, Z. Liu
in Proceedings of IEEE/CVF International Conference on Computer Vision, 2023 (ICCV)
[PDF] [arXiv] [Project Page]
DNA-Rendering: A Diverse Neural Actor Repository for High-Fidelity Human-centric Rendering
W. Cheng, R. Chen, K. Chen, Z. Cai, B. Dai, S. Fan, Y. Gao, Z. Lin, D. Lin, Z. Liu, K.-Y. Lin, C. C. Loy, C. Qian, D. Ren, W. Wu, J. Wang, Z. Yu, W. Yin, L. Yang
in Proceedings of IEEE/CVF International Conference on Computer Vision, 2023 (ICCV)
[PDF] [arXiv] [Project Page]
Correspondence Distillation from NeRF-based GAN
Y. Lan, C. C. Loy, B. Dai
International Journal of Computer Vision, 2023 (IJCV)
[PDF] [DOI] [arXiv] [Project Page]
Self-Supervised Geometry-Aware Encoder for Style-Based 3D GAN Inversion
Y. Lan, X. Meng, S. Yang, C. C. Loy, B. Dai
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2023 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Monocular 3D Object Reconstruction with GAN Inversion
J. Zhang, D. Ren, Z. Cai, C. K. Yeo, B. Dai, C. C. Loy
European Conference on Computer Vision, 2022 (ECCV)
[PDF] [arXiv] [Supplementary Material] [Project Page]
HuMMan: Multi-Modal 4D Human Dataset for Versatile Sensing and Modeling
Z. Cai, D. Ren, A. Zeng, Z. Lin, T. Yu, W. Wang, X. Fan, Y. Gao, Y. Yu, L. Pan, F. Hong, M. Zhang, C. C. Loy, L. Yang, Z. Liu
European Conference on Computer Vision, 2022 (ECCV, Oral)
[PDF] [arXiv] [Supplementary Material] [Project Page] [YouTube]
Chasing the Tail in Monocular 3D Human Reconstruction with Prototype Memory
Y. Rong, Z. Liu, C. C. Loy
IEEE Transactions on Image Processing, 2022 (TIP)
[DOI] [arXiv] [Project Page]
A Shading-Guided Generative Implicit Model for Shape-Accurate 3D-Aware Image Synthesis
X. Pan, X. Xu, C. C. Loy, C. Theobalt, B. Dai
in Proceedings of Neural Information Processing Systems, 2021 (NeurIPS)
[PDF] [arXiv] [Project Page]
Monocular 3D Reconstruction of Interacting Hands via Collision-Aware Factorized Refinements
Y. Rong, J. Wang, Z. Liu, C. C. Loy
in Proceedings of International Conference on 3D Vision, 2021 (3DV)
[arXiv] [Project Page]
3D Human Texture Estimation from a Single Image with Transformers
X. Xu, C. C. Loy
in Proceedings of IEEE/CVF International Conference on Computer Vision, 2021 (ICCV, Oral)
[PDF] [arXiv] [Project Page]
Unsupervised 3D Shape Completion through GAN Inversion
J. Zhang, X. Chen, Z. Cai, L. Pan, H. Zhao, S. Yi, C. K. Yeo, B. Dai, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2021 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Do 2D GANs Know 3D Shape? Unsupervised 3D Shape Reconstruction from 2D Image GANs
X. Pan, B. Dai, Z. Liu, C. C. Loy, P. Luo
International Conference on Learning Representations, 2021 (ICLR, Oral)
[PDF] [arXiv] [Project Page]
Delving Deep into Hybrid Annotations for 3D Human Recovery in the Wild
Y. Rong, Z. Liu, C. Li, K. Cao, C. C. Loy
in Proceedings of International Conference on Computer Vision, 2019 (ICCV)
[PDF] [arXiv] [Project Page] [Supplementary Material]

3D Perception

Position-Guided Point Cloud Panoptic Segmentation Transformer
Z. Xiao, W. Zhang, T. Wang, C. C. Loy, D. Lin, J. Pang
International Journal of Computer Vision, 2024 (IJCV)
[DOI] [arXiv] [Project Page]
Point-In-Context: Understanding Point Cloud via In-Context Learning
M. Liu, Z. Fang, X. Li, J. M. Buhmann, X. Li, C. C. Loy
Technical report, arXiv:2404.12352, 2024
[arXiv] [Project Page]
Explore In-Context Learning for 3D Point Cloud Understanding
Z. Fang, X. Li, X. Li, J. M. Buhmann, C. C. Loy, M. Liu
in Proceedings of Neural Information Processing Systems, 2023 (NeurIPS, Spotlight)
[PDF] [arXiv] [Project Page]
PointHPS: Cascaded 3D Human Pose and Shape Estimation from Point Clouds
Z. Cai, L. Pan, C. Wei, W. Yin, F. Hong, M. Zhang, C. C. Loy, L. Yang, Z. Liu
Technical report, arXiv:2308.14492, 2023
[arXiv] [Project Page]
Improving Data Augmentation for Multi-Modality 3D Object Detection
W. Zhang, Z. Wang, C. C. Loy
International Conference on Learning Representations Workshop, 2023 (ICLRW)
[arXiv] [Project Page]

Deep Learning

We investigate new deep learning methods that are more efficient, robust, accurate, scalable, transferable, and explainable.

Multimodal Large Language Models

Contextual Object Detection with Multimodal Large Language Models
Y. Zang, W. Li, J. Han, K. Zhou, C. C. Loy
International Journal of Computer Vision, 2024 (IJCV)
[arXiv] [Project Page]
F-LMM: Grounding Frozen Large Multimodal Models
S. Wu, S. Jin, W. Zhang, L. Xu, W. Liu, W. Li, C. C. Loy
Technical report, arXiv:2406.05821, 2024
[arXiv] [Project Page]

Unsupervised | Self-Supervised Learning

Correlational Image Modeling for Self-Supervised Visual Pre-Training
W. Li, J. Xie, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2023 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Masked Frequency Modeling for Self-Supervised Visual Pre-Training
J. Xie, W. Li, X. Zhan, Z. Liu, Y. S. Ong, C. C. Loy
International Conference on Learning Representations, 2023 (ICLR)
[arXiv] [Project Page]
Delving into Inter-Image Invariance for Unsupervised Visual Representations
J. Xie, X. Zhan, Z. Liu, Y. S. Ong, C. C. Loy
International Journal of Computer Vision, 2022 (IJCV)
[PDF] [DOI] [arXiv] [Project Page]
Dense Siamese Network for Dense Unsupervised Learning
W. Zhang, J. Pang, K. Chen, C. C. Loy
European Conference on Computer Vision, 2022 (ECCV)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Self-Supervised Representation Learning: Introduction, Advances and Challenges
L. Ericsson, H. Gouk, C. C. Loy, T. M. Hospedales
IEEE Signal Processing Magazine, 2022 (SPM)
[DOI] [arXiv]
Unsupervised Object-Level Representation Learning from Scene Images
J. Xie, X. Zhan, Z. Liu, Y. S. Ong, C. C. Loy
in Proceedings of Neural Information Processing Systems, 2021 (NeurIPS)
[PDF] [arXiv] [Project Page]
Online Deep Clustering for Unsupervised Representation Learning
X. Zhan, J. Xie, Z. Liu, Y. S. Ong, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2020 (CVPR)
[PDF] [arXiv] [Project Page]
Self-Supervised Learning via Conditional Motion Propagation
X. Zhan, X. Pan, Z. Liu, D. Lin, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2019 (CVPR)
[PDF] [arXiv] [Project Page]
Mix-and-Match Tuning for Self-supervised Semantic Segmentation
X. Zhan, Z. Liu, P. Luo, X. Tang, C. C. Loy
in Proceedings of AAAI Conference on Artificial Intelligence, 2018 (AAAI, Spotlight)
[PDF] [arXiv] [Project Page]
Unsupervised Learning of Discriminative Attributes and Visual Representations
C. Huang, C. C. Loy, X. Tang
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2016 (CVPR)
[PDF] [Project Page]

Knowledge Distillation

Computation-Efficient Knowledge Distillation via Uncertainty-Aware Mixup
G. Xu, Z. Liu, C. C. Loy
Pattern Recognition, 2023 (PR)
[DOI] [arXiv] [Project Page]
Mind the Gap in Distilling StyleGANs
G. Xu, Y. Hou, Z. Liu, C. C. Loy
European Conference on Computer Vision, 2022 (ECCV)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Point-to-Voxel Knowledge Distillation for LiDAR Semantic Segmentation
Y. Hou, X. Zhu, Y. Ma, C. C. Loy, Y. Li
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2022 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Knowledge Distillation Meets Self-Supervision
G. Xu, Z. Liu, X. Li, C. C. Loy
European Conference on Computer Vision, 2020 (ECCV)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Residual Knowledge Distillation
M. Gao, Y. Shen, Q. Li, C. C. Loy
Technical report, arXiv:2002.09168, 2020
[arXiv]
Inter-Region Affinity Distillation for Road Marking Segmentation
Y. Hou, Z. Ma, C. Liu, T.-W. Hui, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2020 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Learning Lightweight Lane Detection CNNs by Self Attention Distillation
Y. Hou, Z. Ma, C, Liu, C. C. Loy
in Proceedings of International Conference on Computer Vision, 2019 (ICCV)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Learning to Steer by Mimicking Features from Heterogeneous Auxiliary Networks
Y. Hou, Z. Ma, C. Liu, C. C. Loy
in Proceedings of AAAI Conference on Artificial Intelligence, 2019 (AAAI, Oral)
[PDF] [arXiv] [Project Page]
An Embarrassingly Simple Approach for Knowledge Distillation
M. Gao, Y. Shen, Q. Li, C. C. Loy, X. Tang
Technical report, arXiv:1812.01819, 2018
[arXiv]

Continual Learning

Retrospective Class Incremental Learning
Q. Tao, C. C. Loy, J. Cai, Z. Ge, S. See
in Proceedings of IEEE International Conference on Multimedia and Expo, 2021 (ICME)
[PDF]
Learning a Unified Classifier Incrementally via Rebalancing
S. Hou, X. Pan, C. C. Loy, Z. Wang, D. Lin
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2019 (CVPR)
[PDF] [Project Page]
Lifelong Learning via Progressive Distillation and Retrospection
S. Hou, X. Pan, C. C. Loy, Z. Wang, D. Lin
in Proceedings of European Conference on Computer Vision, 2018 (ECCV)
[PDF] [Project Page]

Long-Tailed Recognition

FASA: Feature Augmentation and Sampling Adaptation for Long-Tailed Instance Segmentation
Y. Zang, C. Huang, C. C. Loy
in Proceedings of IEEE/CVF International Conference on Computer Vision, 2021 (ICCV)
[arXiv] [Project Page]
Learning Deep Representation for Imbalanced Classification
C. Huang, Y. Li, C. C. Loy, X. Tang
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2016 (CVPR, Spotlight)
[PDF] [Project Page]

Model Uncertainty and Robustness

Semi-Supervised Domain Generalization with Stochastic StyleMatch
K. Zhou, C. C. Loy, Z. Liu
International Journal of Computer Vision, 2023 (IJCV)
[PDF] [DOI] [arXiv] [Project Page]
On-Device Domain Generalization
K. Zhou, Y. Zhang, Y. Zang, J. Yang, C. C. Loy, Z. Liu
Technical report, arXiv:2209.07521, 2022
[arXiv]
Domain Generalization: A Survey
K. Zhou, Z. Liu, Y. Qiao, T. Xiang, C. C. Loy
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022 (TPAMI)
[DOI] [arXiv]
Sketch Me That Shoe
Q. Yu, F. Liu, Y. Song, T. Xiang, T. M. Hospedales, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2016 (CVPR, Oral)
[PDF]

Network Compression

Network Pruning via Resource Reallocation
Y. Hou, Z. Ma, C. Liu, Z. Wang, C. C. Loy
Pattern Recognition, 2023 (PR)
[DOI] [arXiv] [Project Page]
EcoNAS: Finding Proxies for Economical Neural Architecture Search
D. Zhou, X. Zhou, W. Zhang, C. C. Loy, S. Yi, X. Zhang, W. Ouyang
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2020 (CVPR)
[PDF] [arXiv] [Supplementary Material]

Zero/Few-shot Learning

Unified Vision and Language Prompt Learning
Y. Zang, W. Li, K. Zhou, C. Huang, C. C. Loy
Technical report, arXiv:2210.07225, 2022
[arXiv]
Extract Free Dense Labels from CLIP
C. Zhou, C. C. Loy, B. Dai
European Conference on Computer Vision, 2022 (ECCV, Oral)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Conditional Prompt Learning for Vision-Language Models
K. Zhou, J. Yang, C. C. Loy, Z. Liu
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2022 (CVPR)
[PDF] [arXiv] [Project Page]
Learning to Prompt for Vision-Language Models
K. Zhou, J. Yang, C. C. Loy, Z. Liu
International Journal of Computer Vision, 2022 (IJCV)
[DOI] [arXiv] [Project Page]

Face Analysis

We are one of the earliest teams that investigate the use of deep learning in various face analysis applications such as face detection and face alignment.

Face Detection

WIDER Face and Pedestrian Challenge 2018: Methods and Results
C. C. Loy, D. Lin, W. Ouyang, Y. Xiong, S. Yang, Q. Huang, D. Zhou, W. Xia, Q. Li, P. Luo, J. Yan, et al.
Technical report, arXiv:1902.06854, 2019
[arXiv] [Project Page]
Face Detection through Scale-Friendly Deep Convolutional Networks
S. Yang, Y. Xiong, C. C. Loy, X. Tang
Technical report, arXiv:1706.02863, 2017
[arXiv] [Project Page]
Faceness-Net: Face Detection through Deep Facial Part Responses
S. Yang, P. Luo, C. C. Loy, X. Tang
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017 (TPAMI)
[DOI] [arXiv] [Project Page]
WIDER FACE: A Face Detection Benchmark
S. Yang, P. Luo, C. C. Loy, X. Tang
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2016 (CVPR, Oral)
[PDF] [arXiv] [Leaderboard | Dataset]
From Facial Part Responses to Face Detection: A Deep Learning Approach
S. Yang, P. Luo, C. C. Loy, X. Tang
in Proceedings of International Conference on Computer Vision, pp. 3676-3684, 2015 (ICCV)
[PDF] [arXiv] [Project Page]

Face Alignment

Unconstrained Face Alignment via Cascaded Compositional Learning
S. Zhu, C. Li, C. C. Loy, X. Tang
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2016 (CVPR)
[PDF] [Project Page]
Learning Deep Representation for Face Alignment with Auxiliary Attributes
Z. Zhang, P. Luo, C. C. Loy, X. Tang
IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 5, pp. 918–930, 2016 (TPAMI)
[DOI] [arXiv] [Project Page]
Face Alignment by Coarse-to-Fine Shape Searching
S. Zhu, C. Li, C. C. Loy, X. Tang
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 4998-5006, 2015 (CVPR)
[PDF] [Project Page]
Towards Arbitrary-View Face Alignment by Recommendation Trees
S. Zhu, C. Li, C. C. Loy, X. Tang
Technical report, arXiv:1511.06627, 2015
[arXiv]
An Empirical Study of Recent Face Alignment Methods
H. Yang, X. Jia, C. C. Loy, P. Robinson
Technical report, arXiv:1511.05049, 2015
[arXiv] [Project Page]
Facial Landmark Detection by Deep Multi-task Learning
Z. Zhang, P. Luo, C. C. Loy, X. Tang
in Proceedings of European Conference on Computer Vision, pp. 94-108, 2014 (ECCV)
[PDF] [arXiv] [Project Page]
Transferring Landmark Annotations for Cross-Dataset Face Alignment
S. Zhu, C. Li, C. C. Loy, X. Tang
Technical report, arXiv:1409.0602, 2014
[PDF] [Project Page]

Face Attribute Recognition

Deep Imbalanced Learning for Face Recognition and Attribute Prediction
C. Huang, Y. Li, C. C. Loy, X. Tang
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019 (TPAMI)
[DOI] [arXiv]
From Facial Expression Recognition to Interpersonal Relation Prediction
Z. Zhang, P. Luo, C. C. Loy, X. Tang
International Journal of Computer Vision, 2017 (IJCV)
[DOI] [arXiv] [Project Page]
Quantifying Facial Age by Posterior of Age Comparisons
Y. Zhang, L. Liu, C. Li, C. C. Loy
in Proceedings of British Machine Vision Conference, 2017 (BMVC, Oral)
[PDF] [arXiv] [Project Page]
Discriminative Sparse Neighbor Approximation for Imbalanced Learning
C. Huang, C. C. Loy, X. Tang
IEEE Transactions on Neural Networks and Learning Systems, 2017 (TNNLS)
[DOI] [PDF] [arXiv]
Deep Learning Face Attributes for Face Detection and Alignment
C. C. Loy, P. Luo, and C. Huang
In R. S. Feris, C. Lampert and D. Parikh (Eds.), Visual Attributes, Springer, 2017
[Book Link]
Learning Deep Representation for Imbalanced Classification
C. Huang, Y. Li, C. C. Loy, X. Tang
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2016 (CVPR, Spotlight)
[PDF] [Project Page]
Learning Social Relation Traits from Face Images
Z. Zhang, P. Luo, C. C. Loy, X. Tang
in Proceedings of International Conference on Computer Vision, pp. 3631-3639, 2015 (ICCV)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Cumulative Attribute Space for Age and Crowd Density Estimation
K. Chen, S. Gong, T. Xiang, and C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 2467-2474, 2013 (CVPR, Oral)
[PDF] [Poster] [Project Page]

Face Clustering

Learning to Cluster Faces via Confidence and Connectivity Estimation
L. Yang, D. Chen, X. Zhan, R. Zhao, C. C. Loy, D. Lin
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2020 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page]
Learning to Cluster Faces on an Affinity Graph
L. Yang, X. Zhan, D. Chen, J. Yan, C. C. Loy, D. Lin
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2019 (CVPR, Oral)
[PDF] [arXiv] [Project Page]
Merge or Not? Learning to Group Faces via Imitation Learning
Y. He, K. Cao, C. Li, C. C. Loy
in Proceedings of AAAI Conference on Artificial Intelligence, 2018 (AAAI, Spotlight)
[PDF] [arXiv] [Project Page]
Joint Face Representation Adaptation and Clustering in Videos
Z. Zhang, P. Luo, C. C. Loy, X. Tang
in Proceedings of European Conference on Computer Vision, 2016 (ECCV)
[PDF] [Project Page]

Face Recognition

Deep Imbalanced Learning for Face Recognition and Attribute Prediction
C. Huang, Y. Li, C. C. Loy, X. Tang
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019 (TPAMI)
[DOI] [arXiv]
Consensus-Driven Propagation in Massive Unlabeled Data for Face Recognition
X. Zhan, Z. Liu, J. Yan, D. Lin, C. C. Loy
in Proceedings of European Conference on Computer Vision, 2018 (ECCV)
[PDF] [arXiv] [Project Page]
The Devil of Face Recognition is in the Noise
F. Wang, L. Chen, C. Li, S. Huang, Y. Chen, C. Qian, C. C. Loy
in Proceedings of European Conference on Computer Vision, 2018 (ECCV)
[PDF] [arXiv] [Project Page]
Pose-Robust Face Recognition via Deep Residual Equivariant Mapping
K. Cao, Y. Rong, C. Li, X. Tang, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2018 (CVPR)
[PDF] [arXiv] [Project Page]
A Survey on Heterogeneous Face Recognition: Sketch, Infra-red, 3D and Low-Resolution
S. Ouyang, T. Hospedales, Y. Song, X. Li, C. C. Loy, X. Wang
Image and Vision Computing, 2016 (IVC)
[DOI] [PDF]

Visual Surveillance

We investigate methods to analyze human behaviours across cameras and estimate crowd density.

Camera Network Analysis

Incremental Activity Modelling in Multiple Disjoint Cameras
C. C. Loy, T. Xiang, S. Gong
IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 9, pp. 1799-1813, 2012 (TPAMI)
[DOI] [PDF] [Project Page]
Comparing Visual Feature Coding for Learning Disjoint Camera Dependencies
X. Zhu, S. Gong, and C. C. Loy
in Proceedings of British Machine Vision Conference, 2012 (BMVC)
[PDF] [Poster]
Security and Surveillance
S. Gong, C. C. Loy, T. Xiang
in T. Moeslund, A. Hilton, V. Krueger, L. Sigal (Eds.), Visual Analysis of Humans: Looking at People, Springer, pp. 455-472, 2011
[DOI] [PDF]
Activity Understanding and Unusual Event Detection in Surveillance Videos
C. C. Loy
PhD Thesis, Queen Mary University of London, 2010
[PDF]
Time-Delayed Correlation Analysis for Multi-Camera Activity Understanding
C. C. Loy, T. Xiang, S. Gong
International Journal of Computer Vision, vol. 90, no. 1, pp. 106-129, 2010 (IJCV)
[DOI] [PDF] [Project Page]
Modelling Activity Global Temporal Dependencies using Time Delayed Probabilistic Graphical Model
C. C. Loy, T. Xiang, S. Gong
in Proceedings of International Conference on Computer Vision, pp. 120-127, 2009 (ICCV, Oral)
[PDF] [Project Page]
Multi-Camera Activity Correlation Analysis
C. C. Loy, T. Xiang, S. Gong
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1988-1995, 2009 (CVPR, Oral)
[PDF] [Project Page]

Person Re-Identification

Pedestrian Color Naming via Convolutional Neural Network
Z. Cheng, X. Li, and C. C. Loy
in Proceedings of Asian Conference on Computer Vision, 2016 (ACCV, Oral, Best Application Paper Honorable Mention)
[PDF] [Project Page]
Human Attribute Recognition by Deep Hierarchical Contexts
Y. Li, C. Huang, C. C. Loy, X. Tang
in Proceedings of European Conference on Computer Vision, 2016 (ECCV)
[PDF] [Project Page]
Pedestrian Attribute Recognition At Far Distance
Y. Deng, P. Luo, C. C. Loy, X. Tang
ACM Multimedia, pp. 789-792, 2014 (ACM MM)
[PDF] [arXiv] [Project Page]
On-the-fly Feature Importance Mining for Person Re-Identification
C. Liu, S. Gong, and C. C. Loy
Pattern Recognition, vol. 47, no. 4, pp. 1602-1615, 2014 (PR)
[DOI] [PDF] [Project Page]
Person Re-Identification
S. Gong, M. Cristani, S. Yan, C. C. Loy (Eds.)
Springer, January 2014
[DOI] [Preface] [Introduction]
The Re-Identification Challenge
S. Gong, M. Cristani, C. C. Loy, and T. Hospedales
In Gong, Cristani, Yan, Loy (Eds.), Person Re-Identification, Springer, January 2014
[PDF]
Evaluating Feature Importance for Re-Identification
C. Liu, S. Gong, C. C. Loy, X. Lin
In Gong, Cristani, Yan, Loy (Eds.), Person Re-Identification, Springer, January 2014
[PDF]
POP: Person Re-Identification Post-Rank Optimisation
C. Liu, C. C. Loy, S. Gong, and G. Wang
in Proceedings of IEEE International Conference on Computer Vision, pp. 441-448, 2013 (ICCV)
[PDF] [Poster] [Project Page]
Person Re-Identification by Manifold Ranking
C. C. Loy, C. Liu, S. Gong
in Proceedings of IEEE International Conference on Image Processing, pp. 3567-3571, 2013 (ICIP)
[PDF] [Poster] [Project Page]
Person Re-Identification: What Features are Important?
C. Liu, S. Gong, C. C. Loy, X. Lin
in Proceedings of European Conference on Computer Vision, International Workshop on Re-Identification, pp. 391-401, 2012 (ECCVW)
[PDF] [Project Page]

Crowd Behaviour Understanding and Profiling

Deep Learning for Scene Independent Crowd Analysis
X. Wang and C. C. Loy
In V. Murino, M. Cristani, S. Shah, S. Savarese (Eds.), Group and Crowd Behavior for Computer Vision, Elsevier, 2017
[Book Link]
Crowded Scene Understanding by Deeply Learned Volumetric Slices
J. Shao, C. C. Loy, X. Wang
IEEE Transactions on Circuits and Systems for Video Technology, 2016 (TCVST)
[DOI]
Slicing Convolutional Neural Network for Crowd Video Understanding
J. Shao, C. C. Loy, K. Kang, X. Wang
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2016 (CVPR, Spotlight)
[PDF]
Learning Scene-Independent Group Descriptors for Crowd Understanding
J. Shao, C. C. Loy, K. Kang, X. Wang
IEEE Transactions on Circuits and Systems for Video Technology, 2016 (TCVST)
[DOI] [PDF]
Deeply Learned Attributes for Crowded Scene Understanding
J. Shao, K. Kang, C. C. Loy, X. Wang
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 4657-4666, 2015 (CVPR, Oral)
[PDF]
Scene-Independent Group Profiling in Crowd
J. Shao, C. C. Loy, X. Wang
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 2227-2234, 2014 (CVPR, Oral)
[PDF] [Demo]
Stream-based Joint Exploration-Exploitation Active Learning
C. C. Loy, T. M. Hospedales, T. Xiang, S. Gong
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1560-1567, 2012 (CVPR, Oral)
[PDF] [Project Page]
Crowd Saliency Detection via Global Similarity Structure
M. K. Lim, V. J. Kok, C. C. Loy, and C. S. Chan
in Proceedings of International Conference on Pattern Recognition, pp. 3957-3962, 2014 (ICPR, Oral)
[PDF]
Salient Motion Detection in Crowded Scenes
C. C. Loy, T. Xiang, S. Gong
in Proceedings of International Symposium on Communications, Control and Signal Processing, pp. 1-4, 2012
[PDF]
Detecting and Discriminating Behavioural Anomalies
C. C. Loy, T. Xiang, S. Gong
Pattern Recognition, vol. 44, no. 1, pp. 117-132, 2011 (PR)
[DOI] [PDF]
Stream-based Active Unusual Event Detection
C. C. Loy, T. Xiang, S. Gong
in Proceedings of Asian Conference on Computer Vision, pp. 161-175, 2010 (ACCV, Oral)
[PDF]
Modelling Multi-object Activity by Gaussian Processes
C. C. Loy, T. Xiang, S. Gong
in Proceedings of British Machine Vision Conference, 2009 (BMVC)
[PDF] [Extended Abstract] [Poster]
Surveillance Video Behaviour Profiling and Anomaly Detection
C. C. Loy, T. Xiang, S. Gong
in Proceedings of Society of Photo-Optical Instrumentation Engineers Conference Series, 2009
From Local Temporal Correlation to Global Anomaly Detection
C. C. Loy, T. Xiang, S. Gong
in Proceedings of European Conference on Computer Vision, International Workshop on Machine Learning for Vision-based Motion Analysis, 2008 (ECCVW)

Crowd Density Estimation

From Semi-Supervised to Transfer Counting of Crowds
C. C. Loy, S. Gong, T. Xiang
in Proceedings of IEEE International Conference on Computer Vision, pp. 2256-2263, 2013 (ICCV)
[PDF] [Poster] [Project Page]
Cumulative Attribute Space for Age and Crowd Density Estimation
K. Chen, S. Gong, T. Xiang, and C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 2467-2474, 2013 (CVPR, Oral)
[PDF] [Poster] [Project Page]
Crowd Counting and Profiling: Methodology and Evaluation
C. C. Loy, K. Chen, S. Gong, T. Xiang
in S. Ali, K. Nishino, D. Manocha, and M. Shah (Eds.), Modeling, Simulation and Visual Analysis of Crowds, Springer, vol. 11, pp. 347-382, 2013
[DOI] [PDF]
Feature Mining for Localised Crowd Counting
K. Chen, C. C. Loy, S. Gong, T. Xiang
in Proceedings of British Machine Vision Conference, 2012 (BMVC)
[PDF] [Poster] [Project Page]

Video Summarization

Learning from Multiple Sources for Video Summarisation
X. Zhu, C. C. Loy, S. Gong
International Journal of Computer Vision, vol. 117, no. 3, pp. 247–268, 2015 (IJCV)
[DOI] [PDF] [arXiv] [Project Page]
Constrained Clustering with Imperfect Oracles
X. Zhu, C. C. Loy, S. Gong
IEEE Transactions on Neural Networks and Learning Systems, 2015 (TNNLS)
[DOI] [PDF] [Project Page]
Constructing Robust Affinity Graphs for Spectral Clustering
X. Zhu, C. C. Loy, S. Gong
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1450-1457, 2014 (CVPR)
[PDF] [Poster] [Project Page]
Video Synopsis by Heterogeneous Multi-Source Correlation
X. Zhu, C. C. Loy, S. Gong
in Proceedings of IEEE International Conference on Computer Vision, pp. 81-88, 2013 (ICCV)
[PDF] [Poster] [Project Page]
Constrained Clustering: Effective Constraint Propagation with Imperfect Oracles
X. Zhu, C. C. Loy, S. Gong
in Proceedings of IEEE International Conference on Data Mining, pp. 1307-1312, 2013 (ICDM)
[PDF] [Project Page]

Media Forensics

We collect large-scale datasets and develop new methods for face forgery detection.

Forgery Detection and Anti-Deepfake

DeepFakes Detection: the DeeperForensics Dataset and Challenge
L. Jiang, W. Wu, C. Qian, C. C. Loy
In C. Rathgeb, R. Tolosana, R. Vera-Rodriguez, C. Busch (Eds.), Handbook of Digital Face Manipulation and Detection, Springer, 2022
[Book Link]
DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection
L. Jiang, R. Li, W. Wu, C. Qian, C. C. Loy
in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2020 (CVPR)
[PDF] [arXiv] [Supplementary Material] [Project Page]

Topics

Sort by

MMLab@NTU