I am a PhD student in Machine Learning supervised by Thomas Schön and co-supervised by Jens Sjölund at Uppsala University. My research interest generally includes Machine Learning, Computer Vision, and Reinforcement Learning. And my recent works mainly focus on Probabilistic deep learning with application in computer vision.

I received my Master of Engineering degree from the School of Computer Science, Chengdu University of Information Technology, advised by Jing Hu. During the school, I participated in the project of performing image registration via reinforcement learning, under the advising of Xin Wang and Siwei Lyu from the University at Buffalo, SUNY. Before that, I obtained the Bachelor degree from Hebei University of Technology. I had been a research assistant at Megvii Technology Ltd., China, working with Shuaicheng Liu.

🔥 News

• 2024.01:  🎉🎉 One paper is accepted by International Conference on Learning Representations (ICLR 2024).
• 2023.04:  🎉🎉 One paper is accepted by International Conference on Machine Learning (ICML 2023).
• 2023.04:  🎉🎉 We win 2nd place in the NTIRE 2023 Shadow Removal Challenge (1st place on perceptual scores).
• 2022.04:  🎉🎉 We win 1st place in the NTIRE 2022 Burst Super-Resolution Challenge (Real-World Track).
• 2022.03:  🎉🎉 One paper is accepted by IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2022).

📝 Publications

Controlling Vision-Language Models for Multi-Task Image Restoration
Ziwei Luo, Fredrik K. Gustafsson, Zheng Zhao, Jens Sjölund, Thomas B. Schön

ICLR 2024 | Paper | Code | Project

• This work presents a degradation-aware vision-language model (DA-CLIP) for multi-task image restoration.
• A controller predicts degradation types and also adapts the fixed CLIP image encoder to predict high-quality feature embeddings.

Image Restoration with Mean-Reverting Stochastic Differential Equations
Ziwei Luo, Fredrik K. Gustafsson, Zheng Zhao, Jens Sjölund, Thomas B. Schön

ICML 2023 | Paper | Code | Project

• This work presents a stochastic differential equation (SDE) approach for general-purpose image restoration.
• A maximum likelihood objective is proposed to learn an optimal reverse trajectory which stabilizes the training and improves accuracy.

Refusion: Enabling Large-Size Realistic Image Restoration with Latent-Space Diffusion Models
Ziwei Luo, Fredrik K. Gustafsson, Zheng Zhao, Jens Sjölund, Thomas B. Schön

CVPRW 2023 | Paper | Code

• Winning solution of the NTIRE Image Shadow Removal Challenge 2023.
• An extention of the IR-SDE.

BSRT: Improving Burst Super-Resolution with Swin Transformer and Flow-Guided Deformable Alignment
Ziwei Luo, Youwei Li, Shen Cheng, Lei Yu, Qi Wu, Zhihong Wen, Haoqiang Fan, Jian Sun, Shuaicheng Liu

CVPRW 2022 | Paper | Code

• Champion of the NTIRE Burst Super-Resolution Challenge 2022 in Real track.

Deep Constrained Least Squares for Blind Image Super-Resolution
Ziwei Luo, Haibin Huang, Lei Yu, Youwei Li, Haoqiang Fan, Shuaicheng Liu

CVPR 2022 | Paper | Code

• This work proposes to disentangle deblurring and upsampling in blind super-resolution, and provides a theoretical guidance to make use of the kernel.

Stochastic Planner-Actor-Critic for Unsupervised Deformable Image Registration
Ziwei Luo*, Jing Hu*, Xin Wang, Shu Hu, Bin Kong, Youbing Yin, Qi Song, Xi Wu, Siwei Lyu

AAAI 2022 | Paper | Code

• It is the first work that performs deformable image registration via high-dimensional action deep reinforcement learning.

EBSR: Feature enhanced burst super-resolution with deformable alignment
Ziwei Luo, Lei Yu, Xuan Mo, Youwei Li, Lanpeng Jia, Haoqiang Fan, Jian Sun, Shuaicheng Liu

CVPRW 2021 | Paper | Code

• Champion of the NTIRE Burst Super-Resolution Challenge 2021 in Real track

• Stochastic Actor-Executor-Critic for Image-to-Image Translation
  Ziwei Luo*, Jing Hu*, Xin Wang, Siwei Lyu, Bin Kong, Youbing Yin, Qi Song, Xi Wu
  IJCAI 2021 | [Paper]

• End-to-end multimodal image registration via reinforcement learning
  Jing Hu, Ziwei Luo, Xin Wang, Shanhui Sun, Youbing Yin, Kunlin Cao, Qi Song, Siwei Lyu, Xi Wu
  Medical Image Analysis 2021 | [Paper]

• A Spatiotemporal Agent for Robust Multimodal Registration
  Ziwei Luo, Xin Wang, Xi Wu, Youbing Yin, Kunlin Cao, Qi Song, Jing Hu
  IEEE ACCESS 2020 | [Paper]

🎖 Honors and Awards

• 2023 2nd place in NTIRE Shadow Removal Challenge (1st place on perceptual scores)
• 2021, 2022 Champion in NTIRE Burst Super-Resolution Challenge (Real track)
• 2021 Outstanding Graduates, Chengdu University of Information Technology
• 2021 Award of Scientific Thesis Excellence, Chengdu University of Information Technology
• 2020 The First Prize Scholarship, Chengdu University of Information Technology
• 2020 National Scholarship, China

📖 Educations

• 2022.09 - present, PhD student, Uppsala University.
• 2018.09 - 2021.06, Master, Chengdu University of Information Technology.
• 2011.09 - 2015.06, Undergraduate, Hebei University of Technology.

💬 Teaching

• Teaching Assistant in the course Advanced Probabilistic Machine Learning, at Uppsala University
• Teaching Assistant in the course Statistical Machine Learning, at Uppsala University
• Teaching Assistant in the course Foundations of Computer Science, at Chengdu University of Information Technology

📫 Academic Services

Journal Reviewer

• IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)
• IEEE Transactions on Image Processing (TIP)
• IEEE Transactions on Circuits and Systems for Video Technology (TCSVT)

Conference Reviewer

• International Conference on Intelligent Robots and Systems (IROS) 2022
• IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2024

💻 Experience

• 2022.09 - present, PhD student, at Uppsala university, Sweden.
• 2021.07 - 2022.07, Research Assistant, at Megvii Technology, China.