Yu Zeng, 曾昱, Ph.D.

I am a Research Scientist at NVIDIA. My research interest lies in computer vision and deep learning. My research advances artificial intelligence through two primary directions: generative AI and label-efficient learning for computer vision. My PhD research focused on generative models for visual synthesis with multimodal and hierarchical inputs. My most recent research focuses on advancing embodied AI using deep generative models. My research has been integrated into several products at NVIDIA, including NVIDIA Cosmos and NVIDIA Edify.

Before joining NVIDIA, I obtained my PhD from Johns Hopkins University. Before that, I worked as a researcher at Tencent. I also worked with the research teams at Adobe during my PhD and Master’s studies.

News

• Our work on text-to-image generation was accepted by CVPR 2024 (paper, page)
• Our work on portrait relighting was accepted by CVPR 2024 (paper , page)
• I was selected as one of the Rising Stars in AI by KAUST AI Initiative
• Our work on image inpainting was accepted by AAAI 2024 (coming soon)

Selected Articles | Full List | Google Scholar

Cosmos: World Foundation Model Platform for Physical AI. NVIDIA Tech report. Best AI and Best Overall of CES 2025
Core contributor
[pdf] | [Porject Page]

Edify Image: High-Quality Image Generation with Pixel Space Laplacian Diffusion Models. NVIDIA Tech report
Core contributor
[pdf] | [Porject Page]

One-step diffusion policy: Fast visuomotor policies via diffusion distillation. arXiv preprint, 2024.
Z. Wang, Z. Li, A. Mandlekar, Z. Xu, J. Fan, Y. Narang, L. Fan, Y. Zhu, Y. Balaji, M. Zhou, M.-Y. Liu, Y. Zeng
[pdf] | [Porject Page]

JeDi: Joint-Image Diffusion Models for Finetuning-free Personalized Text-to-Image generation. CVPR. 2024.
Yu Zeng, Vishal M. Patel, Haocheng Wang, Xun Huang, Ting-Chun Wang, Ming-Yu Liu, Yogesh Balaji
pdf | Project Page

SceneComposer: Any-Level Semantic Image Synthesis. CVPR. 2023. (Highlight, top 2.5% submission)
Yu Zeng, Zhe Lin, Jianming Zhang, Qing Liu, John Collomosse, Jason Kuen, Vishal M. Patel
pdf | Project Page

SketchEdit: Mask-Free Local Image Manipulation with Partial Sketches. CVPR. 2022.
Yu Zeng, Zhe Lin, Vishal M. Patel
pdf | Project | Demo | Code

CR-Fill: Generative Image Inpainting with Auxiliary Contextual Reconstruction. ICCV. 2021.
Yu Zeng, Zhe Lin, Huchuan Lu, Vishal M. Patel
pdf | Code

High-Resolution Image Inpainting with Iterative Confidence Feedback and Guided Upsampling. ECCV. 2020.
Yu Zeng, Zhe Lin, Jimei Yang, Jianming Zhang, Eli Shechtman, Huchuan Lu
pdf | Project | Demo

Joint Learning of Saliency Detection and Weakly Supervised Semantic Segmentation. ICCV. 2019.
Zeng, Yu and Zhuge, Yunzhi and Lu, Huchuan and Zhang, Lihe
pdf | Code

Multi-source Weak Supervision for Saliency Detection. CVPR. 2019.
Zeng, Yu and Zhuge, Yunzhi and Lu, Huchuan and Zhang, Lihe and Qian, Mingyang and Yu, Yizhou
pdf | Code
Learning to Detect Salient Object with Multi-source Weak Supervision. TPAMI. 2021.
H Zhang, Y Zeng, H Lu, L Zhang, J Li, J Qi
pdf | Code

Learning to Promote Saliency Detectors. CVPR. 2018.
Yu Zeng, Huchuan Lu, Lihe Zhang, Mengyang Feng, Ali Borji
pdf | Code