Yunsheng Li is a Senior Researcher Microsoft Azure GenAI Group. He is working on the development for the multi-modality large language model at Microsoft. His research interests include computer vision (segmentation, domain adaptation), deep learning (network architecture design) and multi-modality large language models. His representative works include phi-3-vision, MicroNet, and BDL.

Research Experiences

• 2024-present: Working on the supervised fine-tuning for Phi-3-vision and Phi-3.5-vision project, developing one of the best “small” multi-modal LLMs.
• 2022-2023: Design the background removal models that are used in Windows Paint and Photo and Microsoft Desiger
• 2015-2021: A Ph.D. student in University of California, San Diego, working on overcoming the resource constrained computer vision related topics, e.g., efficient neural network architecture design and domain adaptation.

Publications

1. Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
   Microsoft GenAI team
   [arXiv] [HuggingFace]
2. Rethinking Visual Prompting for Multimodal Large Language Models with External Knowledge
   Yuanze Lin, Yunsheng Li, Dongdong Chen, Weijian Xu, Ronald Clark, Philip Torr, Lu Yuan
   [arXiv]
3. SCHEME: Scalable Channer Mixer for Vision Transformers
   Deepak Sridhar, Yunsheng Li, Nuno Vasconcelos
   [arXiv]
4. Fully authentic visual question answering dataset from online communities
   Chongyan Chen, Mengchen Liu, Noel Codella, Yunsheng Li, Lu Yuan, Danna Gurari
   European Conference on Computer Vision (ECCV), 2024
   [Paper] [Website]
5. Dense network expansion for class incremental learning
   Zhiyuan Hu, Yunsheng Li, Jiancheng Lyu, Dashan Gao, Nuno Vasconcelos
   Conference on Computer Vision and Pattern Recognition (CVPR), 2023
   [Paper]
6. Should all proposals be treated equally in object detection?
   Yunsheng Li, Yinpeng Chen, Xiyang Dai, Dongdong Chen, Mengchen Liu, Pei Yu, Ying Jin, Lu Yuan, Zicheng Liu, Nuno Vasconcelos
   European Conference on Computer Vision (ECCV), 2022
   [Paper] [Code]
7. MicroNet: Towards image recognition with extremely low FLOPs
   Yunsheng Li, Yinpeng Chen, Xiyang Dai, Dongdong Chen, Mengchen Liu, Lu Yuan, Zicheng Liu, Lei Zhang, Nuno Vasconcelos
   International Conference on Computer Vision (*ICCV**), 2021
   [Paper] [Website]
8. Dynamic Transfer for Multi-Source Domain Adaptation
   Yunsheng Li, Lu Yuan, Yinpeng Chen, Pei Wang, Nuno Vasconcelos
   Conference on Computer Vision and Pattern Recognition (CVPR), 2021
   [Paper] [Website]
9. Revisiting Dynamic Convolution via Matrix Decomposition
   Yunsheng Li, Yinpeng Chen, Xiyang Dai, Mengchen Liu, Dongdong Chen, Ye Yu, Zicheng Liu, Mei Chen, Lu Yuan, Nuno Vasconcelos
   International Conference on Learning Representations (ICLR), 2021
   [Paper] [Website]
10. Explainable Object-induced Action Decision for Autonomous Vehicles
    Yiran Xu, Xiaoyin Yang, Lihang Gong, Hsuan-Chu Lin, Tz-Ying Wu, Yungsheng Li, Nuno Vasconcelos
    Conference on Computer Vision and Pattern Recognition (CVPR), 2020
    [Paper] [Website]
11. Bidirectional Learning for Domain Adaptation of Semantic Segmentation
    Yunsheng Li, Lu Yuan, Nuno Vasconcelos
    Conference on Computer Vision and Pattern Recognition (CVPR), 2019
    [Paper] [Website]
12. Efficient Multi-Domain Learning by Covariance Normalization
    Yunsheng Li Nuno Vasconcelos
    Conference on Computer Vision and Pattern Recognition (CVPR), 2019
    [Paper] [Website]
13. Deep scene image classification with the mfafvnet
    Yunsheng Li, Mandar Dixit, Nuno Vasconcelos
    International Conference on Computer Vision (ICCV), 2017
    [Paper] [Website]
14. Deep Hashing with Hash-Consistent Large Margin Proxy Embeddings
    Pedro Morgado, Yunsheng Li, Jose Costa Pereira, Mohammad Saberian, Nuno Vasconcelos
    International Journal of Computer Vision (IJCV)
    [Paper]
15. Semantic Fisher Scores for Task Transfer: Using Objects to Classify Scenes
    Mandar Dixit, Yunsheng Li, Nuno Vasconcelos
    IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)
    [Paper]

Professinonal Activities

• Conference reviewer: CVPR, ICCV, ECCV, ICLR, and et.al.
• Journal reviewer: T-PAMI, T-MM, IJCV, and et.al.