Chien-Yi Wang

I am a Senior Research Scientist at NVIDIA Research. Previously, I worked as a Senior Research SDE at Microsoft AI R&D Center in Taiwan. I had 8+ years of experience specializing in computer vision research, deep learning-based model optimization, and machine learning service integration. My research focus is mainly on cross-modality representation learning, face modeling, and 2D/3D scene understanding. Interested in revolutionizing a machine learning system from the bottom‑up, devising better problem‑solving methods for challenging tasks, and learning new technologies and tools if the need arises.

⭐ We are actively recruiting full-time research scientists / interns at NVIDIA Research Taiwan. Please drop me an email at chienyiw@nvidia.com if you are interested.

Email  /  CV  /  Google Scholar  /  LinkedIn  /  Twitter  /  GitHub

• [Feb. 2024] Our paper "MCPNet" is accepted at CVPR 2024.
• [Jan. 2024] One paper is accepted at ICRA 2024.
• [Jan. 2024] Our paper "RAPPER" is accepted at ICLR 2024.
• [Jul. 2023] One paper "pFedPG" is accepted at ICCV 2023.
• [Jun. 2023] "GAIN" received the best paper award for the CVPR 2023 Biometrics Workshop!
• [Jun. 2023] Our "QuAVF" work ranked 1st place winner at the 2023 CVPR Ego4D Challenge (Social Understanding: Talking-to-me track)!
• [Apr. 2023] One paper "GAIN" is accepted at CVPR 2023 Biometrics Workshop.
• [Jan. 2023] One paper is accepted at TBIOM.
• [Nov. 2022] Our paper "MixFairFace" is accepted at AAAI 2023.
• [Oct. 2022] One paper is accepted at FG 2023.
• [Oct. 2022] I joined NVIDIA Research as a Research Scientist.
• [Jun. 2022] I am selected as an outstanding reviewer for CVPR 2022!
• [Mar. 2022] Two papers "PatchNet" and "LaFR" are accepted at CVPR 2022.
• [Dec. 2021] Our paper "FedFR" is accepted at AAAI 2022 as Oral!
• [Oct. 2021] One paper is accepted at WACV 2022.
• [Sep. 2021] One paper is accepted at FG 2021.

I'm interested in computer vision and multi-modal representation learning.

We presented DoRA, a new parameter-efficient fine-tuning approach, which consistently outperforms LoRA in fine-tuning LLM without incurring additional inference costs. These improvements are particularly notable for smaller ranks with 37.2% improvement over LoRA for rank 8 and 22.4% improvement for rank 4.

We propose the Multi-Level Concept Prototypes Classifier (MCPNet), an inherently interpretable model, that can explain a classifier's prediction via concept prototypes from low-to-high layers in the CNN models.

We propose a Differentiable Multi-Sensor Kalman Filter for 3D Multi-Object Cooperative Tracking (DMSTrack), which is designed to be capable of estimating observation noise covariance of each detection from different Connected Autonomous Vehicles (CAVs) to better take advantage of the Kalman Filter’s theoretical optimality property.

We introduce Rapper, a two-stage Reinforced RationalePrompted Paradigm for Natural Language Explanation (NLE) in Visual Question Answering (VQA).

To leverage robust representations from large-scale models while enabling efficient model personalization for heterogeneous clients, we propose a novel personalized FL framework of client-specific Prompt Generation (pFedPG), which learns to deploy a personalized prompt generator at the server for producing client-specific visual prompts that efficiently adapts frozen backbones to local data distributions.

We propose the Quality-aware audio-visual fusion (QuAVF) framework which achieves 67.4% mean average precision (mAP) on the Ego4D talking-to-me (TTM) test set.

We propose Geometry-Aware Interaction Network (GAIN), which exploits dense facial landmarks with spatio-temporal graph convolutional network (ST-GCN) to establish a more interpretable and modularized Face Anti-Spoofing model.

We propose to augment facial segmentation and depth maps to assist the RGB-D face recognition task. With the multi-modal augmentation and identity-style disentanglement, the proposed RGB-D recognition model could achieve superior performance on several benchmarks.

We propose the MixFair Adapter to determine and reduce the identity bias of training samples. Besides, in order to push for ultimate fairness in face recognition, we propose a new evaluation protocol to fairly evaluate the fairness performance of different approaches.

We introduce a multi-task meta-learning framework for learning more generalized features for face anti-spoofing.

We propose PatchNet which reformulates face anti-spoofing as a fine-grained patch-type recognition problem.

We introduce a new problem setup called "local-adaptive face recognition (LaFR)" and proposed the clustering and adaptation modules to address face recognition in unseen environments.

We propose a Federated Learning based framework called FedFR to improve the generic face representation in a privacy-aware manner. Besides, the framework jointly optimizes personalized models for the corresponding clients via the proposed Decoupled Feature Customization module.

We propose a novel dual-stage disentangled representation learning method that can efficiently untangle spoof-related features from irrelevant ones. Unlike previous FAS disentanglement works with one-stage architecture, we found that the dual-stage training design can improve the training stability and effectively encode the features to detect unseen attack types.

We propose to leverage pseudo facial segmentation and depth maps to assist the RGB-D face recognition task. With the multi-modal augmentation, the proposed mask-guided RGB-D recognition model could achieve superior performance on several benchmarks.

We propose a unified representation learning framework to address the Cross Model Compatibility (CMC) problem in the context of visual search applications. The method can be applied onto face recognition and person re-identification tasks.

We propose a 3D dynamic scene analysis framework as the first step toward driving scene understanding. Specifically, given a sequence of synchronized 2D and 3D sensory data, the framework systematically integrates different perception modules to obtain 3D position, orientation, velocity and category of traffic participants and the ego car in a reconstructed 3D semantically labeled traffic scene.

The contour-guided color palette (CCP) is proposed for robust image segmentation. It efficiently integrates contour and color cues of an image.

Design and source code from Jon Barron's website
Last updated January 2023.