A year spent in artificial intelligence is enough to make one believe in God.

-- Alan Perlis

Welcome to my personal website!

My name is Zhiwei Jia (贾志伟). I am a fourth-year Ph.D. student in Computer Science at UC San Diego (previously an undergrad here). I am luckily advised by prof. Hao Su and have been working with Prof. Zhuowen Tu.

Generally speaking, I am interested in developing new machine learning algorithms for better generalization, especially for problems in Embodied AI.

Publications & Preprints

Improving Policy Optimization with Generalist-Specialist Learning

Zhiwei Jia, Xuanlin Li, Zhan Ling, Shuang Liu, Yiran Wu, Hao SuICML 2022 [Coming Soon]
We proposed a novel generalist-specialist training (GSL) framework for large-scale RL. We justified the key design choices in GSL and show that it effectively and consistently improves several baseline RL algorithms on multiple challenging benchmarks.

Learning to Act with Affordance-Aware Multimodal Neural SLAM

Zhiwei Jia, Kaixiang Lin, Yizhou Zhao, Qiaozi Gao, Govind Thattai, Gaurav Sukhatme[arXiv]
We designed a framework that employs multimodal exploration to acquire an affordance-aware semantic representation for solving complex long-horizon indoor tasks. We established a new SoTA results on ALFRED.

ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrations

Tongzhou Mu, Zhan Ling, Fanbo Xiang, Derek Yang, Xuanlin Li, Stone Tao, Zhiao Huang, Zhiwei Jia, Hao SuNeurIPS 2021 (Dataset Track) [arXiv]
We propose a benchmark for generalizable physical object manipulation from 3D visual inputs. It features large intra-class topological and geometric variations, carefully designed tasks and a large number of demonstrations.

Semantically Robust Unpaired Image Translation for Data with Unmatched Semantics Statistics

Zhiwei Jia, Bodi Yuan, Kangkang Wang, Hong Wu, David Clifford, Zhiqiang Yuan, Hao SuICCV 2021 [arXiv]
We proposed a novel multi-scale "semantic robustness" loss for GAN-based image translation models to reduce semantics flipping that is common in unpaired image-to-image translation tasks.

Refactoring Policy for Compositional Generalizability using Self-Supervised Object Proposals

Tongzhou Mu, Jiayuan Gu, Zhiwei Jia, Hao Tang, Hao Su NeurIPS 2020 [arXiv] [code]
We propose a two-stage framework to achieve compositional generalization in RL tasks by refactoring a teacher policy into a much more generalizable student policy with the help of strong inductive bias.

One-pixel Signature: Characterizing CNN Classifiers for Backdoor Detection

Shanjiaoyang Huang, Weiqi Peng, Zhiwei Jia, Zhuowen Tu ECCV 2020 [arXiv]
We propose a model-agnostic metric, namely One-pixel Signature, that can be used to effectively detect backdoored CNN. Our method achieves a substantial improvement (~30% in the absolute detection accuracy) over the current state-of-the-art approaches.

Information-Theoretic Local Minima Characterization and Regularization

Zhiwei Jia, Hao Su ICML 2020 [arXiv] [code]
We propose a metric of neural network minima that is both strongly indicative of its generalizability and may be effectively applied as a practical regularizer with both theoretical and empirical justifications.

Work Experience

Research Intern @ Amazon Alexa AI (06/2021 ~ 09/2021)

Proposed a new multi-modal neural SLAM-basd method that achieved State-of-the-art performance for ALFRED (an indoor navigation & interaction challenge). Submission in preparation.

Research Intern @ Google X (06/2020 ~ 09/2020)

Proposed a novel multi-scale "semantic robustness" loss for GAN-based image translation models to reduce semantics flipping that is common in unpaired image-to-image translation tasks.

Software Engineer Intern @ Quora (06/2018 ~ 09/2018 & 06/2019 ~ 09/2019)

Developed a novel tree-based method for deep text embedding that benefits extreme-scale multi-label text classification. Paper submission in preparation.

Software Engineering Intern @ Google (06/2017 ~ 09/2017)

Worked on an applied machine learning project.


Ph.D in Computer Science @ UC San Diego

09/2018 ~ present

B.S. in Computer Science and in Applied Math @ UC San Diego

09/2014 ~ 12/2017cGPA: 3.85/4.00


Email: zjia [at] ucsd [dot] edu