A year spent in artificial intelligence is enough to make one believe in God.
-- Alan Perlis
Welcome to my personal website!
My name is Zhiwei Jia (贾志伟). I am a fourth-year Ph.D. student in Computer Science at UC San Diego (previously an undergrad here). I am luckily advised by prof. Hao Su and have been working with Prof. Zhuowen Tu.
Generally speaking, I am interested in developing new machine learning algorithms for better generalization, especially for problems in Embodied AI including multi-modal learning and robot learning.
Publications & Preprints
Improving Policy Optimization with Generalist-Specialist LearningZhiwei Jia, Xuanlin Li, Zhan Ling, Shuang Liu, Yiran Wu, Hao SuICML 2022 [arXiv]
We tackle the large-scale policy optimization problem with a novel framework called GSL that utilizes both joint training and distributed specialist training to ease policy learning. We consider GSL as a step towards generalizable policies.
Learning to Act with Affordance-Aware Multimodal Neural SLAMZhiwei Jia, Kaixiang Lin, Yizhou Zhao, Qiaozi Gao, Govind Thattai, Gaurav SukhatmeIROS 2022 [arXiv]
We designed a framework that employs multimodal exploration to acquire an affordance-aware semantic representation for solving complex long-horizon indoor tasks. We achieved very competitive results on the ALFRED Challenge.
LUMINOUS: Indoor Scene Generation for Embodied AI ChallengesYizhou Zhao, Kaixiang Lin, Zhiwei Jia, Qiaozi Gao, Govind Thattai, Jesse Thomason, Gaurav SukhatmeNeurips 2022 (CtrlGen Workshop) [arXiv]
We developed a framework to synthesize large-scale simulated scenes via randomization for training and evaluating Embodied AI challenges.
ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale DemonstrationsTongzhou Mu, Zhan Ling, Fanbo Xiang, Derek Yang, Xuanlin Li, Stone Tao, Zhiao Huang, Zhiwei Jia, Hao SuNeurIPS 2021 (Dataset Track) [arXiv]
We proposed a benchmark for generalizable physical object manipulation from 3D visual inputs. It features large intra-class topological and geometric variations, carefully designed tasks and a large number of demonstrations.
Semantically Robust Unpaired Image Translation for Data with Unmatched Semantics StatisticsZhiwei Jia, Bodi Yuan, Kangkang Wang, Hong Wu, David Clifford, Zhiqiang Yuan, Hao SuICCV 2021 [arXiv]
We proposed a novel multi-scale "semantic robustness" loss for GAN-based image translation models to reduce semantics flipping that is common in unpaired image-to-image translation tasks.
Refactoring Policy for Compositional Generalizability using Self-Supervised Object ProposalsTongzhou Mu, Jiayuan Gu, Zhiwei Jia, Hao Tang, Hao Su NeurIPS 2020 [arXiv] [code]
We proposed a two-stage framework to achieve compositional generalization in RL tasks by refactoring a teacher policy into a much more generalizable student policy with the help of strong inductive bias.
One-pixel Signature: Characterizing CNN Classifiers for Backdoor DetectionShanjiaoyang Huang, Weiqi Peng, Zhiwei Jia, Zhuowen Tu ECCV 2020 [arXiv]
We proposed a model-agnostic metric, namely One-pixel Signature, that can be used to effectively detect backdoored CNN. Our method achieves a substantial improvement (~30% in the absolute detection accuracy) over the current state-of-the-art approaches.
Information-Theoretic Local Minima Characterization and RegularizationZhiwei Jia, Hao Su ICML 2020 [arXiv] [code]
We proposed a metric of neural network minima that is both strongly indicative of its generalizability and may be effectively applied as a practical regularizer with both theoretical and empirical justifications.
Research Intern @ Google AI (06/2022 ~ 09/2022)Currently working on visual metaphor understanding.
Research Intern @ Amazon Alexa AI (06/2021 ~ 09/2021)Proposed a new multi-modal neural SLAM-basd method that achieved State-of-the-art performance for ALFRED (an indoor navigation & interaction challenge). Submission in preparation.
Research Intern @ Google [x] (06/2020 ~ 09/2020)Proposed a novel multi-scale "semantic robustness" loss for GAN-based image translation models to reduce semantics flipping that is common in unpaired image-to-image translation tasks.
Software Engineer Intern @ Quora (06/2018 ~ 09/2018 & 06/2019 ~ 09/2019)Developed a novel text embedding method by combining BERT and a deep tree classifier to handle extreme-scale multi-label text classification.
Software Engineer Intern @ Google (06/2017 ~ 09/2017)Worked on an applied machine learning project.
Ph.D in Computer Science @ UC San Diego09/2018 ~ present
B.S. in Computer Science and in Applied Math @ UC San Diego09/2014 ~ 12/2017cGPA: 3.85/4.00