I am a postdoctoral researcher at California Institute of Technology, working with Prof. Anima Anandkumar. Prior to joining Caltech, I completed my Ph.D. at Vision Science and Berkeley AI Research (BAIR), UC Berkeley, advised by Prof. Stella Yu and Prof. Meng Lin. My research lies at the intersection of machine learning, computer vision and AI for healthcare.

Research Interests

My research lies at the intersection of machine learning, computer vision and AI for healthcare. Specifically, I study representation learning for real-world applications. Representation learning brings the data structures explicit and its advantages include robust model performance and high interpretability. Real-world data, compared to synthetic or curated data, usually comes with very few labels, and are noisy and imbalanced.

I study structure-aware representation learning to address these fundamental challenges in real world applications. First, it is aware of real-world data structures which are unlabeled and highly imbalanced and achieves robust performance compared to existing representation learning methods. Second, it learns hierarchical structures of multi-level representations, such as high-level semantics, mid-level geometry and low-level materials, which leads to better model interpretation. Whereas existing methods are coarse and incomplete as they only learn high-level representations. We demonstrate superior performance in real-world medical and ecological data. See more details here.

Publications

Human Reposing and Virtual Try-On from Multi-View Images
 Jiayun Wang, Amin Kheradmand, Himanshu Arora.
WACV 2024. [paper]



Compact and Optimal Deep Learning with Recurrent Parameter Generators
 Jiayun Wang, Yubei Chen, Stella X. Yu, Brian Cheung, Yann LeCunn.
WACV 2023. [project] [video] [slides] [paper] [code] [blog] [media coverage (Chinese)]



Predicting demographics from meibography using deep learning
Jiayun Wang, Andrew D. Graham, Stella X. Yu, Meng C. Lin.
Nature - Scientific Reports 2022. [paper] [slides] [media coverage]



Open Long-Tailed Recognition in A Dynamic World
Ziwei Liu, Zhongqi Miao, Xiaohang Zhan, Jiayun Wang, Boqing Gong, Stella X. Yu.
IEEE TPAMI 2022. [paper][slides] [code]



Unsupervised Scene Sketch to Photo Synthesis
 Jiayun Wang, Sangryul Jeon, Stella X. Yu, Xi Zhang, Himanshu Arora, Yu Lou.
ECCVW 2022. [project] [slides] [paper] [code] [media coverage]



3D Shape Reconstruction from Free-Hand Sketches
 Jiayun Wang, Jierui Lin, Qian Yu, Runtao Liu, Yubei Chen, Stella X. Yu.
ECCVW 2022. [paper] [slides] [code]



Quantifying Meibomian Gland Morphology Using Artificial Intelligence
Jiayun Wang, Shixuan Li, Thao N. Yeh, Rudrasis Chakraborty, Andrew D Graham, Stella X. Yu, Meng C. Lin.
Optometry and Vision Science 2021. [paper] [code]



Spatial Transformer for 3D Point Clouds
 Jiayun Wang, Rudrasis Chakraborty, Stella X. Yu.
IEEE TPAMI 2021 [project] [video] [poster] [paper] [code]

We propose to learn different non-rigid transformations of the input point cloud for different local neighborhoods at each layer.

Orthogonal Convolutional Neural Networks
 Jiayun Wang, Yubei Chen, Rudrasis Chakraborty, Stella X. Yu.
CVPR 2020. [project] [video] [slides] [paper] [code] [blog (Chinese)]

Orthogonal convolutional neural networks is a light-cost regularizer which reduces the feature redundancy and improves network performance and robustness under attack.

Sur-Real: Frechet Mean and Distance Transform for Complex-Valued Deep Learning
 Rudrasis Chakraborty, Jiayun Wang, Stella X. Yu.
CVPRW 2019. Best Paper [paper] [poster] [code]

This work develops a novel deep learning architecture for naturally complex-valued data, with improved results and only 10% of the parameters as the baseline model.

A Deep Learning Approach for Meibomian Gland Atrophy Evaluation in Meibography Images
Jiayun Wang, Thao N. Yeh, Rudrasis Chakraborty, Stella X. Yu, Meng C. Lin.
Translational Vision Science and Technology 2019. [paper] [code]

This work develops a deep learning approach to digitally segmenting meibomian gland atrophy area and computing percent atrophy in meibography images.

Large-scale Long-Tailed Recognition in an Open World
Ziwei Liu, Zhongqi Miao, Xiaohang Zhan, Jiayun Wang, Boqing Gong, Stella X. Yu.
CVPR 2019 Oral. [project] [paper] [code] [blog]

This work formally defines the problem of open long-tailed recognition (OLTR) as optimizing for the overall accuracy of a naturally-distributed dataset with the presence of open classes.

Insights and Approaches Using Deep Learning to Classify Wildlife
 Zhongqi Miao, Kaitlyn M Gaynor, Jiayun Wang, Ziwei Liu, Oliver Muellerklein, Mohammad S. Norouzzadeh, Alex McInturff, Rauri C. K. Bowie, Ran Nathon, Stella X. Yu, Wayne M. Getz.
Nature - Scientific Reports 2019. [Paper]

This work aims to interpret the concepts behind the convolutional neural networks (CNNs) in classifying wildlife.

Deep Ranking Model by Large Adaptive Margin Learning for Person Re-identification
Jiayun Wang, Sanping Zhou, Jinjun Wang, Qiqi Hou.
Pattern Recognition 2018. [paper] [arxiv] [code]

This work designs a dynamically adaptive loss function to overcome the drawbacks of conventional loss functions for person re-identification.

Successive Embedding and Classification Loss for Aerial Image Classification
Jiayun Wang, Patrick Virtue, Stella X. Yu.
arXiv 2017. [paper] [code]

This work aims to address the overfitting problem in remote sensing image classification.

Point to Set Similarity Based Deep Feature Learning for Person Re-identification
Sanping Zhou, Jinjun Wang, Jiayun Wang, Yihong Gong, Nanning Zheng.
CVPR 2017. [paper] [code]

This work presents a novel metric learning method based on P2S similarity comparison for person re-identification.

Teaching

At Berkeley, I taught the following classes as a graduate student instructor:
      CS182/282, Deep Learning, Spring 2023
      CS189/289, Machine Learning, Fall 2020
      VS205, Visual Perception, Fall 2019, 2018

Talks

I gave talks at seminars of Apple, Amazon, Bosch, Oxyopia, etc. You could watch some of the conferencepresentations of my works on YouTube:
      Towards Real-World Structure-Aware Representation Learning, 2023
      Compact and Optimal Deep Learning with Recurrent Parameter Generators, 2022
      Spatial Transformer for 3D Point Clouds, 2021
      Orthogonal Convolutional Neural Networks, 2020