I am a PhD candidate at UC Berkeley, advised by Prof. Stella Yu and Prof. Meng Lin. My research lies at the intersection of machine learning, computer vision and AI for healthcare. I received my B.E in Electronic Engineering from Xi'an Jiaotong University in 2018, where I worked with Prof. Jinjun Wang at the Institute of Artificial Intelligence and Robotics.

News! I am on schedule to graduate in May 2023.

My research lies at the intersection of machine learning, computer vision and AI for healthcare. Specifically, I study representation learning for real-world applications. Representation learning brings the data structures explicit and its advantages include robust model performance and high interpretability. Real-world data, compared to synthetic or curated data, usually comes with very few labels, and are noisy and imbalanced.

I study structure-aware representation learning to address these fundamental challenges in real world applications. First, it is aware of real-world data structures which are unlabeled and highly imbalanced and achieves robust performance compared to existing representation learning methods. Second, it learns hierarchical structures of multi-level representations, such as high-level semantics, mid-level geometry and low-level materials, which leads to better model interpretation. Whereas existing methods are coarse and incomplete as they only learn high-level representations. We demonstrate superior performance in real-world medical and ecological data. See more details here.

Unsupervised Scene Sketch to Photo Synthesis
Jiayun Wang, Sangryul Jeon, Stella X. Yu, Xi Zhang, Himanshu Arora, Yu Lou.
ECCVW 2022. [project] [slides] [paper] [code] [media coverage]

Spatial Transformer for 3D Point Clouds
Jiayun Wang, Rudrasis Chakraborty, Stella X. Yu.
IEEE TPAMI 2021 [project] [video] [poster] [paper] [code]

We propose to learn different non-rigid transformations of the input point cloud for different local neighborhoods at each layer.

Orthogonal Convolutional Neural Networks
Jiayun Wang, Yubei Chen, Rudrasis Chakraborty, Stella X. Yu.
CVPR 2020. [project] [video] [slides] [paper] [code] [blog (Chinese)]

Orthogonal convolutional neural networks is a light-cost regularizer which reduces the feature redundancy and improves network performance and robustness under attack.

Sur-Real: Frechet Mean and Distance Transform for Complex-Valued Deep Learning
Rudrasis Chakraborty, Jiayun Wang, Stella X. Yu.
CVPRW 2019. Best Paper [paper] [poster] [code]

This work develops a novel deep learning architecture for naturally complex-valued data, with improved results and only 10% of the parameters as the baseline model.

A Deep Learning Approach for Meibomian Gland Atrophy Evaluation in Meibography Images
Jiayun Wang, Thao N. Yeh, Rudrasis Chakraborty, Stella X. Yu, Meng C. Lin.
Translational Vision Science and Technology 2019. [paper] [code]

This work develops a deep learning approach to digitally segmenting meibomian gland atrophy area and computing percent atrophy in meibography images.

Large-scale Long-Tailed Recognition in an Open World
Ziwei Liu, Zhongqi Miao, Xiaohang Zhan, Jiayun Wang, Boqing Gong, Stella X. Yu.
CVPR 2019 Oral. [project] [paper] [code] [blog]

This work formally defines the problem of open long-tailed recognition (OLTR) as optimizing for the overall accuracy of a naturally-distributed dataset with the presence of open classes.

Insights and Approaches Using Deep Learning to Classify Wildlife
Zhongqi Miao, Kaitlyn M Gaynor, Jiayun Wang, Ziwei Liu, Oliver Muellerklein, Mohammad S. Norouzzadeh, Alex McInturff, Rauri C. K. Bowie, Ran Nathon, Stella X. Yu, Wayne M. Getz.
Nature - Scientific Reports 2019. [Paper]

This work aims to interpret the concepts behind the convolutional neural networks (CNNs) in classifying wildlife.

Deep Ranking Model by Large Adaptive Margin Learning for Person Re-identification
Jiayun Wang, Sanping Zhou, Jinjun Wang, Qiqi Hou.
Pattern Recognition 2018. [paper] [arxiv] [code]

This work designs a dynamically adaptive loss function to overcome the drawbacks of conventional loss functions for person re-identification.

Successive Embedding and Classification Loss for Aerial Image Classification
Jiayun Wang, Patrick Virtue, Stella X. Yu.
arXiv 2017. [paper] [code]

This work aims to address the overfitting problem in remote sensing image classification.

Point to Set Similarity Based Deep Feature Learning for Person Re-identification
Sanping Zhou, Jinjun Wang, Jiayun Wang, Yihong Gong, Nanning Zheng.
CVPR 2017. [paper] [code]

This work presents a novel metric learning method based on P2S similarity comparison for person re-identification.

At Berkeley, I taught the following classes as a graduate student instructor:
      CS182/282, Deep Learning, Spring 2023
      CS189/289, Machine Learning, Fall 2020
      VS205, Visual Perception, Fall 2019, 2018

I gave talks at seminars of Apple, Amazon, Bosch, Oxyopia, etc. You could watch some of the conferencepresentations of my works on YouTube:
      Towards Real-World Structure-Aware Representation Learning, 2023
      Compact and Optimal Deep Learning with Recurrent Parameter Generators, 2022
      Spatial Transformer for 3D Point Clouds, 2021
      Orthogonal Convolutional Neural Networks, 2020