I'm a PhD student in Tsinghua University, supervised by Profs Jun Zhu and Bo Zhang.
I work in probablistic machine learning. I'm particularly interested in uncertainty quantification, Bayes-inspired ideas, and learning in out-of-distribution settings.
You can reach me through gmail at wzy196. My CV.
Other persons with the same name.
Quasi-Bayesian Dual Instrumental Variable Regression.
Ziyu Wang*, Yuhao Zhou*, Tongzheng Ren, Jun Zhu.
Short version in NeurIPS 2021. PDF
- Quasi-Bayesian inference for kernelized and (heuristically) NN-parameterized IV models, based on the dual/minimax formulation of IVR.
- Quasi-Bayes is needed for IV because we can't do Bayesian modeling, which is because we don't know the full data generating process.
- A guess will likely be wrong, and still difficult to make use of, because you will have to do Bayesian inference over deep [conditional] generative models.
- We establish optimal posterior contraction rates in L2 and Sobolev norms, and study frequentist validity of credible balls. These results improve the understanding of both quasi-Bayesian and kernelized IV methods.
- We also present an inference algorithm using a modified randomized prior trick, which enables application to wide NNs.
Fork or Fail: Cycle-Consistent Training with Many-to-One Mappings.
Qipeng Guo, Zhijing Jin, Ziyu Wang, Xipeng Qiu, Weinan Zhang, Jun Zhu, Zheng Zhang, David Wipf.
- Domain alignment without paired data, when bijections do not exist.
Further Analysis of Outlier Detection with Deep Generative Models.
Ziyu Wang, Bin Dai, David Wipf, Jun Zhu.
- This is about the observation that DGMs assign higher likelihood to semantically different outliers.
Intuitively this is due to concentration of measure/typicality ("Gaussian distributions are soap bubbles"), but it seemed difficult to confirm empirically.
- We argue previous attempts relied on tests that were more prone to estimation error, and propose a fix which connects to the idea of atypicality and the longitudinal view of high-dimensional data.
- A few other observations are difficult to summarize, so check out the paper if you're interested.
The Usual Suspects? Reassessing Blame for VAE Posterior Collapse.
Bin Dai, Ziyu Wang, David Wipf.
- Reasons for posterior collapse in nonlinear VAEs, which may or may not be similar to the linear case.
- Of particular importance is the practicality of designing AE architecture with low reconstruction errors.
A Wasserstein Minimum Velocity Approach to Learning Unnormalized Models.
Ziyu Wang, Shuyu Cheng, Yueru Li, Jun Zhu, Bo Zhang.
- An alternative approximation to the score matching objectives, which works with DNNs. And generalizations.
- The title was a tribute to the unpublished work "A Minimum Velocity Approach to Learning".
Function Space Particle Optimization for Bayesian Neural Networks.
Ziyu Wang, Tongzheng Ren, Jun Zhu, Bo Zhang.
- A curious SVGD/Particle VI-like algorithm, but in function space. GIF.
- The function-space view is important for overparameterized priors like BNNs, because there is a combinatorial number of local maximas in the "weight space", and you can't believe your inference algorithm covers them all.
Too bad we still haven't figured out how to do it properly in the most general case, after all these years.
- But if you can afford to train an ensemble of models, this works well in practice.
Reading group slides, for a general ML audience: