We designed wide-FoV RGBD imagers with off-aperture phase encodings, optimized with hybrid refractive-diffractive optics. Our approach explores additional degree of freedom and provides local wavefront control, achieving over 5dB PSNR gain on 45 degrees FoV.
@article{wei2025learned,author={Wei, Haoyu and Liu, Xin and Liu, Yuhui and Fu, Qiang and Heidrich, Wolfgang and Lam, Edmund Y and Peng, Yifan (Evan)},title={Learned Off-aperture Encoding for Wide Field-of-view RGBD Imaging},journal={Accepted to IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), Unpublished},year={2025},}
2024
Learned Multi-aperture Color-coded Optics for Snapshot Hyperspectral Imaging
Zheng Shi, Xiong Dun, Haoyu Wei, Shiyu Dong, Zhanshan Wang, Xinbin Cheng, Felix Heide, and Yifan (Evan) Peng
We propose a snapshot hyperspectral imager with multi-aperture color-coded optics, optimized with an image reconstruction network. Our approach significantly enhances optical encoding, achieving over a 5dB improvement in PSNR for spectral reconstruction, and successfully recovers up to 31 spectral bands from 429 to 700 nm across diverse environments.
@article{ArrayHSI2024,author={Shi, Zheng and Dun, Xiong and Wei, Haoyu and Dong, Shiyu and Wang, Zhanshan and Cheng, Xinbin and Heide, Felix and Peng, Yifan (Evan)},title={Learned Multi-aperture Color-coded Optics for Snapshot Hyperspectral Imaging},year={2024},journal={ACM Transactions on Graphics (ToG)},publisher={Association for Computing Machinery},volume={43},number={6},articleno={208},issn={0730-0301},month=dec,paper={https://doi.org/10.1145/3687976},}
2023
Learned Large Field-of-View Imaging with Efficient Shift-Variant Wave Optics Modeling
Haoyu Wei, Xin Liu, Edmund Y Lam, and Yifan Peng
In Computational Optical Sensing and Imaging, Dec 2023
@inproceedings{wei2023learned,title={Learned Large Field-of-View Imaging with Efficient Shift-Variant Wave Optics Modeling},author={Wei, Haoyu and Liu, Xin and Lam, Edmund Y and Peng, Yifan},booktitle={Computational Optical Sensing and Imaging},pages={CTu3B--3},year={2023},organization={Optica Publishing Group},paper={https://opg.optica.org/abstract.cfm?uri=COSI-2023-CTu3B.3}}
Modeling Off-Axis Diffraction with the Least-Sampling Angular Spectrum Method
(Optica Top downloads in Jul and Aug)
Haoyu Wei, Xin Liu, Xiang Hao, Edmund Y Lam, and Yifan Peng
We propose the Least-Sampling ASM that minimizes and unifies the sampling requirements for all types of input fields. Ultra-fast computations at ultra-large angles (35 degrees) are demonstrated. High-frequency modulation such as diffusers is also considered.
@article{wei2023modeling,title={Modeling Off-Axis Diffraction with the Least-Sampling Angular Spectrum Method <p style="color:Red;">(Optica Top downloads in Jul and Aug)</p>},author={Wei, Haoyu and Liu, Xin and Hao, Xiang and Lam, Edmund Y and Peng, Yifan},journal={Optica},volume={10},number={7},pages={959--962},publisher={Optica Publishing Group},year={2023},month=jul,doi={10.1364/OPTICA.490223},paper={https://opg.optica.org/optica/fulltext.cfm?uri=optica-10-7-959&id=535343},}
2022
Millisecond Autofocusing Microscopy Using Neuromorphic Event Sensing
Zhou Ge, Haoyu Wei, Feng Xu, Yizhao Gao, Zhiqin Chu, Hayden K-H So, and Edmund Y Lam
Rapid autofocusing is essential for many microscopic imaging applications. Existing methods either require complicated hardware implementations or slow -stack image acquisition. In this work, we develop a new approach to achieve fast autofocusing using the neuromorphic event sensing technology, which detects the sparse brightness changes asynchronously and responds quickly to the specimen movement. A simple yet efficient autofocusing system is tailored for fast acquisition and processing of the non-redundant event data, allowing for autofocusing in only tens of milliseconds, which is thousands of times faster than current technologies. Experimental results show a substantial performance improvement and capability for biopsy specimen inspections. To the best of our knowledge, this is the first reported work on a neuromorphic methodology for autofocusing microscopy.
@article{ge:2023:millisecond,title={Millisecond Autofocusing Microscopy Using Neuromorphic Event Sensing},author={Ge, Zhou and Wei, Haoyu and Xu, Feng and Gao, Yizhao and Chu, Zhiqin and So, Hayden K-H and Lam, Edmund Y},journal={Optics and Lasers in Engineering},volume={160},pages={107247},year={2022},publisher={Elsevier},paper={https://doi.org/10.1016/j.optlaseng.2022.107247}}
@inproceedings{ge:2022:event,title={Event-Based Automatic Focusing Under Photon-Limited Conditions},author={Ge, Zhou and Wei, Haoyu and Lam, Edmund Y},booktitle={Computational Optical Sensing and Imaging},pages={CM4A--2},year={2022},organization={Optica Publishing Group},paper={https://opg.optica.org/abstract.cfm?uri=COSI-2022-CM4A.2}}
2020
2-Step Sparse-View CT Reconstruction with a Domain-Specific Perceptual Network
Haoyu Wei, Florian Schiffers, Tobias Würfl, Daming Shen, Daniel Kim, Aggelos K Katsaggelos, and Oliver Cossairt
Reconstruction of Computed Tomography (CT) and Magnetic Resonance Imaging (MRI) images using highly undersampled data.
@article{wei:2020:2,title={2-Step Sparse-View CT Reconstruction with a Domain-Specific Perceptual Network},author={Wei, Haoyu and Schiffers, Florian and W{\"u}rfl, Tobias and Shen, Daming and Kim, Daniel and Katsaggelos, Aggelos K and Cossairt, Oliver},journal={arXiv},year={2020},}
Learned Large Field-of-View Imaging with Efficient Shift-Variant Wave Optics ModelingIn Computational Optical Sensing and Imaging, Dec 2023
