IV. Deep Learning for recovering the 3D structure of microdevices

In advanced micro-scale manufacturing, such as in the semiconductor assembly and packaging field, machine vision plays a key role in inspection and process control. Cost, throughput, and space limitations can however restrict the integration of complex 3D capture devices for certain applications. Active and passive stereoscopic imaging systems are the primary tools used for 3D inspection, but an attractive option is to obtain the 3D information computationally from conventional 2D cameras. Recently, this approach regained attention due to the possibilities offered by deep convolutional networks (CNN). A CNN model can be trained on pairs of 2D and 3D images, learning the geometric structure of the objects of interest. This can potentially turn ordinary industrial vision cameras into 3D acquisition devices. The goal of this project is to explore the application of DL to monocular 3D reconstruction, and to the recovery of high-quality depth maps from both single images and sequences (focal stacks or videos).   The internship tasks are outlines in the following:

  • Perform a literature search on the latest deep learning – based 3D reconstruction techniques.
  • Select two or three methods for evaluation and further development.
  • Train suitable 3D reconstruction models, using ASM and public datasets.
  • Explore the incorporation of prior knowledge about device geometry and materials in the reconstruction process.
  • Compile the results and analysis in a final and mid-term reports.

For more information about this project, please contact fboughorbel@alsi.asmpt.com  or call  +31 24 204 2824.

References

[1] Eigen, David, Christian Puhrsch, and Rob Fergus. “Depth map prediction from a single image using a multi-scale deep network.” In Advances in neural information processing systems, pp. 2366-2374. 2014.

[2] Garg, Ravi, Vijay Kumar BG, Gustavo Carneiro, and Ian Reid. “Unsupervised cnn for single view depth estimation: Geometry to the rescue.” In European Conference on Computer Vision, pp. 740-756. Springer, Cham, 2016.

[3] Rezende, Danilo Jimenez, SM Ali Eslami, Shakir Mohamed, Peter Battaglia, Max Jaderberg, and Nicolas Heess. “Unsupervised learning of 3d structure from images.” In Advances in Neural Information Processing Systems, pp. 4996-5004. 2016.

[5] Alhashim I, Wonka P. High quality monocular depth estimation via transfer learning. arXiv preprint arXiv:1812.11941. Dec 31, 2018.

[6] Zhou, Tinghui, Matthew Brown, Noah Snavely, and David G. Lowe. “Unsupervised learning of depth and ego-motion from video.” In CVPR, vol. 2, no. 6, p. 7. 2017.

[7] Kendall, Alex, Hayk Martirosyan, Saumitro Dasgupta, Peter Henry, Ryan Kennedy, Abraham Bachrach, and Adam Bry. “End-to-end learning of geometry and context for deep stereo regression.” In Proceedings of the IEEE International Co–nference on Computer Vision, pp. 66-75. 2017.