Internship: Micro-device 3D structure inference using deep learning

In advanced micro-scale manufacturing, such as in the semiconductor assembly and packaging field, machine vision plays a key role in inspection and process control.

The use of 3D micro-scale imaging is increasingly popular for several critical tasks. Cost, throughput, and space limitations can however restrict the integration of complex 3D capture devices for certain applications. An attractive option is to use the already available optics and 2D sensors and reconstruct 3D information computationally. Conventionally, in such approach one would need to acquire multiple 2D images, from different viewpoints, to obtain a good reconstruction. Other techniques can be used as well, including depth recovery from focus stacks. These methods are, however, considered too slow for online semiconductor manufacturing applications. More recently, the task of reconstructing 3D from single images regained attention due to the possibilities offered by deep convolutional networks. A CNN model can be trained on pairs of 2D and 3D images, learning the geometric structure of the objects of interest. This can potentially turn ordinary industrial vision cameras into 3D acquisition devices. The goal of this Master thesis project is to:

  • Perform a literature search on the latest deep-learning based 3D reconstruction techniques, with a focus on monocular methods.
  • Select two or three methods for evaluation and further development.
  • Acquire 2D/3D training pairs using available ASM equipment (confocal microscopes, stereo cameras, and regular 2D cameras.) The imagery will be acquired on ASM application-related devices.
  • Train suitable 3D reconstruction models, using internal and external datasets, and employing transfer learning when relevant.
  • Explore the incorporation of prior knowledge about device geometry and materials in the reconstruction process.
  • Compile the results and analysis in a final and mid-term reports.

The intern will be supported and advised in his various tasks by ASM Pacific Technology staff, interacting with the local team in the Netherlands, as well as with the Asia-based groups.

To apply for this internship please contact gknippels@alsi.asmpt.com , 024 678 2873

References:

[1] Eigen, David, Christian Puhrsch, and Rob Fergus. “Depth map prediction from a single image using a multi-scale deep network.” In Advances in neural information processing systems, pp. 2366-2374. 2014.

[2] Garg, Ravi, Vijay Kumar BG, Gustavo Carneiro, and Ian Reid. “Unsupervised cnn for single view depth estimation: Geometry to the rescue.” In European Conference on Computer Vision, pp. 740-756. Springer, Cham, 2016.

[3] Rezende, Danilo Jimenez, SM Ali Eslami, Shakir Mohamed, Peter Battaglia, Max Jaderberg, and Nicolas Heess. “Unsupervised learning of 3d structure from images.” In Advances in Neural Information Processing Systems, pp. 4996-5004. 2016.

[4] Fan, Haoqiang, Hao Su, and Leonidas J. Guibas. “A Point Set Generation Network for 3D Object Reconstruction from a Single Image.” CVPR. Vol. 2. No. 4. 2017.

[5] Wu, Jiajun, Chengkai Zhang, Tianfan Xue, Bill Freeman, and Josh Tenenbaum. “Learning a probabilistic latent space of object shapes via 3d generative-adversarial modeling.” In Advances in Neural Information Processing Systems, pp. 82-90. 2016.

[6] Zhou, Tinghui, Matthew Brown, Noah Snavely, and David G. Lowe. “Unsupervised learning of depth and ego-motion from video.” In CVPR, vol. 2, no. 6, p. 7. 2017.