III. Simulators and Generative Models for understanding micro-scale devices from minimum labeled data

Imaging and computer vision play a key role in micro-scale manufacturing processes. They enable both online assembly processes and subsequent defect inspection steps. Recent progress in deep learning (DL) allows for more reliable visual recognition leading to DL adoption within ASM for various assembly and packaging workflows. Nonetheless, supervised deep learning, is proving data and labor intensive for many real-world applications. In a semiconductor manufacturing environment, for example, the configuration of assembled devices can frequently change, and rare defects can have a large impact on production yield. The need for relabeling new defects and devices on the factory floor can be a bottleneck in deployment scenarios. While common-practice techniques such as transfer learning are helping with these challenges, developing a capacity for generalization in machine learning workflows will be greatly beneficial in advanced manufacturing. ASM Pacific Technology is currently exploring data augmentation and generalization techniques based on realistic simulations. To complement this effort, we aim at using Generative models, such as those based on GANs, for enabling prediction from limited labeled data. Such models are showing promise in various applications and would potentially enable the synthesis of large-scale datasets and lead to a better prediction performance. The goal of this internship is to explore the use of Generative models and other forms of image simulation for improving the visual recognition of semiconductor device components and defects. During his work, the intern will undertake the following tasks:


  • Literature search on the state-of-the art generative models and their application to realistic image synthesis and data augmentation.
  • Make a choice of a limited number of models for experimentation and further development.
  • Study the inclusion of various physical constraints in the Generative models.
  • Train the models on ASM device images and demonstrate image synthesis results.
  • Analyze the generalization capabilities of the investigated models on object detection and semantic segmentation tasks, working with the ASM center of competency staff.
  • Summarize the results and analysis in a final and mid-term reports.

For more information about this project, please contact fboughorbel@alsi.asmpt.com  or call  +31 24 204 2824.


[1] Goodfellow, Ian, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. “Generative adversarial nets.” In Advances in neural information processing systems, pp. 2672-2680. 2014.

[2] Radford, Alec, Luke Metz, and Soumith Chintala. “Unsupervised representation learning with deep convolutional generative adversarial networks.” arXiv preprint arXiv:1511.06434 (2015).

[3] Karras, Tero, et al. “Progressive growing of GANs for improved quality, stability, and variation.” arXiv preprint arXiv:1710.10196 (2017).

[4] Brock, Andrew, Jeff Donahue, and Karen Simonyan. “Large Scale GAN Training for High Fidelity Natural Image Synthesis.” arXiv preprint arXiv:1809.11096 (2018).

[5] Dahmen T, Trampert P, Boughorbel F, Sprenger J, Klusch M, Fischer K, Kübel C, Slusallek P. Digital reality: a model-based approach to supervised learning from synthetic data. AI Perspectives. 1(1):1-2. Dec 2019.