Category: Internship

The use of deep learning (DL) and other state of the art machine learning (ML) techniques is expanding to new arenas such as precision manufacturing. At ASM Pacific Technology ML/DL methods are employed for diverse tasks such as visual inspection of microdevices and for complex process control. One of the key bottlenecks in deploying ML solutions at scale is the need for frequent model updating in response to drift in the data obtained from the field and to changes in the required tasks.

Such model re-training/fine-tuning usually requires skilled operators for performing hyperparameters optimization and for introducing modifications in the deployed architectures when needed. This project aims at exploring applications to ASM manufacturing tasks of state the art Automated ML and Meta-learning methods. The goal is to achieve a high degree of autonomy in the optimization of both model structure and related hyperparameters. Meta-learning, Neural Architecture Search (NAS), and other AutoML approaches are expected to play a critical role in advanced manufacturing, enabling both inspection and complex process control. During this project we will introduce the intern to real-world use-cases in microdevice inspection and other ASM tasks, using these as a testbed for the target Meta-learning/AutoML methods. The intern is expected to perform the following tasks:

• Survey the state of the art in Meta-learning and related techniques.
• Select, with support from company advisers, ASM PT applications that are highly relevant to Meta-learning.
• Conduct experiments to demonstrate increased automation and quality in DL model production.
• Summarize the conclusions of the research in a report.

For more information about this project, please contact fboughorbel@alsi.asmpt.com  or call  +31 24 204 2824.

References

[1] Vanschoren J. Meta-learning: A survey. arXiv preprint arXiv:1810.03548. Oct 8, 2018.

[2] He X, Zhao K, Chu X. AutoML: A Survey of the State-of-the-Art. arXiv preprint arXiv:1908.00709. Aug 2, 2019.

[3] Elsken T, Metzen JH, Hutter F. Neural architecture search: A survey. arXiv preprint arXiv:1808.05377. Aug 16, 2018.

In complex micro-scale device manufacturing representative defect examples are rare and are often hard to collect at scale. Such defects, although subtle, can have major consequences on systems assembled from the affected components. Using machine learning algorithms to identify these anomalies is challenged by this scarcity of defect data. A possible way to overcome this challenge is to use anomaly detectors that train only on the, widely available, non-defective examples. Anomaly detection models capture statistical representations of the good cases, allowing for defective cases to be identified by their distance in a latent space to the good distributions. At ASM we recently experimented with using Generative Adversarial Networks (GANs) for anomaly detection [1][2]. A GAN trained on images of good devices was able to identify defective ones as ‘anomalous’ and not part of its data set. The goal of the new internship project is to broaden the previous study to a wider range of methods, including Bayesian ones, and to experiment with different types of microdevices and defects. The key tasks that the intern will undertake are the following:

• Conduct a literature study of various state of the art anomaly detection algorithms.
• Propose and motivate a choice of a limited number of methods for further experimentation. The purpose is to use ASM use-cases (device inspection using vision and other sensing modalities) to compare the relative performance of different approaches.
• Propose and implement any required improvements to the most relevant SOTA anomaly detection technique and data processing pipelines.
• Compile a mid-term and final reports summarizing results and conclusions.

For more information about this project, please contact fboughorbel@alsi.asmpt.com  or call  +31 24 204 2824.

References

[1] Schlegl T, Seeböck P, Waldstein SM, Schmidt-Erfurth U, Langs G. Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. InInternational conference on information processing in medical imaging, pp. 146-157, Springer, Cham, Jun 25, 2017.

[2] Zenati H, Foo CS, Lecouat B, Manek G, Chandrasekhar VR. Efficient gan-based anomaly detection. arXiv preprint arXiv:1802.06222. 2018 Feb 17.

[3] Chalapathy, Raghavendra, and Sanjay Chawla. “Deep learning for anomaly detection: A survey.” arXiv preprint arXiv:1901.03407 (2019).

[4] Chandola V, Banerjee A, Kumar V. “Anomaly detection: A survey,” ACM computing surveys (CSUR), 41(3):1-58, Jul 30, 2009.

Imaging and computer vision play a key role in micro-scale manufacturing processes. They enable both online assembly processes and subsequent defect inspection steps. Recent progress in deep learning (DL) allows for more reliable visual recognition leading to DL adoption within ASM for various assembly and packaging workflows. Nonetheless, supervised deep learning, is proving data and labor intensive for many real-world applications. In a semiconductor manufacturing environment, for example, the configuration of assembled devices can frequently change, and rare defects can have a large impact on production yield. The need for relabeling new defects and devices on the factory floor can be a bottleneck in deployment scenarios. While common-practice techniques such as transfer learning are helping with these challenges, developing a capacity for generalization in machine learning workflows will be greatly beneficial in advanced manufacturing. ASM Pacific Technology is currently exploring data augmentation and generalization techniques based on realistic simulations. To complement this effort, we aim at using Generative models, such as those based on GANs, for enabling prediction from limited labeled data. Such models are showing promise in various applications and would potentially enable the synthesis of large-scale datasets and lead to a better prediction performance. The goal of this internship is to explore the use of Generative models and other forms of image simulation for improving the visual recognition of semiconductor device components and defects. During his work, the intern will undertake the following tasks:

• Literature search on the state-of-the art generative models and their application to realistic image synthesis and data augmentation.
• Make a choice of a limited number of models for experimentation and further development.
• Study the inclusion of various physical constraints in the Generative models.
• Train the models on ASM device images and demonstrate image synthesis results.
• Analyze the generalization capabilities of the investigated models on object detection and semantic segmentation tasks, working with the ASM center of competency staff.
• Summarize the results and analysis in a final and mid-term reports.

For more information about this project, please contact fboughorbel@alsi.asmpt.com  or call  +31 24 204 2824.

References

[1] Goodfellow, Ian, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. “Generative adversarial nets.” In Advances in neural information processing systems, pp. 2672-2680. 2014.

[2] Radford, Alec, Luke Metz, and Soumith Chintala. “Unsupervised representation learning with deep convolutional generative adversarial networks.” arXiv preprint arXiv:1511.06434 (2015).

[3] Karras, Tero, et al. “Progressive growing of GANs for improved quality, stability, and variation.” arXiv preprint arXiv:1710.10196 (2017).

[4] Brock, Andrew, Jeff Donahue, and Karen Simonyan. “Large Scale GAN Training for High Fidelity Natural Image Synthesis.” arXiv preprint arXiv:1809.11096 (2018).

[5] Dahmen T, Trampert P, Boughorbel F, Sprenger J, Klusch M, Fischer K, Kübel C, Slusallek P. Digital reality: a model-based approach to supervised learning from synthetic data. AI Perspectives. 1(1):1-2. Dec 2019.

In advanced micro-scale manufacturing, such as in the semiconductor assembly and packaging field, machine vision plays a key role in inspection and process control. Cost, throughput, and space limitations can however restrict the integration of complex 3D capture devices for certain applications. Active and passive stereoscopic imaging systems are the primary tools used for 3D inspection, but an attractive option is to obtain the 3D information computationally from conventional 2D cameras. Recently, this approach regained attention due to the possibilities offered by deep convolutional networks (CNN). A CNN model can be trained on pairs of 2D and 3D images, learning the geometric structure of the objects of interest. This can potentially turn ordinary industrial vision cameras into 3D acquisition devices. The goal of this project is to explore the application of DL to monocular 3D reconstruction, and to the recovery of high-quality depth maps from both single images and sequences (focal stacks or videos).   The internship tasks are outlines in the following:

• Perform a literature search on the latest deep learning – based 3D reconstruction techniques.
• Select two or three methods for evaluation and further development.
• Train suitable 3D reconstruction models, using ASM and public datasets.
• Explore the incorporation of prior knowledge about device geometry and materials in the reconstruction process.
• Compile the results and analysis in a final and mid-term reports.

For more information about this project, please contact fboughorbel@alsi.asmpt.com  or call  +31 24 204 2824.

References

[1] Eigen, David, Christian Puhrsch, and Rob Fergus. “Depth map prediction from a single image using a multi-scale deep network.” In Advances in neural information processing systems, pp. 2366-2374. 2014.

[2] Garg, Ravi, Vijay Kumar BG, Gustavo Carneiro, and Ian Reid. “Unsupervised cnn for single view depth estimation: Geometry to the rescue.” In European Conference on Computer Vision, pp. 740-756. Springer, Cham, 2016.

[3] Rezende, Danilo Jimenez, SM Ali Eslami, Shakir Mohamed, Peter Battaglia, Max Jaderberg, and Nicolas Heess. “Unsupervised learning of 3d structure from images.” In Advances in Neural Information Processing Systems, pp. 4996-5004. 2016.

[5] Alhashim I, Wonka P. High quality monocular depth estimation via transfer learning. arXiv preprint arXiv:1812.11941. Dec 31, 2018.

[6] Zhou, Tinghui, Matthew Brown, Noah Snavely, and David G. Lowe. “Unsupervised learning of depth and ego-motion from video.” In CVPR, vol. 2, no. 6, p. 7. 2017.

[7] Kendall, Alex, Hayk Martirosyan, Saumitro Dasgupta, Peter Henry, Ryan Kennedy, Abraham Bachrach, and Adam Bry. “End-to-end learning of geometry and context for deep stereo regression.” In Proceedings of the IEEE International Co–nference on Computer Vision, pp. 66-75. 2017.