Benchmarking software for Machine Health Management

Introduction
The ASM Centre of Competency, located in the Netherlands, is responsible for identifying new high-tech innovation opportunities for the full global ASM Pacific Technology product portfolio [1]. The equipment portfolio of ASM Pacific Technology covers the complete back-end section of the semiconductor equipment market, and offers comprehensive assembly and advanced packaging solutions for its customers in the fields of optoelectronics, electronics, solar energy, automotive and other segments.

Challenges
The semiconductor industry is one of the most technology-evolving and capital-intensive market sectors [2-5]. To strengthen a competitive position in the semiconductor industry, the key challenges a manufacturing plant has to face are the increase of production throughput and yield. Monitoring health conditions of the semiconductor machines is therefore necessary to secure the required product quality. Furthermore, effective prediction of the failures of the semicon equipment is a way to decrease a machine downtime, improve productivity, and reduce production costs and repairing time.
The semiconductor manufacturing equipment is equipped with different sensors for real-time machine control and equipment condition monitoring. Data from these sensors can be used to detect deviations of the machine performance from the nominal one, in order to avoid degradations in the product quality and losses in the production yield. The same data is useful in the process of discovering when the equipment needs a maintenance to avoid machine failures and unscheduled breaks in the production.
To achieve automated machine condition monitoring and predictive maintenance functionalities, we need to address the following challenges [2-7]:

  • make appropriate selection of sensors for machine condition monitoring and choose optimal locations where these sensors have to be installed,
  • develop algorithms for machine health management, starting from extraction of machine health condition indicators (CI’s) out of the sensor signals, up to design of predictive methods for forecasting equipment behaviour and health diagnostics,
  • optimally schedule maintenance operations according to the equipment conditions.

Company ASM Pacific Technology is developing proprietary software for machine health management, which is customized for applications and deployment on own semicon machines. The question how well this software performs in comparison with commercial off-the-shelf software packages for condition monitoring, health diagnostics and predictive maintenance. Therefore, it is instructive to benchmark the proprietary software of ASM Pacific Technology with commercial software tools in terms of available functionalities, calculation efficiency, user friendliness, level of automation, reliability and accuracy of forecasting machine behaviour, etc.

Internship Assignment
In this project, we will benchmark the proprietary software of ASM Pacific Technology with at least one commercial software package for machine health management. An interesting off-the-shelf candidate is the LabVIEW Analytics and Machine Learning Toolkit [8]. If time would allow, we may also include the Predictive Maintenance Toolbox of Matlab [9] in benchmarking.
The Center of Competency has a large collection of data measured on many ASMPT machines operating in the field. These data involve standard readings from motion and current sensors. The idea is to process these data using the company proprietary software and commercial tools, such as to benchmark functionalities, performance and autonomy of the company software with respect to the off-the-shelf solutions. Furthermore, the Center of Competency has several models of a wirebonder machine that can be used for simulation of machine behaviours that might not be captured in the available measurements. If needed, synthetic data could be generated in simulations to make more complete evaluation of the software capabilities.
The investigation objectives are:

  • literature study on methods for machine condition monitoring, health diagnostics and predictive maintenance,
  • learn how to perform machine health management using the proprietary software of ASM Pacific Technology and LabVIEW Analytics and Machine Learning Toolkit,
  • process available machine data using the proprietary and commercial software packages and critically evaluate advantages and limitations of these software tools,
  • if needed, generate synthetic data by simulation of machine behaviours that are not captured in the existing measurements but could be used for evaluation of some specific software functionalities,
  • eventually, include Predictive Maintenance Toolbox of Matlab in the benchmarking,
  • identify pro’s and con’s of the proprietary software for machine health management and give recommendations for its eventual improvements.

The results of this project will be disseminated via the internship report and presentations at the knowledge institution and Center of Competency. The generated knowledge can become a valuable academic know-how and drive technological advantages against other producers of the semiconductor equipment.

 

For more information about this project, please contact dkostic@alsi.asmpt.com or call +31 24 204 2827.

References
[1] ASM Center of Competency, https://alsi.asmpt.com
[2] S. Munirathinam and B. Ramadoss, “Predictive Models for Equipment Fault Detection in the Semiconductor Manufacturing Process,” IACSIT International Journal of Engineering and Technology, Vol. 8, No. 4, pp. 273-285, 2016.
[3] Y. Liu, Predictive modeling for intelligent maintenance in complex semiconductor manufacturing processes, PhD thesis, The University of Michigan, 2008.
[4] J. Iskandar, J. Moyne, K. Subrahmanyam, P. Hawkins, and M. Armacos: “Predictive Maintenance in semiconductor manufacturing,” 26th Annual SEMI Advanced Semiconductor Manufacturing Conference, Saratoga Springs, NY, USA, 2015.
[5] P. Scheibelhofer, D. Gleispach, G. Hayderer, and E. Stadlober: “A Methodology for Predictive Maintenance in Semiconductor Manufacturing,” Austrian Journal of Statistics, Vol. 41, No. 3, pp. 161–173, 2012.
[6] A. Khanore, Predictive Maintenance Algorithms Development for a Wire Bonder, DCT internship report, 2019.
[7] R. van Bussel: Obtaining Machine Health Condition Indicators Using Wavelet Transforms, DCT internship report, 2019.
[8] LabVIEW Analytics and Machine Learning Toolkit,
http://www.ni.com/pdf/manuals/377061a.html
[9] Predictive Maintenance Toolbox of Matlab, https://nl.mathworks.com/products/predictive-maintenance.html

Posted in Uncategorized

I. Meta-learning and AutoML for Precision Manufacturing

The use of deep learning (DL) and other state of the art machine learning (ML) techniques is expanding to new arenas such as precision manufacturing. At ASM Pacific Technology ML/DL methods are employed for diverse tasks such as visual inspection of microdevices and for complex process control. One of the key bottlenecks in deploying ML solutions at scale is the need for frequent model updating in response to drift in the data obtained from the field and to changes in the required tasks.

Such model re-training/fine-tuning usually requires skilled operators for performing hyperparameters optimization and for introducing modifications in the deployed architectures when needed. This project aims at exploring applications to ASM manufacturing tasks of state the art Automated ML and Meta-learning methods. The goal is to achieve a high degree of autonomy in the optimization of both model structure and related hyperparameters. Meta-learning, Neural Architecture Search (NAS), and other AutoML approaches are expected to play a critical role in advanced manufacturing, enabling both inspection and complex process control. During this project we will introduce the intern to real-world use-cases in microdevice inspection and other ASM tasks, using these as a testbed for the target Meta-learning/AutoML methods. The intern is expected to perform the following tasks:

  • Survey the state of the art in Meta-learning and related techniques.
  • Select, with support from company advisers, ASM PT applications that are highly relevant to Meta-learning.
  • Conduct experiments to demonstrate increased automation and quality in DL model production.
  • Summarize the conclusions of the research in a report.

For more information about this project, please contact fboughorbel@alsi.asmpt.com  or call  +31 24 204 2824.

References

[1] Vanschoren J. Meta-learning: A survey. arXiv preprint arXiv:1810.03548. Oct 8, 2018.

[2] He X, Zhao K, Chu X. AutoML: A Survey of the State-of-the-Art. arXiv preprint arXiv:1908.00709. Aug 2, 2019.

[3] Elsken T, Metzen JH, Hutter F. Neural architecture search: A survey. arXiv preprint arXiv:1808.05377. Aug 16, 2018.

Posted in Uncategorized

II. Anomaly Detectors for identifying rare defects

In complex micro-scale device manufacturing representative defect examples are rare and are often hard to collect at scale. Such defects, although subtle, can have major consequences on systems assembled from the affected components. Using machine learning algorithms to identify these anomalies is challenged by this scarcity of defect data. A possible way to overcome this challenge is to use anomaly detectors that train only on the, widely available, non-defective examples. Anomaly detection models capture statistical representations of the good cases, allowing for defective cases to be identified by their distance in a latent space to the good distributions. At ASM we recently experimented with using Generative Adversarial Networks (GANs) for anomaly detection [1][2]. A GAN trained on images of good devices was able to identify defective ones as ‘anomalous’ and not part of its data set. The goal of the new internship project is to broaden the previous study to a wider range of methods, including Bayesian ones, and to experiment with different types of microdevices and defects. The key tasks that the intern will undertake are the following:

  • Conduct a literature study of various state of the art anomaly detection algorithms.
  • Propose and motivate a choice of a limited number of methods for further experimentation. The purpose is to use ASM use-cases (device inspection using vision and other sensing modalities) to compare the relative performance of different approaches.
  • Propose and implement any required improvements to the most relevant SOTA anomaly detection technique and data processing pipelines.
  • Compile a mid-term and final reports summarizing results and conclusions.

For more information about this project, please contact fboughorbel@alsi.asmpt.com  or call  +31 24 204 2824.

References

[1] Schlegl T, Seeböck P, Waldstein SM, Schmidt-Erfurth U, Langs G. Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. InInternational conference on information processing in medical imaging, pp. 146-157, Springer, Cham, Jun 25, 2017.

[2] Zenati H, Foo CS, Lecouat B, Manek G, Chandrasekhar VR. Efficient gan-based anomaly detection. arXiv preprint arXiv:1802.06222. 2018 Feb 17.

[3] Chalapathy, Raghavendra, and Sanjay Chawla. “Deep learning for anomaly detection: A survey.” arXiv preprint arXiv:1901.03407 (2019).

[4] Chandola V, Banerjee A, Kumar V. “Anomaly detection: A survey,” ACM computing surveys (CSUR), 41(3):1-58, Jul 30, 2009.

Posted in Uncategorized

III. Simulators and Generative Models for understanding micro-scale devices from minimum labeled data

Imaging and computer vision play a key role in micro-scale manufacturing processes. They enable both online assembly processes and subsequent defect inspection steps. Recent progress in deep learning (DL) allows for more reliable visual recognition leading to DL adoption within ASM for various assembly and packaging workflows. Nonetheless, supervised deep learning, is proving data and labor intensive for many real-world applications. In a semiconductor manufacturing environment, for example, the configuration of assembled devices can frequently change, and rare defects can have a large impact on production yield. The need for relabeling new defects and devices on the factory floor can be a bottleneck in deployment scenarios. While common-practice techniques such as transfer learning are helping with these challenges, developing a capacity for generalization in machine learning workflows will be greatly beneficial in advanced manufacturing. ASM Pacific Technology is currently exploring data augmentation and generalization techniques based on realistic simulations. To complement this effort, we aim at using Generative models, such as those based on GANs, for enabling prediction from limited labeled data. Such models are showing promise in various applications and would potentially enable the synthesis of large-scale datasets and lead to a better prediction performance. The goal of this internship is to explore the use of Generative models and other forms of image simulation for improving the visual recognition of semiconductor device components and defects. During his work, the intern will undertake the following tasks:

 

  • Literature search on the state-of-the art generative models and their application to realistic image synthesis and data augmentation.
  • Make a choice of a limited number of models for experimentation and further development.
  • Study the inclusion of various physical constraints in the Generative models.
  • Train the models on ASM device images and demonstrate image synthesis results.
  • Analyze the generalization capabilities of the investigated models on object detection and semantic segmentation tasks, working with the ASM center of competency staff.
  • Summarize the results and analysis in a final and mid-term reports.

For more information about this project, please contact fboughorbel@alsi.asmpt.com  or call  +31 24 204 2824.

References

[1] Goodfellow, Ian, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. “Generative adversarial nets.” In Advances in neural information processing systems, pp. 2672-2680. 2014.

[2] Radford, Alec, Luke Metz, and Soumith Chintala. “Unsupervised representation learning with deep convolutional generative adversarial networks.” arXiv preprint arXiv:1511.06434 (2015).

[3] Karras, Tero, et al. “Progressive growing of GANs for improved quality, stability, and variation.” arXiv preprint arXiv:1710.10196 (2017).

[4] Brock, Andrew, Jeff Donahue, and Karen Simonyan. “Large Scale GAN Training for High Fidelity Natural Image Synthesis.” arXiv preprint arXiv:1809.11096 (2018).

[5] Dahmen T, Trampert P, Boughorbel F, Sprenger J, Klusch M, Fischer K, Kübel C, Slusallek P. Digital reality: a model-based approach to supervised learning from synthetic data. AI Perspectives. 1(1):1-2. Dec 2019.

Posted in Uncategorized

IV. Deep Learning for recovering the 3D structure of microdevices

In advanced micro-scale manufacturing, such as in the semiconductor assembly and packaging field, machine vision plays a key role in inspection and process control. Cost, throughput, and space limitations can however restrict the integration of complex 3D capture devices for certain applications. Active and passive stereoscopic imaging systems are the primary tools used for 3D inspection, but an attractive option is to obtain the 3D information computationally from conventional 2D cameras. Recently, this approach regained attention due to the possibilities offered by deep convolutional networks (CNN). A CNN model can be trained on pairs of 2D and 3D images, learning the geometric structure of the objects of interest. This can potentially turn ordinary industrial vision cameras into 3D acquisition devices. The goal of this project is to explore the application of DL to monocular 3D reconstruction, and to the recovery of high-quality depth maps from both single images and sequences (focal stacks or videos).   The internship tasks are outlines in the following:

  • Perform a literature search on the latest deep learning – based 3D reconstruction techniques.
  • Select two or three methods for evaluation and further development.
  • Train suitable 3D reconstruction models, using ASM and public datasets.
  • Explore the incorporation of prior knowledge about device geometry and materials in the reconstruction process.
  • Compile the results and analysis in a final and mid-term reports.

For more information about this project, please contact fboughorbel@alsi.asmpt.com  or call  +31 24 204 2824.

References

[1] Eigen, David, Christian Puhrsch, and Rob Fergus. “Depth map prediction from a single image using a multi-scale deep network.” In Advances in neural information processing systems, pp. 2366-2374. 2014.

[2] Garg, Ravi, Vijay Kumar BG, Gustavo Carneiro, and Ian Reid. “Unsupervised cnn for single view depth estimation: Geometry to the rescue.” In European Conference on Computer Vision, pp. 740-756. Springer, Cham, 2016.

[3] Rezende, Danilo Jimenez, SM Ali Eslami, Shakir Mohamed, Peter Battaglia, Max Jaderberg, and Nicolas Heess. “Unsupervised learning of 3d structure from images.” In Advances in Neural Information Processing Systems, pp. 4996-5004. 2016.

[5] Alhashim I, Wonka P. High quality monocular depth estimation via transfer learning. arXiv preprint arXiv:1812.11941. Dec 31, 2018.

[6] Zhou, Tinghui, Matthew Brown, Noah Snavely, and David G. Lowe. “Unsupervised learning of depth and ego-motion from video.” In CVPR, vol. 2, no. 6, p. 7. 2017.

[7] Kendall, Alex, Hayk Martirosyan, Saumitro Dasgupta, Peter Henry, Ryan Kennedy, Abraham Bachrach, and Adam Bry. “End-to-end learning of geometry and context for deep stereo regression.” In Proceedings of the IEEE International Co–nference on Computer Vision, pp. 66-75. 2017.

Posted in Uncategorized