Improving Defect Inspection on PCB

[Audio] Improving Defect Inspection on PCB Data Science Practice – FINAL Project Group 28 Hello everyone! On behalf of group 28, I would like to present our final project "Improving Defect Inspection on PCB"..

[Audio] Data Collection Data Preprocessing Feature Extraction Model Selection Testing Evaluation Data Science Flow As you know, there are 6 essential steps in a Data Science project. In our project, we target all the steps in the process, which includes data collection, data preprocessing, feature extraction, model selection, testing and evaluation..

[Audio] Content What you can learn Problem definition Methods Evaluation Conclusion Our content contains 5 parts..

[Audio] 1. What you can learn? A novel approach for generating PCB synthetic data for improving data problem A lightweight, fast and accurate model for tiny defect detection in PCB An effective model for training on small dataset 1. What you can learn from our project? - Generating PCB synthetic data still a challenge by variety of defects and quality of synthetic data. In this project, we applied a novel method that can generate variety of defects while keeping the quality of synthetic data. - Proposed a lightweight model with inferencing faster without model compression, pruning, or quantization techniques. It is necessary for a real-time application, where speed and performance of the model needs to be taken into account, especially in PCB defect inspection. - Besides speed, the proposed model is also designed to learn effectively on a small dataset..

[Audio] 2. Problem definition - Common problems of deep learning-based defect inspection on PCB: High Resolution, Small Defects Data Imbalance Lack of Data Challenging Feature Extraction 2. Problem definition Recently, deep learning models have been applied to overcome the limitations of conventional rule-based methods for defect detection. However, deep learning models are not perfect for all cases, performance can still be degraded by several factors such as: High resolution but small defects. Data imbalance. Lack of data. Challenging feature extraction..

[Audio] - Characteristics of the PCB dataset that we have: Small dataset (only ~ 270 images) Spur and spurious copper classes are imbalanced Defects looks similar (mouse bite and spur) Defective objects are tiny objects, which are difficult to detect and classify accurately 6 types of common defects Here's characteristics of the PCB dataset that we have: The dataset only contains 270 images, it's very small dataset compared with the PCB public dataset. Spur and spurious copper classes are imbalanced. Mouse bite and spur defects look similar. Defective objects are tiny objects, which are difficult to detect and classify accurately..

[Audio] 3. Methods Collect and annotate a small number of PCB images on Roboflow Diversify the dataset with open source library - Albumentations Generate synthetic data with generative diffusion model and hyper network Training data and synthetic data with RepVGG GELAN – an enhanced version of GELAN for object detection 3. Methods For solving the above problems, we applied the following methods: Collect and annotate a small number of PCB images on Roboflow. This small dataset then will be used to train and experiment the effectiveness of the proposed model. Diversify the dataset with open source library called Albumentations. Generate synthetic data with generative diffusion model and hyper network. Training data and synthetic data with RepVGG GELAN – an enhanced version of GELAN for object detection..

[Audio] 3. Methods 3.1. Data Collection – Collect and annotate data on Roboflow Roboflow Resize Original image size 3034x1586 Image with labels Resized image 640x640 Let us show more details of each methods. Firstly, about data collection, we collect 270 PCB images from the public dataset, then annotate on Roboflow in YOLO format and resize them to 640x640 resolution..

[Audio] 3.2. Data Preprocessing - Data augmentation with Albumentations Aug 3. Random Rotate (-5 to 5 degree) Aug 1. Horizontal Flip Aug 4. Darken Image Aug 2. Vertical Flip Secondly, for data preprocessing, we applied the following techniques to diversify our dataset: - Horizontal flip. Vertical flip. Random rotate and darken image..

[Audio] 3.2. Data Preprocessing – Diversify the dataset with synthetic data Synthetic data generation diagram Next, for generating synthetic data, we first masked the defect area of PCB image with Inpainting tool in Stable Diffusion. Then passed this masked image and a condition image to Hyper Network. For example, we want to generate new spurious copper defect, our condition image will be a spurious copper defect image. About Hyper Network, it's a neural network that controls image generation in Stable Diffusion by adding condition. In our case, input condition is spurious copper defect image. And next step, at the end of the diagram, by adding prompt and condition to diffusion model, we have a new PCB image with only defect area is synthesized and the remaining of the image will keep unchange. By that way, we only generate new defects in masked area while still keeping the quality and content of original PCB image. By doing this, we have more 30 synthetic images for training..

[Audio] 3.3. Model Selection – RepVGG GELAN - Reparameterized VGG or RepVGG is a novel type of convolutional neural network architecture designed to streamline and improve the training and performance of deep learning models. Simplified Design Efficient Feature Extraction Training Stability Performance Enhancement RepVGG Next, let's move on to model selection. In our project, we select RepVGG GELAN, an architecture that combines RepVGG and GELAN. Let's us introduce about RepVGG first. Reparameterized VGG or RepVGG is a novel type of convolutional neural network architecture designed to streamline and improve the training and performance of deep learning models. The followings are features of RepVGG: - Simplified Design: RepVGG simplifies the complexity of VGG-style ConvNets by introducing reparameterized convolutions, which streamline the network architecture. - Efficient Feature Extraction: The architecture is designed to improve feature extraction capabilities, making it suitable for tasks requiring high-resolution analysis, such as tiny objects. - Training Stability: By combining ReLU and identity mappings in RepVGG blocks, the architecture enhances training stability and performance, leading to better convergence during training. - Performance Enhancement: RepVGG architecture aims to achieve a balance between computational efficiency and detection performance, making it a valuable tool for various object detection tasks..

[Audio] - Generalized Efficient Layer Aggregation Network or GELAN is a novel architecture that combines CSPNet and ELAN principles for gradient path planning. Efficient Architecture Gradient Path Planning Improved Parameter Utilization Programmable Gradient Information (PGI) GELAN About GELAN, Generalized Efficient Layer Aggregation Network or GELAN is a novel architecture that combines CSPNet and ELAN principles for gradient path planning. GELAN can achieve high performance on small datasets due to the following factors: 1. Efficient Architecture: model is designed to be a lightweight and efficient neural network architecture. This efficiency allows GELAN to make optimal use of the limited data available in small datasets. 2. Gradient Path Planning: GELAN is based on gradient path planning, which helps in guiding the learning process of the network architecture. By the way, model can effectively utilize the available data in small datasets to improve performance. 3. Improved Parameter Utilization: GELAN achieves higher parameter utilization compared to other designs, even those based on advanced technologies like depth-wise convolution. This efficient parameter usage contributes to better performance on small datasets. 4. Programmable Gradient Information: The integration of PGI into GELAN provides reliable gradient information for updating network weights. This ensures that model can learn effectively from the limited data and establish accurate correlations between data and targets..

[Audio] - By combining the strengths of the RepVGG and GELAN, we have RepVGG-GELAN model for object detection tasks, particularly for small dataset. RepVGG-GELAN Finally, by combining the strengths of the RepVGG and GELAN, we have RepVGG-GELAN model for object detection tasks, particularly for small dataset. The architecture of RepVGG-GELAN is mainly comprised of RepVGG and RepNCSPELAN4, which are key components that enhance the model's performance in detecting objects..

[Audio] 3.4. Model Training Hyperparameters: Epochs Batch size Image size Optimizer Learning rate Momentum Workers 200 16 640 SGD 0.01 0.937 4 Training strategies: Splitting the dataset into train set and test set with ratio 8-2 respectively. Setting up “Early Stopping” that will stop the training after 50 epochs with no improvement observed. About model training, here's hyperparameters that we used to train the model. Besides, we splitted the dataset into train set and test set with ratio 8-2, and also setup "Early Stopping" to prevent overfitting after 50 epochs with no improvement..

[Audio] 4. Evaluation 4.1. Intersection over Union (IoU) Evaluation metric 1. Intersection over Union Before evaluating the proposed method, let us introduce some metrics in object detection. First metric is Intersection over Union (IoU), which is a commonly used evaluation metric in object detection tasks. IoU measures the overlap between the predicted bounding box and the ground truth bounding box. IoU 0.5 is a specific evaluation metric that considers the overlap between the predicted bounding box and the ground truth bounding box when the intersection area is at least 50% of the union area. IoU 0.95 is another evaluation metric that measures the overlap between the predicted and ground truth bounding boxes when the intersection area is at least 95% of the union area. This metric is stricter than IoU 0.5 and evaluates the model's ability to precisely localize objects with high accuracy..

[Audio] 4.2. Precision, Recall, Accuracy Evaluation metric 2. Precision, Recall and Accuracy Second metrics are precision, recall, and accuracy, which are important evaluation metrics that assess the performance of a model in detecting and localizing objects. Precision measures the proportion of correctly predicted positive instances (true positives) out of all instances predicted as positive (true positives + false positives). Precision indicates the model's ability to avoid false positives, correctly identifying relevant objects while minimizing incorrect detections. 2. Recall measures the proportion of correctly predicted positive instances (true positives) out of all actual positive instances in the dataset (true positives + false negatives). Recall indicates the model's ability to capture all relevant objects in the dataset without missing any. 3. Accuracy is a general metric that measures the overall correctness of the model's predictions across all classes (both positive and negative instances). Accuracy calculates the proportion of correctly predicted instances (true positives and true negatives) out of all instances in the dataset..

[Audio] 4. Evaluation 4.3. Training results The plots showing on the slide are our training results. In Precision-Recall Curve, the proposed model achieves 94.4% mAP with threshold 0.5..

[Audio] 4.4. Testing results Class Images Instances Precision Recall mAP50 mAP50-95 all 60 293 0.945 0.913 0.944 0.503 missing hole 60 45 0.991 1 0.995 0.637 mouse bite 60 54 0.944 0.934 0.959 0.493 open circuit 60 40 0.89 0.949 0.973 0.505 short 60 40 0.96 0.946 0.968 0.518 spur 60 58 0.935 0.776 0.827 0.385 spurious copper 60 58 0.948 0.872 0.939 0.481 Table 1. Report on test set Next, the table showing testing results on test set (60 images). Overall, model achieves high accuracy (>90%)..

[Audio] 4. Evaluation 4.5. Experiment on unseen data Predicted Ground truth Predicted Ground truth Ground truth Predicted We also experiment on unseen data. As you can see, model can detect defects correctly with IoU from 0.5 to 0.8..

[Audio] Ground truth Predicted Predicted Ground truth Predicted Ground truth Besides, there're still cases where defects haven't been detected (spur and spurious copper)..

[Audio] 5. Conclusion In this project, we try to complete whole process in a data science project. Improving PCB data problem by generating synthetic data and applying data augmentation techniques. Achieving high accuracy with a small dataset and training a lightweight model for faster inference. We improved deep learning-based defect inspection in PCB in terms of: Improving PCB data problem by generating synthetic data and applying data augmentation techniques. Achieving high accuracy with a small dataset and training a lightweight model for faster inference..

[Audio] Thank you. Thank you.