Improving Defect Inspection on PCB

[Audio] Hello everyone! On behalf of group 28, I would like to present our final project "Improving Defect Inspection on PCB"..

[Audio] As you know, there are 6 essential steps in a Data Science project. In our project, we target all the steps in the process, which includes data collection, data preprocessing, feature extraction, model selection, testing and evaluation..

[Audio] Our content contains 5 parts.. Content. What you can learn Problem definition Methods Evaluation Conclusion.

[Audio] 1. What you can learn from our project? - Generating PCB synthetic data still a challenge by variety of defects and quality of synthetic data. In this project, we applied a novel method that can generate variety of defects while keeping the quality of synthetic data. - Proposed a lightweight model with inferencing faster without model compression, pruning, or quantization techniques. It is necessary for a real-time application, where speed and performance of the model needs to be taken into account, especially in PCB defect inspection. - Besides speed, the proposed model is also designed to learn effectively on a small dataset..

[Audio] 2. Problem definition Recently, deep learning models have been applied to overcome the limitations of conventional rule-based methods for defect detection. However, deep learning models are not perfect for all cases, performance can still be degraded by several factors such as: High resolution but small defects. Data imbalance. Lack of data. Challenging feature extraction..

[Audio] Here's characteristics of the PCB dataset that we have: The dataset only contains 270 images, it's very small dataset compared with the PCB public dataset. Spur and spurious copper classes are imbalanced. Mouse bite and spur defects look similar. Defective objects are tiny objects, which are difficult to detect and classify accurately..

[Audio] 3. Methods For solving the above problems, we applied the following methods: Collect and annotate a small number of PCB images on Roboflow. This small dataset then will be used to train and experiment the effectiveness of the proposed model. Diversify the dataset with open source library called Albumentations. Generate synthetic data with generative diffusion model and hyper network. Training data and synthetic data with RepVGG GELAN – an enhanced version of GELAN for object detection..

[Audio] Let us show more details of each methods. Firstly, about data collection, we collect 270 PCB images from the public dataset, then annotate on Roboflow in YOLO format and resize them to 640x640 resolution..

[Audio] Secondly, for data preprocessing, we applied the following techniques to diversify our dataset: - Horizontal flip. Vertical flip. Random rotate and darken image..

[Audio] Next, for generating synthetic data, we first masked the defect area of PCB image with Inpainting tool in Stable Diffusion. Then passed this masked image and a condition image to Hyper Network. For example, we want to generate new spurious copper defect, our condition image will be a spurious copper defect image. About Hyper Network, it's a neural network that controls image generation in Stable Diffusion by adding condition. In our case, input condition is spurious copper defect image. And next step, at the end of the diagram, by adding prompt and condition to diffusion model, we have a new PCB image with only defect area is synthesized and the remaining of the image will keep unchange. By that way, we only generate new defects in masked area while still keeping the quality and content of original PCB image. By doing this, we have more 30 synthetic images for training..

[Audio] Next, let's move on to model selection. In our project, we select RepVGG GELAN, an architecture that combines RepVGG and GELAN. Let's us introduce about RepVGG first. Reparameterized VGG or RepVGG is a novel type of convolutional neural network architecture designed to streamline and improve the training and performance of deep learning models. The followings are features of RepVGG: - Simplified Design: RepVGG simplifies the complexity of VGG-style ConvNets by introducing reparameterized convolutions, which streamline the network architecture. - Efficient Feature Extraction: The architecture is designed to improve feature extraction capabilities, making it suitable for tasks requiring high-resolution analysis, such as tiny objects. - Training Stability: By combining ReLU and identity mappings in RepVGG blocks, the architecture enhances training stability and performance, leading to better convergence during training. - Performance Enhancement: RepVGG architecture aims to achieve a balance between computational efficiency and detection performance, making it a valuable tool for various object detection tasks..

[Audio] About GELAN, Generalized Efficient Layer Aggregation Network or GELAN is a novel architecture that combines CSPNet and ELAN principles for gradient path planning. GELAN can achieve high performance on small datasets due to the following factors: 1. Efficient Architecture: model is designed to be a lightweight and efficient neural network architecture. This efficiency allows GELAN to make optimal use of the limited data available in small datasets. 2. Gradient Path Planning: GELAN is based on gradient path planning, which helps in guiding the learning process of the network architecture. By the way, model can effectively utilize the available data in small datasets to improve performance. 3. Improved Parameter Utilization: GELAN achieves higher parameter utilization compared to other designs, even those based on advanced technologies like depth-wise convolution. This efficient parameter usage contributes to better performance on small datasets. 4. Programmable Gradient Information: The integration of PGI into GELAN provides reliable gradient information for updating network weights. This ensures that model can learn effectively from the limited data and establish accurate correlations between data and targets..

[Audio] Finally, by combining the strengths of the RepVGG and GELAN, we have RepVGG-GELAN model for object detection tasks, particularly for small dataset. The architecture of RepVGG-GELAN is mainly comprised of RepVGG and RepNCSPELAN4, which are key components that enhance the model's performance in detecting objects..

[Audio] About model training, here's hyperparameters that we used to train the model. Besides, we splitted the dataset into train set and test set with ratio 8-2, and also setup "Early Stopping" to prevent overfitting after 50 epochs with no improvement..

[Audio] Before evaluating the proposed method, let us introduce some metrics in object detection. First metric is Intersection over Union (IoU), which is a commonly used evaluation metric in object detection tasks. IoU measures the overlap between the predicted bounding box and the ground truth bounding box. IoU 0.5 is a specific evaluation metric that considers the overlap between the predicted bounding box and the ground truth bounding box when the intersection area is at least 50% of the union area. IoU 0.95 is another evaluation metric that measures the overlap between the predicted and ground truth bounding boxes when the intersection area is at least 95% of the union area. This metric is stricter than IoU 0.5 and evaluates the model's ability to precisely localize objects with high accuracy..

[Audio] Second metrics are precision, recall, and accuracy, which are important evaluation metrics that assess the performance of a model in detecting and localizing objects. Precision measures the proportion of correctly predicted positive instances (true positives) out of all instances predicted as positive (true positives + false positives). Precision indicates the model's ability to avoid false positives, correctly identifying relevant objects while minimizing incorrect detections. 2. Recall measures the proportion of correctly predicted positive instances (true positives) out of all actual positive instances in the dataset (true positives + false negatives). Recall indicates the model's ability to capture all relevant objects in the dataset without missing any. 3. Accuracy is a general metric that measures the overall correctness of the model's predictions across all classes (both positive and negative instances). Accuracy calculates the proportion of correctly predicted instances (true positives and true negatives) out of all instances in the dataset..

[Audio] The plots showing on the slide are our training results. In Precision-Recall Curve, the proposed model achieves 94.4% mAP with threshold 0.5..

[Audio] Next, the table showing testing results on test set (60 images). Overall, model achieves high accuracy (>90%)..

[Audio] We also experiment on unseen data. As you can see, model can detect defects correctly with IoU from 0.5 to 0.8..

[Audio] Besides, there're still cases where defects haven't been detected (spur and spurious copper)..

[Audio] We improved deep learning-based defect inspection in PCB in terms of: Improving PCB data problem by generating synthetic data and applying data augmentation techniques. Achieving high accuracy with a small dataset and training a lightweight model for faster inference..

Thank you.