A Perfect Match: The Critical Role of Matching in Machine Vision Technologies
Why matching is one of the most fundamental machine vision technologies and what it takes to find the perfect match.
Shape-based matching also works with blurred images. Image Source: MVTec Software
The search for a perfect match isn’t unique to dating platforms. It also applies to machine vision, where users are also looking for perfect “counterparts.” Matching technologies in machine vision software ensure that objects in the image are precisely located. So instead of a person seeking a relationship, software is seeking an object. This has many benefits in industrial process chains. Robots can handle components safely and don’t reach into empty space. Workpieces can be precision machined. The quality control team doesn’t overlook any production errors. Here I’ll shed light on why matching is one of the most fundamental machine vision technologies and what it takes to find the perfect match.
In nearly all machine vision applications, recognizing objects is one of the first steps. After the image has been acquired, the key question is usually: Where exactly is the relevant object to be found in the image? For example, if a robot is to grip an object safely, it has to know its exact position. This knowledge is also crucial for many other tasks in production and assembly. When manufacturing semiconductors, for example, the position of all micro-components has to be precisely located. Otherwise, they can’t be optimally processed. For filling and control processes in the beverage industry to be performed seamlessly, the exact location of bottles and their labels must be known. These are just some examples.
Matching should be fast, robust, and precise
The machine vision software now has the task of precisely generating the relevant information from the image data. The operative phrase is matching technologies. Three main parameters are important for a successful solution: the speed of execution, the robustness of the method, and the accuracy of the results. What does this mean exactly? Where speed is concerned, it often takes the system only a few milliseconds to process an image. In the case of robustness, objects also need to be detected in very noisy images, in which even human vision reaches its limits. Finally, some applications have to achieve a locating accuracy of 1/20 pixels and a repeatability of 1/100 pixels (meaning the repeated accuracy of hits during the pixel-precise detection of defects).
If the performance requirements have been met, matching technologies can optimize a wide range of machine vision applications. Robots, for example, are sufficiently fine-tuned to grip objects safely. At the same time, quality control is able to determine whether the component meets the exact requirements. In production workflows, matching also allows workpieces to be accurately aligned for further processing steps on machines. Objects to be measured can also be precisely pinpointed. Finally, matching functions locate labels on products in order to read out information in the right place using optical character recognition(OCR). As this wide range of applications shows, matching technologies are used in the production of almost all products in everyday use.
What specifically are the different matching methods?
To locate the relevant part of an image unerringly, the matching process comprises multiple specific methods that differ from each other in fundamental ways. In correlation-based matching, the gray values, or shades of gray, of the individual pixels are compared with each other. In other words, a correlation is drawn between the object’s gray values and those of the image content. This method is a particularly good choice if the images are unclear or the object doesn’t have any well-defined edges. However, distinctly differentiating between the different gray values requires powerful lighting.
Another method is descriptor-based matching. This method originated in academic research, where it’s common practice. A clear texture and prominent dots are needed for accurate results. Because many objects in the production environment don’t meet these requirements, the technology plays only a minor role in an industrial setting.
The jewel in the crown: shape-based matching
The most important method is shape-based matching, which can be used to locate a wide range of objects precisely and robustly in real time. What’s special about this method is that the technology delivers outstanding, subpixel-precise results even if objects are rotated, scaled, or partially covered. This is true even if the objects are outside the image or if the lighting fluctuates.
Matching in 3D space. Image Source: MVTec Software
Matching also works in 3D. Perspective matching, for example, is a further development of shape-based matching that allows the position of flat object parts of any shape to be determined in 3D space. This method is particularly suitable for components that are tilted, creating a deformation in perspective.
Shape-based matching also makes it possible to reliably determine the position and orientation of randomly arranged 3D objects in three-dimensional space based on their CAD model.
Cutting-edge matching features created a boost in terms of speed, robustness, and accuracy of matching methods in recent years. These innovations include Deep 3D Matching. In this case, a deep learning network is extensively trained based on a CAD model of the particular object. This method yields extremely robust results and raises the performance of industrial bin picking and pick-and-place applications to a new level. In shape-based matching, Extended Parameter Estimation is a brand-new development that allows a large number of parameters to be automatically estimated for relevant matching applications, thus avoiding the effort of generating them manually. Even newcomers to the field of machine vision can carry out complex parametrization work that would otherwise require experts.
Matching in Practice
Matching technologies have numerous applications, particularly in quality control. Optical inspection methods can detect even the smallest defects and sort out faulty products. Use cases such as wafer inspection in semiconductor production, razor blade testing, and print image inspection during bottle labeling are just a few examples of how matching can improve quality control across various industries.
Wafer Inspection
In semiconductor chip manufacturing, it is essential to ensure that no contamination or structural defects, no matter how small, are present on the wafers. Defective chips must be sorted out to ensure that only flawless products continue to further processing.
Quality control begins with capturing high-resolution images of the wafers. Models of defect-free chips are generated from these images. During production, new images are taken, with the exact position of the chips determined and adjusted through “alignment.” A pixel-by-pixel comparison is then performed between the newly captured images and the previously created models.
Since even the slightest deviations are critical, defect detection must be performed with subpixel accuracy. A matching method is used for this purpose, as it provides higher precision compared to other approaches, such as deep learning. For object recognition, Shape-Based Matching (SBM) and Normalized Cross-Correlation (NCC) are utilized. The actual comparison is carried out using a “Variation Model” to reliably identify even the smallest defects.
Detected defects are marked with red boxes. Image Source: MVTec Software
Razor Blade Inspection
The quality of razor blades is largely dependent on the precision of the blade geometry. It is particularly important to ensure that all teeth are present.
Inspection begins with identifying anchor points at the ends of the blade, which serve as the basis for subsequent measurements. A metrology model is then applied to verify whether edge transitions are consistently present at predefined positions. If an edge is missing, it is recognized as a defect.
High accuracy is also required here, as incorrect positioning of the anchor points can lead to inaccurate measurements. The matching technologies used—Shape-Based Matching (SBM) and Normalized Cross-Correlation (NCC)—ensure robust and fast detection.
Print Image Inspection
In the food and beverage industry, a product’s appearance plays a crucial role. A misaligned or damaged label can negatively influence purchasing decisions. Therefore, efficient print image inspection is essential, especially in high-speed production lines such as bottle filling.
There are two approaches: a simple rotation check, which ensures that the label is in the correct position, or a more advanced method in which the label is virtually “unwrapped” to analyze more complex deviations beyond positioning. This allows for the identification of holes or creases in the label so that defective bottles can be sorted out.
The bottles pass through the inspection system at high speed, making speed the primary requirement in this case. While accuracy is less critical than in wafer or razor blade inspections, the process must deliver results within just a few milliseconds. SBM and NCC are used here as well, as they enable fast and reliable detection.
Matching as an Enabler
All matching technologies share one common trait—they significantly drive industrial automation across various fields. Their implementation generally leads to cost and effort savings while improving production quality. When a system can rapidly, robustly, and accurately determine the location of a target object within an image, it enables automated quality inspections that were previously impossible. Therefore, matching is the ultimate enabler technology.
