Computational 3D Metrology for Industrial Inspection and Mixed Reflectance Scenes

Fringe Projection Triangulation is known as one of the “gold standard” principles in optical 3D imaging due to its beneficial noise characteristics and the very high precision that can be achieved. However, the principle is designed for diffuse surfaces and suffers heavy artifacts when applied to shiny or specular surfaces. In this research track, we investigate novel computational and hardware-based methods to suppress the specular reflection component for a broad variety of “shiny” object surfaces, which allows us to measure these surfaces at high accuracy in single-shot using different flavors of Fringe Projection Triangulation. Moreover, we explore novel calibration paradigms that improve the measurement accuracy in industrial inspection settings.

Phase Measuring Deflectometry (PMD) is a well established method in optical metrology that allows for accurate 3D measurements of specular surfaces. Standard single-camera PMD setups suffer an inherent limitation: the height-normal ambiguity problem. Current solutions to this problem either require prior knowledge of the object’s shape or exploit additional cameras, which impacts the compactness and usability of the device. In this research track, we propose a novel PMD paradigm that leverages polarization cues to solve the ambiguity problem. Our method is capable of simultaneously measuring the shape and normal field of complex surfaces without any prior knowledge about the object and without adding additional cameras.

The depth resolution of state-of-the-art Time-of-Flight cameras is often restricted by technical limitations in electronics manufacturing and does not satisfy the requirements in industrial inspection in many cases. High precision Time-of-Flight principles like single-wavelength interferometry are limited to smooth surfaces and have problems at parts with larger height variation, which renders them impractical for many applications in industrial inspection. We close the gap and solve this problem by applying teachings known from signal processing and multiwavelength interferometry to measure the object at a so-called “synthetic wavelength” whose size can be freely tuned over multiple orders. Paired with focal plane array sensors, our techniques acquire sub-mm resolution full field 3D data of any surface type within milliseconds or even in single-shot. The formation and computational shaping of “synthetic pulses” enables advanced inspection possibilities, e.g., to detect small defects. The synthetic wave principle is of interest for many applications in industrial inspection, because it combines the benefits of ToF imaging (compact, no occlusions) with the high resolution of triangulation (‘structured light’) methods.

Event-based structured light systems have recently been introduced as an exciting alternative to conventional frame-based triangulation systems for the 3D measurements of diffuse surfaces. Important benefits include the fast capture speed and the high dynamic range provided by the used event camera - albeit at the cost of increased noise. So far, both low-accuracy event-based and high-accuracy frame-based 3D imaging systems are tailored to a specific surface type, such as diffuse or specular, and can not be used for a broader class of object surfaces (“mixed reflectance scenes"). In this research track, we present a novel event-based structured light system that enables fast 3D imaging of mixed reflectance scenes with high accuracy. We decompose the measured signals into diffuse, two-bounce specular, and other multi-bounce reflections. The diffuse surfaces in the scene are reconstructed using triangulation. Then, the reconstructed diffuse scene parts are leveraged as a ”display" to evaluate the specular scene parts via deflectometry.

This novel procedure allows us to use the entire scene as a virtual screen, using only a scanning laser and an event camera. The resulting system achieves fast and motion-robust reconstructions of mixed reflectance scenes with high accuracy. Moreover, we demonstrate an ”ultrafast" capture mode (250Hz) for the 3D measurement of diffuse scenes.

(Multi-) line triangulation systems deliver a nearly perfect 3D profile of the measured part for each projected line in single-shot. However, the number of projected lines is limited by physical and information theoretical restrictions, leading to low data densities. This research track introduces novel concepts to increase the data density of multi-line triangulation up to the physical and information-theoretical limit, leading to accurate and dense 3D models. Each model is captured in single-shot, which allows for “3D-videos” of fast-moving objects.

In a related concept dubbed “Flying Triangulation”, we pair a “sparse” single-shot line triangulation sensor projecting ~10 straight narrow lines with sophisticated real-time registration algorithms. The captured sparse 3D line profiles are registered to each other ‘on-the-fly’ while the sensor is free-hand guided around the object, or the object is moved in front of the sensor (see videos). The result is a dense 3D model of the object with high depth precision.

This research track introduces a series of systems that only require commodity devices such as screens, (web-) cameras, low-end tablets or mobile phones to capture high-quality 3D data: The developed “Mobile Multiview Deflectometry” system exploits screen and front camera of mobile devices for qualitative deflectometry-based measurements. It works without the need for a calibration and is optimized for specular surfaces. To compensate for the small screen, a multi-view registration technique is applied so that large surfaces can be densely reconstructed in their entirety. The “SkinScan” sensor principle uses the same hardware components but exploits photometric stereo-inspired algorithms for the measurement of matte object surfaces.