Synthetic Wavelength Imaging

This research track seeks to solve the problem of “seeing the unseen”, specifically, imaging (moving) objects through (dynamic) scattering media like living tissue, smoke, fog, turbid air and water, or clouds. A robust, flexible, and accurate approach to seeing through scattering media is still one of the major unsolved problems in imaging. When traversing through scattering media, visible light completely loses its spatial structure, which makes imaging with conventional active and passive methods impossible. A potential solution is of high relevance in many fields of research and industry, such as medical imaging, robotics, automotive driving assistance, search and rescue, or security.

Here, we tackle the problem of seeing through scatter with our Synthetic Wavelength Imaging (SWI) approaches - an ensemble of techniques that leverage the information encoded in the beat frequency between two coherent optical fields at closely spaced wavelengths to image through scatter. Our approaches probe the scene at two (or more) closely spaced optical wavelengths and computationally assemble a complex “synthetic field” at a “synthetic wavelength,” which is used for further processing. As the synthetic wavelength is the beat wavelength of the two used optical “carrier” wavelengths, it is freely tunable and can be picked orders of magnitudes larger, so that the computationally assembled synthetic field becomes immune to scatter. Moreover, the fact that “optical” light (e.g., in the VIS or NIR) is still the carrier or the information allows us to use of off-the shelf detector technology such as CMOS cameras. As the synthetic wavelength depends solely on the difference between the two used optical “carrier” wavelengths, the same synthetic wavelength can be realized for different carrier wavelengths pairs in different wavebands best suited for specific tasks (e.g., minimizing absorption).

Besides the various SWI-based approached to image through scattering environments discussed below, our group has harnessed SWI as “high precision Time-of-Flight imaging” method to recover 3D profiles of optically rough surfaces in industrial inspection and computer vision applications (please see ”High-Precision Time-of-Flight Sensing with Synthetic Waves” below for more information).

SWI-based Imaging through scattering media (multi-shot and single-shot)

The robustness of the synthetic field to scattering effects can be exploited to image hidden objects through scattering media or around corners. In many application examples, this can be done by performing holography at the synthetic wavelength, i.e., capturing a synthetic wavelength hologram (SWH) of the hidden object through the scattering environment. The SWH can then be computationally back propagated at the synthetic wavelength to reconstruct the obscured objects obscured. In our recent work, we have demonstrated both imaging hidden objects around corners (so-called “Non-Line-of-Sight imaging”) and through thin and thick volumetric scattering media via SWI. Moreover, we have developed a method to acquire the synthetic field information in single-shot, using only an off-the-shelf CMOS camera, which is crucial for the motion-robust measurement in scenes with moving objects and/or a moving imager and/or moving scattering media, like smoke, fog, or dynamic turbulence.

Synthetic Wavelength Holography was Northwestern’s #3 media story in the week of Nov. 18-24, 2021 with a total reach of 3.2 million people within this week.


Official Northwestern research news release:
“New holographic camera sees the unseen with high precision” (Northwestern Now Research News, Nov. 17, 2021)
For Journalists: News release for media contacts and assets.


Research news featured on Northwestern websites:
[Northwestern McCormick School of Engineering (retrieved Nov. 18, 2021)]
[Northwestern Electrical and Computer Engineering (retrieved Nov. 18, 2021)]
[Northwestern Computer Science (retrieved Nov. 18, 2021)]


Research news featured on external websites (selection):

Newsweek: “This Holographic Camera Can See Around Corners, Under Human Skin” (Nov. 19, 2021)

NSF.gov: “High-resolution camera can see around corners and through scattering media” (Dec. 7, 2021)

SCIENTIFIC AMERICAN: “Holographic Camera Instantly Peeks around Obstacles” (Jan. 19, 2022)

Optics.org: “Northwestern University camera sees around corners” (Nov. 18, 2021)

Phys.org: “New holographic camera sees the unseen with high precision” (Nov. 17, 2021)

Daily Mail: “Powerful holographic camera is developed that can see through …” (Nov. 19, 2021)

Photonics.com: “Holographic Camera Peers Through Fog, Around Corners” (Dec., 2021)

AI in Healthcare: “Novel camera images objects around corners, behind barriers” (Nov. 19, 2021)

Science X: “Best of Last Week – Camera sees around corners, ….” (Nov. 22, 2021)

New Atlas: “Holographic camera reconstructs objects around corners in milliseconds” (Nov. 17, 2021)

IFL Science: “New Holographic Camera Can See Around Corners – Or Inside Your Skull” (Nov. 19, 2021)

Mashable: “Researchers Develop Camera That Can See Through Skin And Around Corners” (Nov. 18, 2021)

Universal-Sci: “Scientists developed a remarkable camera that can see around corners” (Nov. 17, 2021)

ZME Science: “Holographic camera can see around corners or even through the skin” (Nov. 18, 2021)

More…
[ScienceDaily] | [SciTechDaily] | [SpaceLiveNews] | [Unite AI] | [AZO Optics] | [Newsfounded] | [Nerdswire] | [Florida News Times] | [PetaPixel] | [Brinkwire] | [Remonews] | [Futurism] | [InterestingEngineering] | [TodayIn24] | [Engineering and Technology] | [XNewsNet] | [NewsBeezer] | [CVBJ] | [The Washington Newsday] | [CaliforniaNewsTimes] | [ScienceTimes] | [Techeblog] | [WoonderfulEngineering] | [6parkNews] | [Newscon] | [iTechpost] | [WorldIndustrialReporter] | [California18] | [Jioforme] | [Techgamingreport] | [Newsbrig] | [Algulf] | [Funtitech] | [Leakherald] | [Shouzy] | [NewsInseconds] | [MessageToEagle] | [XNewsNet] | [MarketResearchTelecast] | [ScienceSprings] | [MORE…]

Nürnberger Nachrichten (German): “Fürther Wissenschaftler entwickelt Kamera die um die Ecke blicken kann” (May 10, 2022)

Pressetext (German): “Neuartige Kamera zeigt versteckte Objekte” (Nov. 18, 2021)

c’t (German): “Holo-Kamera sieht um die Ecke " (Dez. 12, 2021)

VIDEO: Yahoo News (German): “Neue Technologie: Holografische Kamera kann um Ecken sehen” (Nov. 18, 2021)

FutureZone (German): “Neue holografische Kamera macht das Unsichtbare sichtbar” (Nov. 17, 2021)

Germanic (German): “Neue holografische Kamera sieht das Unsichtbare mit hoher Präzision” (Nov. 17, 2021)

EuropaPress (Spanish): “Nueva cámara holográfica ve lo invisible con alta precisión” (Nov. 17, 2021)

Trust My Science (French): “Une caméra qui permet de voir à travers les objets” (Nov. 17, 2021)

Engineers Online (Dutch): “Nieuwe holografische camera kijkt om hoekjes en ziet het ongeziene met grote precisie” (Nov. 23, 2021)

More…
[VIDEO: MSN News (German)] | [VIDEO: Wissen.de (German)] | [NachWelt (German)] | [Inuit (German)] | [ICT (German)] | [SputnikNews (Turkish)] | [Descopera (Romanian)] | RT News (Spanish) | [Techvet (Czech)] | [KisaKisa (Turkish)] | [Tamindir (Turkish)] | [Fizikist (Turkish)] | [ScienceNet (Chinese)] | [ITMedia (Chinese)] | [MORE…]

Selected Publications

Synthetic Wavelength Imaging: Utilizing Spectral Correlations for High-Precision Time-of-Flight Sensing.

Computational Imaging for Scene Understanding: Transient, Spectral, and Polarimetric Analysis, 2024.

Synthetic Light-in-Flight (SLiF)

In this project, we extend the capabilities of synthetic wavelength imaging by probing the scene at multiple synthetic wavelengths, effectively assembling a synthetic pulse. This synthetic pulse can then be computationally advanced to visualize light propagation (happening at the speed of light) in arbitrary volumetric scenes or through scattering media. The resulting “Synthetic Light-in-Flight” (SLiF) videos and measurements provide valuable insights into light-matter interactions on ultrafast timescales, enabling multiple potential future applications in, e.g., medical imaging, industrial inspection, or imaging through degraded visual environments. Typical light-in-flight approaches require specialized equipment, such as ultrashort pulse light sources and/or high-speed detectors. In contrast, SLiF uses only tunable continuous wave lasers and standard “slow” CMOS cameras. Moreover, the captured complex synthetic fields can be freely manipulated in the computer after their acquisition, which allows for spatial and temporal shaping of different sets of pulses from the same set of measurements to maximize the decoded information output for each scene. In addition, the relative speed at which the pulse travels through the scene can be used to characterize physical scene properties such as depth or indices of refraction.

Selected Publications
SWI-based imaging through fibers

Many small structures within the body are difficult or impossible to visualize using external imaging methods. Endoscopy has long been the preferred technique, when feasible, for observing such structures. Thinner endoscopes can access more regions with minimal disruption to surrounding tissue. However, the benefits of thinner endoscopes are often counterbalanced by imaging challenges. Optical fields relayed through each fiber core can be significantly distorted due to phase scrambling caused by surface irregularities at the fiber ends or the presence of multi-mode cores. Additionally, achieving high-quality endoscopic images typically requires the fiber tip to be positioned close to the target or necessitates the integration of a lens. Imaging structured in 3D or through scattering layers beyond the fiber tip is often not feasible with conventional methods.

In this project, we address these limitations by integrating Synthetic Wavelength Imaging with fiber endoscopy. The resulting system eliminates the need for lenses and is inherently resilient to phase scrambling introduced by scattering and fiber bending. Using this approach, we have successfully demonstrated the endoscopic imaging of features approximately 750 micrometers in size on objects located behind a scattering layer. This advancement opens new possibilities for achieving spatially resolved three-dimensional imaging of structures concealed beneath tissue, significantly expanding the potential of fiber endoscopes for biomedical applications.

Selected Publications
High-Precision Time-of-Flight Sensing with Synthetic Waves

The depth resolution of state-of-the-art Time-of-Flight cameras is often restricted by technical limitations in electronics manufacturing and does not satisfy the requirements in industrial inspection in many cases. High precision Time-of-Flight principles like single-wavelength interferometry are limited to smooth surfaces and have problems at parts with larger height variation, which renders them impractical for many applications in industrial inspection. We close the gap and solve this problem by applying teachings known from signal processing and multiwavelength interferometry to measure the object at a so-called “synthetic wavelength” whose size can be freely tuned over multiple orders. Paired with focal plane array sensors, our techniques acquire sub-mm resolution full field 3D data of any surface type within milliseconds or even in single-shot. The formation and computational shaping of “synthetic pulses” enables advanced inspection possibilities, e.g., to detect small defects. The synthetic wave principle is of interest for many applications in industrial inspection, because it combines the benefits of ToF imaging (compact, no occlusions) with the high resolution of triangulation (‘structured light’) methods.

Selected Publications