Researchers at Princeton University and the University of Washington have created an ultra-compact camera the size of a coarse grain of salt. The new system, which is studded with 1.6 million cylindrical posts, can produce full-colour images on par with a conventional compound camera lens 500,000 times larger in volume.
The project was undertaken by researchers at Princeton University and the University of Washington, with support coming from the National Science Foundation, the U.S. Department of Defense, the UW Reality Lab, Facebook, Google, Futurewei Technologies, and Amazon.
Announced by the researchers in a paper published November 29th in Nature Communications, the new optical system relies on a technology called a metasurface, which can be produced much like a computer chip. The camera measures half a millimetre wide and is studded with 1.6 million cylindrical posts that function as optical antennas. With the help of machine learning-based algorithms, the posts’ interactions with light combine to produce images.
A key innovation in the camera’s creation was the integrated design of the optical surface and the signal processing algorithms that produce the image. This boosted the camera’s performance in natural light conditions, in contrast to previous metasurface cameras that required the pure laser light of a laboratory or other ideal conditions to produce high-quality images.
Felix Heide, the study’s senior author and an assistant professor of computer science at Princeton, said: “Whilst sensors have been miniaturized over the last decades, the lenses of point-and-shoot are bulky — largely defining the size of the entire camera. To build a drastically smaller camera, we had to devise a new type of lens system that does not use conventional glass optics but an array of nano-sized antennas.
“Instead of grinding glass or injection-moulding glass, these new optics can be fabricated at an ultra-small scale in a similar fashion to computer chips. To be able to design these novel lenses, we needed to solve a number of technical challenges. The key one was to design an array of millions of nano-scatterers jointly with a computational reconstruction. We developed an artificial intelligence method to allow the computer to ‘learn’ these novel optics”.
Previous ultracompact metasurface lenses have suffered from major image distortions, small fields of view, and limited ability to capture the full spectrum of visible light. Image: Princeton University.
Enabled by a joint design of the camera’s hardware and computational processing, the system could enable minimally invasive endoscopy with medical robots to diagnose and treat diseases, and improve imaging for other robots with size and weight constraints. Arrays of thousands of such cameras could be used for full-scene sensing, turning surfaces into cameras.
When Automate Pro Europe asked what was most pleasing about the project for Heide, he responded: “In my opinion, the fact that we were able to devise a neural design method that overcomes the limitations of conventional full-wave simulation methods, like Finite-difference time-domain methods, by over 3000x. This could be a game-changer and may allow us to build not only ultra-small lenses but also compress large lenses, e.g., for space telescopes, to a much smaller form factor and weight”.
Heide and his colleagues are now working to add more computational abilities to the camera itself. Beyond optimizing image quality, they would like to add capabilities for object detection and other sensing modalities relevant for medicine and robotics.
You can find more information about the camera the size of a grain of salt on Princeton University’s website.
Stay up to date with the most recent automation, computer vision, machine vision and robotics news on Automate Pro Europe, CVPro, MVPro and RBPro.
