Nanotechnology Now

Our NanoNews Digest Sponsors
Heifer International



Home > Press > Silicon image sensor that computes: Device speeds up, simplifies image processing for autonomous vehicles and other applications

SEAS researchers developed the first in-sensor processor that could be integrated into commercial silicon imaging sensor chips. The array (illustrated here) simplifies image processing for autonomous vehicles and other applications.
CREDIT
(Credit: Donhee Ham Research Group/Harvard SEAS)
SEAS researchers developed the first in-sensor processor that could be integrated into commercial silicon imaging sensor chips. The array (illustrated here) simplifies image processing for autonomous vehicles and other applications. CREDIT (Credit: Donhee Ham Research Group/Harvard SEAS)

Abstract:
As any driver knows, accidents can happen in the blink of an eye — so when it comes to the camera system in autonomous vehicles, processing time is critical. The time that it takes for the system to snap an image and deliver the data to the microprocessor for image processing could mean the difference between avoiding an obstacle or getting into a major accident.

Silicon image sensor that computes: Device speeds up, simplifies image processing for autonomous vehicles and other applications

Cambridge, MA | Posted on August 26th, 2022

In-sensor image processing, in which important features are extracted from raw data by the image sensor itself instead of the separate microprocessor, can speed up the visual processing. To date, demonstrations of in-sensor processing have been limited to emerging research materials which are, at least for now, difficult to incorporate into commercial systems.

Now, researchers from the Harvard John A. Paulson School of Engineering and Applied Sciences (SEAS) have developed the first in-sensor processor that could be integrated into commercial silicon imaging sensor chips ––known as complementary metal-oxide-semiconductor (CMOS) image sensors –– that are used in nearly all commercial devices that need capture visual information, including smartphones.

The research is published in Nature Electronics.

“Our work can harnesses the mainstream semiconductor electronics industry to rapidly bring in-sensor computing to a wide variety of real-world applications,” said Donhee Ham, the Gordon McKay Professor of Electrical Engineering and Applied Physics at SEAS and senior author of the paper.

Ham and his team developed a silicon photodiode array. Commercially-available image sensing chips also have a silicon photodiode array to capture images, but the team’s photodiodes are electrostatically doped, meaning that sensitivity of individual photodiodes, or pixels, to incoming light can be tuned by voltages. An array that connects multiple voltage-tunable photodiodes together can perform an analog version of multiplication and addition operations central to many image processing pipelines, extracting the relevant visual information as soon as the image is captured.

“These dynamic photodiodes can concurrently filter images as they are captured, allowing for the first stage of vision processing to be moved from the microprocessor to the sensor itself,” said Houk Jang, a postdoctoral fellow at SEAS and first author of the paper.

The silicon photodiode array can be programmed into different image filters to remove unnecessary details or noise for various applications. An imaging system in an autonomous vehicle, for example, may call for a high-pass filter to track lane markings, while other applications may call for a filter that blurs for noise reduction.

“Looking ahead, we foresee the use of this silicon-based in-sensor processor not only in machine vision applications, but also in bio-inspired applications, wherein early information processing allows for the co-location of sensor and compute units, like in the brain,” said Henry Hinton, a graduate student at SEAS and co-first author of the paper.

Next, the team aims to increase the density of photodiodes and integrate them with silicon integrated circuits.

“By replacing the standard non-programmable pixels in commercial silicon image sensors with the programmable ones developed here, imaging devices can intelligently trim out unneeded data, thus could be made more efficient in both energy and bandwidth to address the demands of the next generation of sensory applications,” said Jang.

The research was co-authored by Woo-Bin Jung, Min-Hyun Lee, Changhyun Kim, Min Park, Seoung-Ki Lee and Seongjun Park. It was supported by the Samsung Advanced Institute of Technology under Contract A30216 and by the National Science Foundation Science and Technology Center for Integrated Quantum Materials under Contract DMR-1231319.

####

For more information, please click here

Contacts:
Leah Burrows
Harvard John A. Paulson School of Engineering and Applied Sciences

Office: 617-496-1351

Copyright © Harvard John A. Paulson School of Engineering and Applied Sciences

If you have a comment, please Contact us.

Issuers of news releases, not 7th Wave, Inc. or Nanotechnology Now, are solely responsible for the accuracy of the content.

Bookmark:
Delicious Digg Newsvine Google Yahoo Reddit Magnoliacom Furl Facebook

Related Links

Paper:

Related News Press

News and information

Stability of perovskite solar cells reaches next milestone January 27th, 2023

Qubits on strong stimulants: Researchers find ways to improve the storage time of quantum information in a spin rich material January 27th, 2023

UCF researcher receives Samsung International Global Research Outreach Award: The award from the multinational electronics corporation will fund the development of infrared night vision and thermal sensing camera technology for cell phones and consumer electronics January 27th, 2023

Temperature-sensing building material changes color to save energy January 27th, 2023

Imaging

Wafer-scale 2D MoTe₂ layers enable highly-sensitive broadband integrated infrared detector January 6th, 2023

Shrinking hydrogels enlarge nanofabrication options: Researchers from Pittsburgh and Hong Kong print intricate, 2D and 3D patterns December 29th, 2022

New X-ray imaging technique to study the transient phases of quantum materials December 29th, 2022

Rapid fluorescent mapping of electrochemically induced local pH changes December 9th, 2022

Possible Futures

One of the causes of aggressive liver cancer discovered: a 'molecular staple' that helps repair broken: DNA Researchers describe a new DNA repair mechanism that hinders cancer treatment January 27th, 2023

Stability of perovskite solar cells reaches next milestone January 27th, 2023

Danish quantum physicists make nanoscopic advance of colossal significance January 27th, 2023

UC Irvine researchers decipher atomic-scale imperfections in lithium-ion batteries: Team used super high-resolution microscopy enhanced by deep machine learning January 27th, 2023

Sensors

Quantum sensors see Weyl photocurrents flow: Boston College-led team develops new quantum sensor technique to image and understand the origin of photocurrent flow in Weyl semimetals January 27th, 2023

Department of Energy announces $9.1 million for research on quantum information science and nuclear physics: Projects span the development of quantum computing, algorithms, simulators, superconducting qubits, and quantum sensors for advancing nuclear physics January 27th, 2023

Development of bio-friendly transparent temperature sensor technology that precisely measures temperature changes by light January 6th, 2023

Ultrathin vanadium oxychloride demonstrates strong optical anisotropic properties Two-dimensional material could make novel strain sensors, photodetectors and other nanodevices a reality January 6th, 2023

Discoveries

One of the causes of aggressive liver cancer discovered: a 'molecular staple' that helps repair broken: DNA Researchers describe a new DNA repair mechanism that hinders cancer treatment January 27th, 2023

Stability of perovskite solar cells reaches next milestone January 27th, 2023

Qubits on strong stimulants: Researchers find ways to improve the storage time of quantum information in a spin rich material January 27th, 2023

Temperature-sensing building material changes color to save energy January 27th, 2023

Announcements

UCF researcher receives Samsung International Global Research Outreach Award: The award from the multinational electronics corporation will fund the development of infrared night vision and thermal sensing camera technology for cell phones and consumer electronics January 27th, 2023

Temperature-sensing building material changes color to save energy January 27th, 2023

Quantum sensors see Weyl photocurrents flow: Boston College-led team develops new quantum sensor technique to image and understand the origin of photocurrent flow in Weyl semimetals January 27th, 2023

Department of Energy announces $9.1 million for research on quantum information science and nuclear physics: Projects span the development of quantum computing, algorithms, simulators, superconducting qubits, and quantum sensors for advancing nuclear physics January 27th, 2023

Interviews/Book Reviews/Essays/Reports/Podcasts/Journals/White papers/Posters

Qubits on strong stimulants: Researchers find ways to improve the storage time of quantum information in a spin rich material January 27th, 2023

Temperature-sensing building material changes color to save energy January 27th, 2023

Quantum sensors see Weyl photocurrents flow: Boston College-led team develops new quantum sensor technique to image and understand the origin of photocurrent flow in Weyl semimetals January 27th, 2023

Danish quantum physicists make nanoscopic advance of colossal significance January 27th, 2023

Automotive/Transportation

UC Irvine researchers decipher atomic-scale imperfections in lithium-ion batteries: Team used super high-resolution microscopy enhanced by deep machine learning January 27th, 2023

New nanowire sensors are the next step in the Internet of Things January 6th, 2023

NYU Tandon researchers explore a more frictionless future: Elisa Riedo’s and her lab team’s discovery of a fundamental law of friction leads to new materials that can minimize energy loss November 4th, 2022

Scientists count electric charges in a single catalyst nanoparticle down to the electron: Tenfold improvement in the sensitivity of electron holography reveals the net charge in a single platinum nanoparticle with a precision of just one electron, providing fundamental informatio October 14th, 2022

NanoNews-Digest
The latest news from around the world, FREE




  Premium Products
NanoNews-Custom
Only the news you want to read!
 Learn More
NanoStrategies
Full-service, expert consulting
 Learn More











ASP
Nanotechnology Now Featured Books




NNN

The Hunger Project