Nanotechnology Now

Our NanoNews Digest Sponsors
Heifer International



Home > Press > New chip ramps up AI computing efficiency

The NeuRRAM chip is not only twice as energy efficient as state-of-the-art, it’s also versatile and delivers results that are just as accurate as conventional digital chips.
CREDIT
David Baillot/University of California San Diego
The NeuRRAM chip is not only twice as energy efficient as state-of-the-art, it’s also versatile and delivers results that are just as accurate as conventional digital chips. CREDIT David Baillot/University of California San Diego

Abstract:
AI-powered edge computing is already pervasive in our lives. Devices like drones, smart wearables, and industrial IoT sensors are equipped with AI-enabled chips so that computing can occur at the “edge” of the internet, where the data originates. This allows real-time processing and guarantees data privacy.

New chip ramps up AI computing efficiency

Menlo Park, CA | Posted on August 19th, 2022

However, AI functionalities on these tiny edge devices are limited by the energy provided by a battery. Therefore, improving energy efficiency is crucial. In today’s AI chips, data processing and data storage happen at separate places – a compute unit and a memory unit. The frequent data movement between these units consumes most of the energy during AI processing, so reducing the data movement is the key to addressing the energy issue.

Stanford University engineers have come up with a potential solution: a novel resistive random-access memory (RRAM) chip that does the AI processing within the memory itself, thereby eliminating the separation between the compute and memory units. Their “compute-in-memory” (CIM) chip, called NeuRRAM, is about the size of a fingertip and does more work with limited battery power than what current chips can do.

“Having those calculations done on the chip instead of sending information to and from the cloud could enable faster, more secure, cheaper, and more scalable AI going into the future, and give more people access to AI power,” said H.-S Philip Wong, the Willard R. and Inez Kerr Bell Professor in the School of Engineering.

“The data movement issue is similar to spending eight hours in commute for a two-hour workday,” added Weier Wan, a recent graduate at Stanford leading this project. “With our chip, we are showing a technology to tackle this challenge.”

They presented NeuRRAM in a recent article in the journal Nature. While compute-in-memory has been around for decades, this chip is the first to actually demonstrate a broad range of AI applications on hardware, rather than through simulation alone.

Putting computing power on the device
To overcome the data movement bottleneck, researchers implemented what is known as compute-in-memory (CIM), a novel chip architecture that performs AI computing directly within memory rather than in separate computing units. The memory technology that NeuRRAM used is resistive random-access memory (RRAM). It is a type of non-volatile memory – memory that retains data even once power is off – that has emerged in commercial products. RRAM can store large AI models in a small area footprint, and consume very little power, making them perfect for small-size and low-power edge devices.

Even though the concept of CIM chips is well established, and the idea of implementing AI computing in RRAM isn’t new, “this is one of the first instances to integrate a lot of memory right onto the neural network chip and present all benchmark results through hardware measurements,” said Wong, who is a co-senior author of the Nature paper.

The architecture of NeuRRAM allows the chip to perform analog in-memory computation at low power and in a compact-area footprint. It was designed in collaboration with the lab of Gert Cauwenberghs at the University of California, San Diego, who pioneered low-power neuromorphic hardware design. The architecture also enables reconfigurability in dataflow directions, supports various AI workload mapping strategies, and can work with different kinds of AI algorithms – all without sacrificing AI computation accuracy.

To show the accuracy of NeuRRAM’s AI abilities, the team tested how it functioned on different tasks. They found that it’s 99% accurate in letter recognition from the MNIST dataset, 85.7% accurate on image classification from the CIFAR-10 dataset, 84.7% accurate on Google speech command recognition and showed a 70% reduction in image-reconstruction error on a Bayesian image recovery task.

“Efficiency, versatility, and accuracy are all important aspects for broader adoption of the technology,” said Wan. “But to realize them all at once is not simple. Co-optimizing the full stack from hardware to software is the key.”

“Such full-stack co-design is made possible with an international team of researchers with diverse expertise,” added Wong.

Fueling edge computations of the future
Right now, NeuRRAM is a physical proof-of-concept but needs more development before it’s ready to be translated into actual edge devices.

But this combined efficiency, accuracy, and ability to do different tasks showcases the chip’s potential. “Maybe today it is used to do simple AI tasks such as keyword spotting or human detection, but tomorrow it could enable a whole different user experience. Imagine real-time video analytics combined with speech recognition all within a tiny device,” said Wan. “To realize this, we need to continue improving the design and scaling RRAM to more advanced technology nodes.”

“This work opens up several avenues of future research on RRAM device engineering, and programming models and neural network design for compute-in-memory, to make this technology scalable and usable by software developers”, said Priyanka Raina, assistant professor of electrical engineering and a co-author of the paper.

If successful, RRAM compute-in-memory chips like NeuRRAM have almost unlimited potential. They could be embedded in crop fields to do real-time AI calculations for adjusting irrigation systems to current soil conditions. Or they could turn augmented reality glasses from clunky headsets with limited functionality to something more akin to Tony Stark’s viewscreen in the Iron Man and Avengers movies (without intergalactic or multiverse threats – one can hope).

If mass produced, these chips would be cheap enough, adaptable enough, and low power enough that they could be used to advance technologies already improving our lives, said Wong, like in medical devices that allow home health monitoring.

They can be used to solve global societal challenges as well: AI-enabled sensors would play a role in tracking and addressing climate change. “By having these kinds of smart electronics that can be placed almost anywhere, you can monitor the changing world and be part of the solution,” Wong said. “These chips could be used to solve all kinds of problems from climate change to food security.”

Additional co-authors of this work include researchers from University of California San Diego (co-lead), Tsinghua University, University of Notre Dame, and University of Pittsburgh. Former Stanford graduate student Sukru Burc Eryilmaz is also a co-author. Wong is a member of Stanford Bio-X and the Wu Tsai Neurosciences Institute, and an affiliate of the Precourt Institute for Energy. He is also Faculty Director of the Stanford Nanofabrication Facility and the founding faculty co-director of the Stanford SystemX Alliance – an industrial affiliate program at Stanford focused on building systems.

This research was funded by the National Science Foundation Expeditions in Computing, SRC JUMP ASCENT Center, Stanford SystemX Alliance, Stanford NMTRI, Beijing Innovation Center for Future Chips, National Natural Science Foundation of China, and the Office of Naval Research.

####

For more information, please click here

Contacts:
Jill Wu
Stanford University

Copyright © Stanford University

If you have a comment, please Contact us.

Issuers of news releases, not 7th Wave, Inc. or Nanotechnology Now, are solely responsible for the accuracy of the content.

Bookmark:
Delicious Digg Newsvine Google Yahoo Reddit Magnoliacom Furl Facebook

Related Links

ARTICLE TITLE

Related News Press

News and information

Chung-Ang University researchers develop novel DNA biosensor for early diagnosis of cervical cancer: The electrochemical sensor, made of a graphitic nano-onion/molybdenum disulfide nanosheet composite, detects human papillomavirus (HPV)-16 and HPV-18, with high specificity September 8th, 2023

New compound unleashes the immune system on metastases September 8th, 2023

Machine learning contributes to better quantum error correction September 8th, 2023

Tests find no free-standing nanotubes released from tire tread wear September 8th, 2023

Wearable electronics

CityU awarded invention: Soft, ultrathin photonic material cools down wearable electronic devices June 30th, 2023

Liquid metal sticks to surfaces without a binding agent June 9th, 2023

Breaking through the limits of stretchable semiconductors with molecular brakes that harness light June 9th, 2023

Vertical electrochemical transistor pushes wearable electronics forward: Biomedical sensing is one application of efficient, low-cost transistors January 20th, 2023

Tin selenide nanosheets enables to develop wearable tracking devices December 9th, 2022

Internet-of-Things

New nanowire sensors are the next step in the Internet of Things January 6th, 2023

Lightening up the nanoscale long-wavelength optoelectronics May 13th, 2022

Law enforcement/Anti-Counterfeiting/Security/Loss prevention

How randomly moving electrons can improve cyber security May 27th, 2022

Self-propelled, endlessly programmable artificial cilia: Simple microstructures that bend, twist and perform stroke-like motions could be used for soft robotics, medical devices and more May 6th, 2022

Turning any camera into a polarization camera: Metasurface attachment can be used with almost any optical system, from machine vision cameras to telescopes March 18th, 2022

Govt.-Legislation/Regulation/Funding/Policy

Previously unknown pathway to batteries with high energy, low cost and long life: Newly discovered reaction mechanism overcomes rapid performance decline in lithium-sulfur batteries September 8th, 2023

Quantum powers researchers to see the unseen September 8th, 2023

Chloride ions from seawater eyed as possible lithium replacement in batteries of the future August 11th, 2023

Tattoo technique transfers gold nanopatterns onto live cells August 11th, 2023

Possible Futures

Chung-Ang University researchers develop novel DNA biosensor for early diagnosis of cervical cancer: The electrochemical sensor, made of a graphitic nano-onion/molybdenum disulfide nanosheet composite, detects human papillomavirus (HPV)-16 and HPV-18, with high specificity September 8th, 2023

New compound unleashes the immune system on metastases September 8th, 2023

Machine learning contributes to better quantum error correction September 8th, 2023

Tests find no free-standing nanotubes released from tire tread wear September 8th, 2023

Chip Technology

University of Chicago scientists invent smallest known way to guide light: 2D optical waveguides could point way to new technology August 11th, 2023

Researchers discover a potential application of unwanted electronic noise in semiconductors: Random telegraph noises in vanadium-doped tungsten diselenide can be tuned with voltage polarity August 11th, 2023

The present and future of computing get a boost from new research July 21st, 2023

Scientists edge toward scalable quantum simulations on a photonic chip: A system using photonics-based synthetic dimensions could be used to help explain complex natural phenomena June 30th, 2023

Discoveries

Electronic detection of DNA nanoballs enables simple pathogen detection Peer-Reviewed Publication September 8th, 2023

Training quantum computers: physicists win prestigious IBM Award September 8th, 2023

Unlocking quantum potential: Harnessing high-dimensional quantum states with QDs and OAM: Generation of nearly deterministic OAM-based entangled states offers a bridge between photonic technologies for quantum advancements September 8th, 2023

Tests find no free-standing nanotubes released from tire tread wear September 8th, 2023

Announcements

Electronic detection of DNA nanoballs enables simple pathogen detection Peer-Reviewed Publication September 8th, 2023

Training quantum computers: physicists win prestigious IBM Award September 8th, 2023

Machine learning contributes to better quantum error correction September 8th, 2023

Tests find no free-standing nanotubes released from tire tread wear September 8th, 2023

Military

Quantum powers researchers to see the unseen September 8th, 2023

Scientists edge toward scalable quantum simulations on a photonic chip: A system using photonics-based synthetic dimensions could be used to help explain complex natural phenomena June 30th, 2023

The picture of health: Virginia Tech researchers enhance bioimaging and sensing with quantum photonics June 30th, 2023

With new experimental method, researchers probe spin structure in 2D materials for first time: By observing spin structure in “magic-angle” graphene, a team of scientists led by Brown University researchers have found a workaround for a long-standing roadblock in the field of two May 12th, 2023

Artificial Intelligence

Data can now be processed at the speed of light! April 14th, 2023

Light meets deep learning: computing fast enough for next-gen AI March 24th, 2023

Stanford researchers develop a new way to identify bacteria in fluids: An innovative adaptation of the technology in an old inkjet printer plus AI-assisted imaging leads to a faster, cheaper way to spot bacteria in blood, wastewater, and more March 3rd, 2023

3D-printed decoder, AI-enabled image compression could enable higher-res displays December 9th, 2022

Grants/Sponsored Research/Awards/Scholarships/Gifts/Contests/Honors/Records

Training quantum computers: physicists win prestigious IBM Award September 8th, 2023

The present and future of computing get a boost from new research July 21st, 2023

Manchester graphene spin-out signs $1billion game-changing deal to help tackle global sustainability challenges: Landmark deal for the commercialisation of graphene April 14th, 2023

Optical switching at record speeds opens door for ultrafast, light-based electronics and computers: March 24th, 2023

NanoNews-Digest
The latest news from around the world, FREE




  Premium Products
NanoNews-Custom
Only the news you want to read!
 Learn More
NanoStrategies
Full-service, expert consulting
 Learn More











ASP
Nanotechnology Now Featured Books




NNN

The Hunger Project