Researchers Target AI Efficiency Gains with Stochastic Hardware

Kaushik Roy, Adarsh Kosta, Tanvi Sharma, and colleagues from Purdue University and the Georgia Institute of Technology detail advancements in next-generation artificial intelligence hardware designed to overcome limitations in traditional von Neumann architectures. Published in Frontiers in Science on December 16, 2025, their work addresses the “memory wall” problem—the separation of compute units and memory—which hinders computational efficiency. This article explores compute-in-memory (CIM) techniques utilizing diverse memory types—including embedded non-volatile memory, SRAM, DRAM, and flash memory—and investigates the potential of stochastic hardware to enhance energy efficiency by exploiting error resilience within AI algorithms. Advancements in Machine Learning Algorithms Advancements in machine learning algorithms are driving a need for more efficient hardware solutions. The relentless progress of AI across sectors like healthcare and automotive is increasing computational demands. Traditional hardware, based on the von Neumann architecture, suffers from a “memory wall” – a bottleneck created by separating compute and memory units. This separation limits efficiency and necessitates novel approaches to overcome these limitations for diverse learning algorithms. To address the “memory wall”, researchers are exploring compute-in-memory (CIM) techniques. CIM integrates computing capabilities directly into the memory system, potentially alleviating energy consumption and latency issues. This approach utilizes different memory technologies, including embedded non-volatile memory, static random-access memory, dynamic random-access memory, and flash memory. The goal is to perform computations within the memory array itself, rather than constantly moving data back and forth. Furthermore, the source highlights the potential of stochastic hardware, leveraging the error-resiliency inherent in many AI algorithms. This allows for approximate computing – faster operations with reduced energy consumption – while maintaining acceptable system-level accuracy. Combining these approaches – CIM and stochasticity – with brain-inspired algorithms like spiking neural networks, could lead to a “converged platform” optimized for diverse AI applications and co-designed hardware and algorithms. Evolution of AI Models and Hardware The relentless advancement of artificial intelligence necessitates more efficient hardware solutions, particularly due to limitations in traditional systems. These traditional systems, based on the von Neumann architecture, suffer from a “memory wall” – a bottleneck caused by the separation of compute and memory units. This separation limits computational efficiency and increases energy consumption, hindering the processing of increasingly complex AI algorithms and large models like those used in language and vision tasks. A key approach to overcoming the memory wall is “compute-in-memory” (CIM). CIM integrates computation within the memory array itself, rather than shuttling data back and forth. The source details that CIM can utilize various memory technologies—embedded non-volatile memory, SRAM, DRAM, and flash—to create energy-efficient AI hardware. Furthermore, leveraging the inherent error-resiliency of AI algorithms alongside stochastic hardware, like spin–orbit transfer magnetic tunnel junctions, can further reduce energy consumption. The evolution of AI models—from multi-layer perceptrons to transformers—has driven increasing computational demands. The source highlights a shift toward brain-like computation, such as spiking neural networks (SNNs), which aim for sparse, event-driven processing. Achieving efficient implementation requires optimization at the hardware level, and a co-design approach—considering both algorithms and hardware constraints—is crucial for developing a converged platform for both artificial and spiking neural networks. The Challenge of the Memory Wall A significant challenge in modern computing is the “memory wall” problem, stemming from the separation of compute and memory units in the traditional von Neumann architecture. This separation causes substantial energy consumption and latency as data is constantly shuttled between these units. Addressing this issue is critical for enhancing AI computational efficiency and enabling faster processing of complex algorithms, especially with the rise of multi-billion parameter models. To overcome the memory wall, researchers are exploring “compute-in-memory” (CIM) paradigms. CIM aims to integrate computing capabilities directly within the memory array itself, performing operations where the data resides. Different memory technologies – including embedded non-volatile memory, SRAM, DRAM, and flash memory – are being investigated for CIM implementation. This approach promises to alleviate bottlenecks and improve energy efficiency for AI workloads. Furthermore, the source highlights the potential to leverage the error-resiliency inherent in AI algorithms. By designing hardware that is approximate yet maintains system-level accuracy, faster operations with reduced energy consumption can be achieved. This, combined with CIM, supports an algorithm-hardware co-design approach, essential for developing brain-inspired AI systems suitable for diverse applications.

Von Neumann Architecture Limitations Traditional hardware relies on the von Neumann architecture, which separates compute and memory units. This separation creates a significant bottleneck known as the “memory wall” problem, responsible for most energy consumption and latency in computing systems. The source highlights that constantly shuttling data between these units limits efficiency, especially with increasingly complex AI models requiring substantial computational resources and energy. To overcome the limitations of the von Neumann architecture, researchers are exploring “compute-in-memory” (CIM) paradigms. CIM aims to integrate computing capabilities directly within the memory array itself, eliminating the need for constant data transfer. The source details that CIM utilizes different memory technologies – including embedded non-volatile memory, SRAM, DRAM, and flash memory – to develop more energy-efficient AI hardware and tackle the memory wall problem. AI algorithms exhibit a degree of error-resiliency, which can be exploited for hardware optimization. The source suggests that approximate computing, leveraging this resilience, can lead to faster operations with reduced energy consumption without significantly impacting overall system accuracy. This approach, alongside CIM, represents a pathway toward brain-inspired AI algorithms and hardware through co-design principles, addressing both algorithmic and hardware constraints. Compute-in-Memory (CIM) Techniques Compute-in-memory (CIM) techniques are emerging as a solution to the “memory wall” problem inherent in traditional von Neumann architectures. This bottleneck arises from the separation of compute and memory units, leading to significant energy consumption and latency. CIM aims to alleviate this by integrating computation within the memory array itself, offering a promising path towards more efficient AI hardware. The source highlights CIM’s potential for tackling this issue and supporting essential AI compute functions. The source details that various memory technologies are being explored for CIM implementation, including embedded non-volatile memory, static random-access memory (SRAM), dynamic random-access memory (DRAM), and flash memory. This broad approach allows for energy-efficient AI hardware development by directly addressing the limitations of data movement between processing and storage. The article suggests CIM is critical for realizing brain-like computation and optimizing fundamental operations for machine learning workloads. Furthermore, the source indicates that combining CIM with the error-resiliency of AI algorithms—like those utilizing spike timing-dependent plasticity—can further enhance energy efficiency. This approach allows for approximate computing, potentially unlocking novel capabilities and faster operations while maintaining system accuracy. A co-design approach, considering both algorithms and hardware, is key to developing a “converged platform” for both artificial and spiking neural networks. CIM for Different Memory Types Compute-in-memory (CIM) is presented as a promising solution to overcome the “memory wall” problem inherent in traditional von Neumann architectures. This bottleneck stems from the separation of compute and memory units, causing significant energy consumption and latency. CIM aims to alleviate this issue by integrating computational operations within the memory array itself, enabling more efficient processing. The article details state-of-the-art developments in CIM for various memory types to support essential AI computations. The source highlights that CIM can be implemented using different memory technologies including embedded non-volatile memory, static random-access memory (SRAM), dynamic random-access memory (DRAM), and flash memory. This versatility allows for energy-efficient AI hardware development by directly addressing the limitations of data movement between processing and storage. Furthermore, the article suggests exploring stochastic hardware alongside CIM to potentially unlock novel capabilities and improve energy efficiency for diverse AI workloads. Beyond CIM, the source indicates that leveraging the error-resiliency of AI algorithms can further enhance energy efficiency. This involves developing approximate hardware that performs faster operations with reduced energy consumption while maintaining system-level accuracy. Ultimately, the article advocates for a co-design approach—simultaneously optimizing algorithms and hardware—to create a “converged platform” suitable for both artificial and spiking neural networks. Neuromorphic Hardware and Spiking Neural Networks The article highlights the limitations of traditional von Neumann architecture hardware due to the separation of compute and memory units, creating what’s known as the “memory wall” problem. This separation causes significant energy consumption and latency in computing systems, hindering the efficient implementation of increasingly complex AI algorithms. Addressing this bottleneck is critical for enabling faster, more effective processing and reducing the resource demands of applications like autonomous drone navigation and large language models. Compute-in-memory (CIM) techniques offer a promising solution by integrating computation directly within the memory system. This approach utilizes different memory technologies – including embedded non-volatile memory, SRAM, DRAM, and flash memory – to perform computations within the memory array itself, alleviating the data movement bottleneck. Furthermore, the development of neuromorphic hardware capable of accelerating biologically inspired algorithms, such as spiking neural networks (SNNs), is also detailed as a key advancement. The article also explores leveraging the error-resilience of AI algorithms to create more energy-efficient hardware. Stochastic hardware, for example, can exploit this resilience, potentially unlocking novel capabilities. Ultimately, a co-design approach – optimizing both hardware and algorithms – is proposed to create a “converged platform” suitable for both artificial and spiking neural networks, ultimately supporting diverse AI applications and achieving brain-like computation. Stochastic Hardware for Energy Efficiency Stochastic hardware is presented as a potential method for improving energy efficiency in AI systems. The source highlights that leveraging the error-resiliency inherent in AI algorithms can enable the development of hardware that operates faster and consumes less energy, all while maintaining acceptable system-level accuracy. This approach suggests a move toward approximate computing, trading off absolute precision for gains in speed and power consumption, particularly beneficial for resource-constrained applications. The “memory wall” problem, stemming from the separation of compute and memory in traditional von Neumann architectures, significantly limits AI computational efficiency. To address this, the article details “compute-in-memory” (CIM) paradigms, integrating computation directly into the memory array. Furthermore, the source suggests that combining CIM with stochastic hardware could unlock novel capabilities and further enhance energy efficiency for diverse AI workloads. The article emphasizes a need for co-designing algorithms and hardware to optimize energy, latency, and accuracy. This co-design approach is crucial for developing a “converged platform” suitable for both artificial and spiking neural networks. Specifically, exploiting stochasticity in both AI algorithms – such as spike timing-dependent plasticity – and hardware – such as spin–orbit transfer magnetic tunnel junctions – offers a path towards improved performance and efficiency. AI Algorithm and Hardware Co-Design AI hardware is evolving to overcome limitations of the traditional von Neumann architecture, which separates compute and memory units. This separation creates the “memory wall” problem, significantly contributing to energy consumption and latency. A promising solution is “compute-in-memory” (CIM), integrating computation within the memory array itself. CIM utilizes various memory technologies—including embedded non-volatile, static random-access, dynamic random-access, and flash memory—to develop more energy-efficient AI hardware. The source highlights the need for co-designing algorithms and hardware to optimize energy, latency, and accuracy. This collaborative approach is crucial for developing a “converged platform” capable of supporting both artificial neural networks and spiking neural networks for diverse AI applications. Leveraging the error-resiliency inherent in AI algorithms, along with stochastic hardware like spin–orbit transfer magnetic tunnel junctions (SOT-MTJs), can further enhance energy efficiency. Advancements in AI models, from multi-layer perceptrons to transformers, have driven increasing computational demands. While GPUs initially aided training through parallel processing, the source emphasizes that current large language and vision models require substantial resources, raising sustainability concerns. Ultimately, brain-inspired computation, like spiking neural networks, and optimized hardware are needed to achieve efficiency comparable to the human brain. Importance of Efficient AI Hardware Efficient AI hardware is increasingly crucial as AI expands into sectors like healthcare and transportation. The source highlights that such hardware enhances performance, reduces costs, and supports real-time decision-making in resource-constrained applications. Traditional hardware, based on the von Neumann architecture, suffers from a significant limitation known as the “memory wall,” where data movement between compute and memory units consumes substantial energy and increases latency. A key approach to overcoming this “memory wall” is through “compute-in-memory” (CIM) paradigms. CIM integrates computing directly within the memory array, utilizing technologies like embedded non-volatile memory, static random-access memory, dynamic random-access memory, and flash memory. This integration aims to alleviate energy consumption and latency issues inherent in traditional systems. The source emphasizes that efficient implementation of machine learning workloads requires optimization at the hardware level. Furthermore, the source suggests leveraging the error-resiliency of AI algorithms to develop hardware that can operate with reduced energy consumption while maintaining accuracy. This includes exploring stochastic hardware, like spin–orbit transfer magnetic tunnel junctions (SOT-MTJs), and brain-inspired approaches like spiking neural networks (SNNs), which perform sparse, event-driven computations similar to the human brain. Co-designing algorithms and hardware is proposed to create a “converged platform” for artificial and spiking neural networks. AI Applications in Healthcare and Transportation Efficient artificial intelligence (AI) hardware is crucial for applications in sectors like healthcare and transportation, enhancing performance, reducing costs, and enabling real-time decision-making. Traditional hardware, based on the von Neumann architecture, suffers from a “memory wall” – the separation of compute and memory – limiting computational efficiency. This creates bottlenecks during training and raises sustainability concerns with large models, as data movement consumes significant energy and increases latency. A key solution to overcome the memory wall is “compute-in-memory” (CIM). CIM integrates computing capabilities directly within the memory system, utilizing different memory technologies like embedded non-volatile memory, static random-access memory, and dynamic random-access memory. This approach tackles the energy consumption and latency issues of traditional architectures. Furthermore, exploiting the error-resiliency of AI algorithms can lead to even more energy-efficient hardware designs. The source highlights the need for co-designing hardware and algorithms to optimize energy, latency, and accuracy, ultimately leading to a “converged platform” for both artificial and spiking neural networks. This co-design is essential for diverse AI applications, and is inspired by brain-like computation, such as spiking neural networks which perform sparse and event-driven computations similar to the human brain.

Addressing Sustainability Concerns in AI Addressing sustainability concerns in AI requires overcoming limitations in traditional hardware architecture. The source highlights the “memory wall” problem—caused by separating compute and memory units in the von Neumann architecture—as a major contributor to energy consumption and latency. This inefficiency is particularly problematic with increasingly complex, multi-billion parameter models used in applications like language and vision processing, raising concerns about real-world sustainability. A promising solution is “compute-in-memory” (CIM) – integrating computation directly within the memory array itself. The source details CIM approaches using various memory technologies—embedded non-volatile memory, SRAM, DRAM, and flash memory—to tackle the memory wall and build more energy-efficient AI hardware. This contrasts with the dense, synchronous, high-precision computations and extensive data movement of current systems, aiming for brain-like computation instead. Furthermore, leveraging the error-resiliency of AI algorithms offers another path toward efficiency. The source suggests utilizing stochastic hardware—like spin–orbit transfer magnetic tunnel junctions—to enable faster operations with reduced energy consumption, all while maintaining acceptable system accuracy. This, combined with co-designing algorithms and hardware, is key to developing a “converged platform” suitable for both artificial and spiking neural networks. The Role of Error Resilience in AI Hardware Efficient AI hardware is critical because traditional systems, based on the von Neumann architecture, suffer from a “memory wall” problem. This separation of compute and memory units causes significant energy consumption and latency. To overcome this, researchers are exploring “compute-in-memory” (CIM) paradigms, integrating computation directly within the memory array itself. CIM utilizes various memory technologies including embedded non-volatile memory, SRAM, DRAM, and flash memory to tackle this bottleneck and improve energy efficiency. The source highlights that AI algorithms possess inherent “error resilience,” presenting an opportunity to develop approximate hardware. This approach allows for faster operations with reduced energy consumption without sacrificing overall system accuracy. Leveraging this resilience, alongside CIM techniques, is key to building more sustainable and efficient AI systems. The goal is to move beyond the limitations of dense, synchronous, high-precision computations found in current models. Furthermore, the article advocates for a co-design approach, simultaneously considering AI algorithms and hardware constraints. This mutual consideration is essential for creating brain-inspired AI systems. The evolution of AI models and hardware is shown to have a functional gap relative to the brain, and co-design is crucial for bridging this gap and optimizing energy, latency, and accuracy for diverse AI applications like spiking neural networks. Brain-Inspired Computation and SNNs The source highlights a need for brain-inspired computation to address limitations in current AI hardware. Traditional systems, based on the von Neumann architecture, suffer from the “memory wall” – a bottleneck caused by separating compute and memory units. To mimic the efficiency of the human brain, researchers are exploring spiking neural networks (SNNs), which perform sparse, event-driven computations. This approach aims to reduce energy consumption and improve performance in AI applications. Compute-in-memory (CIM) is presented as a promising solution to the memory wall problem. CIM integrates computing capabilities directly into the memory system, allowing operations to occur within the memory array itself. The source details that various memory technologies – including embedded non-volatile memory, SRAM, DRAM, and flash memory – are being investigated for CIM implementation. This technique is crucial for developing energy-efficient hardware for machine learning workloads. Furthermore, the source suggests leveraging the error-resiliency of AI algorithms to create more efficient hardware. By embracing approximate computing, potentially using stochastic hardware like spin–orbit transfer magnetic tunnel junctions (SOT-MTJs), systems can achieve faster operations with reduced energy consumption while maintaining accuracy. Co-designing algorithms and hardware is key to developing a “converged platform” suitable for both artificial and spiking neural networks. Key Developments in AI Hardware Design Advancements in artificial intelligence necessitate more efficient hardware solutions due to increasing computational demands, particularly with models like large language and vision models. Traditional hardware, based on the von Neumann architecture, suffers from a critical limitation known as the “memory wall.” This occurs because of the separation between compute and memory units, leading to significant energy consumption and latency as data is constantly shuttled between them. To overcome the memory wall, researchers are exploring compute-in-memory (CIM) techniques. CIM aims to integrate computing capabilities within the memory system itself, alleviating the need for extensive data movement. This approach utilizes various memory technologies including embedded non-volatile memory, static random-access memory, dynamic random-access memory, and flash memory to perform computations directly where the data resides, enhancing energy efficiency for AI workloads. Furthermore, the inherent error-resiliency of AI algorithms is being leveraged to develop stochastic hardware. By embracing approximate computing, these systems can achieve faster operations with reduced energy consumption while maintaining acceptable accuracy. A co-design approach, considering both algorithm and hardware constraints, is essential for creating a “converged platform” suitable for both artificial and spiking neural networks, and diverse AI applications. Source: https://www.frontiersin.org/journals/science/articles/10.3389/fsci.2025.1611658/full Tags: