# All-in-One Analog AI Hardware: On-Chip Training and Inference with Conductive-Metal-Oxide/HfOx ReRAM Devices
**Authors**: VictoriaClerico, WooseokChoi, TommasoStecconi, FolkertHorst, LauraBĂ©gon-Lours, MatteoGaletta, AntonioLa Porta, NikhilGarg, FabienAlibart, Bert JanOffrein, ValeriaBragaglia
[1] Donato Francesco Falcone
1] IBM Research - Europe, RĂŒschlikon, 8803, ZĂŒrich, Switzerland
2] Institut Interdisciplinaire dâInnovation Technologique (3IT), UniversitĂ© de Sherbrooke, Sherbrooke, QC J1K 0A5, Quebec, Canada
3] Institute of Electronics, Microelectronics and Nanotechnology (IEMN), UniversitĂ© de Lille, Villeneuve dâAscq, 59650, France
## Abstract
Analog in-memory computing is an emerging paradigm designed to efficiently accelerate deep neural network workloads. Recent advancements have focused on either inference or training acceleration. However, a unified analog in-memory technology platformâcapable of on-chip training, weight retention, and long-term inference accelerationâhas yet to be reported. This work presents an all-in-one analog AI accelerator, combining these capabilities to enable energy-efficient, continuously adaptable AI systems. The platform leverages an array of analog filamentary conductive-metal-oxide (CMO)/HfO x resistive switching memory cells (ReRAM) integrated into the back-end-of-line (BEOL). The array demonstrates reliable resistive switching with voltage amplitudes below 1.5 V, compatible with advanced technology nodes. The arrayâs multi-bit capability (over 32 stable states) and low programming noise (down to 10 nS) enable a nearly ideal weight transfer process, more than an order of magnitude better than other memristive technologies. Inference performance is validated through matrix-vector multiplication simulations on a 64Ă64 array, achieving a root-mean-square error improvement by a factor of 20 at 1 second and 3 at 10 years after programming, compared to state-of-the-art. Training accuracy closely matching the software equivalent is achieved across different datasets. The CMO/HfO x ReRAM technology lays the foundation for efficient analog systems accelerating both inference and training in deep neural networks.
keywords: In-memory computing, Analog ReRAM, Deep Neural Networks, Training, Inference
## 1 Introduction
Modern computing systems rely on von Neumann architectures, where instructions and data must be transferred between memory and the processing unit to perform computational tasks. This data transfer, particularly recurrent and massive in prominent artificial intelligence (AI)-related workloads, results in significant latency and energy overhead [1]. Digital AI accelerators address this challenge through computational parallelism, bringing memory closer to the processing units, and exploiting application-specific processors [2, 3]. This approach has demonstrated to bring significant improvements in throughput and efficiency for running deep neural networks (DNNs) [4], but the physical separation between memory and compute units persists. Analog in-memory computing (AIMC) [5] is a promising approach to eliminate this separation and so achieve further power and efficiency improvements in deep-learning workloads [6], by enabling some arithmetic and logic operations to be performed directly at the location where the data is stored. By mapping the weights of DNNs onto crossbar arrays of resistive devices and by leveraging Ohmâs and Kirchhoffâs physical laws, matrix-vector multiplications (MVMs)âthe most recurrent operation in AI-workloads [7] âare performed in memory with $O(1)$ time complexity [5, 8, 4]. Recent demonstrations of the AIMC paradigm have primarily focused on accelerating the inference step of digitally trained DNNs [9, 10, 11, 12]. However, the increasing computing demands of modern AI models make the training phase orders of magnitude more costly in time and expenses than inference, highlighting the need for efficient hardware acceleration based on the AIMC paradigm. For instance, Gemini 1.0 Ultra required over $5\cdot 10^{25}$ floating-point operations (FLOPs), approximately 100 days, $\mathrm{24\,MW}$ of power, and an estimated cost of 30 million dollars for training [13]. Analog training acceleration imposes even more stringent requirements on resistive devices. In addition to inference (i.e., the forward pass), the back-propagation of errors, gradient computation, and weight update steps must be performed during the learning phase. However, in the digital domain updating the weights of a matrix of size NxN requires $O(N^{2})$ digital operations, leading to a significant drop in efficiency and speed. Beyond the forward pass, the AIMC approach enables acceleration of (1) backward pass through MVMs transposing the inputs and outputs, (2) gradient computation, and (3) the weight update through gradual bidirectional conductance changes upon external stimuli, all with $O(1)$ time complexity. To achieve this, the ideal analog resistive device should exhibit bidirectional, linear, and symmetric conductance updates in response to an open-loop programming pulse scheme (i.e., without the need for verification following each pulse) [4, 14]. Promising technologies include redox-based resistive switching memory (ReRAM) [15, 16], electro-chemical random access memory (ECRAM) [17], and capacitive weight elements [18]. Addressing the various non-idealities of these technologies [19] requires the co-optimization of technology and designated training algorithms. Gokmen et al. [20] proposed an efficient, fully parallel approach that leverages the coincidence of stochastic voltage pulse trains to carry out outer-product calculations and weight updates entirely within memory, in $O(1)$ time complexity. To relax the device symmetry requirements, a novel training algorithm, known as Tiki-Taka, was designed based on this parallel scheme [21]. The primary advantage of the Tiki-Taka approach lies in reduced device symmetry constraints across the entire conductance (G) range, focusing instead on a localized symmetry point where increases and decreases in G are balanced [21]. More recently, the Tiki-Taka version 2 (TTv2) algorithm was demonstrated in hardware [22] on small-scale tasks using optimized analog ReRAM technology in a 6-Transistor-1ReRAM unit cell crossbar array configuration. However, TTv2 faces some convergence issues when the reference conductance is not programmed with high precision [23]. Analog gradient accumulation with dynamic reference (AGAD) learning algorithm (i.e., TTv4) was proposed to overcome the reference conductance limitation, providing enhanced and robust performance [23]. From a technology perspective, the addition of an engineered conductive-metal-oxide (CMO) layer in a conventional HfO x -based ReRAM metal/insulator/metal (M/I/M) stack has been shown to improve switching characteristics in terms of the number of analog states, stochasticity, symmetry point, and endurance, compared to conventional M/I/M technology [24, 25, 26]. However, while CMO/HfO x ReRAM technology has proven to meet all the fundamental device criteria for on-chip training [24], array-level assessment and BEOL integration remain unexplored. Furthermore, although accelerating DNN training using AIMC is more challenging than inference, a unified technology platform capable of performing on-chip training, retaining the weights, and enabling long-term inference acceleration has yet to be reported. This work fills this gap by demonstrating an all-in-one AI accelerator based on CMO/HfO x ReRAM technology, able to perform analog acceleration of both training and long-term inference operations. Such an integrated approach paves the way for highly autonomous, energy-efficient, and continuously adaptable AI systems, opening new paths for real-time learning and inference applications. The flowchart in Fig. 1 a illustrates the all-in-one analog training and inference challenge addressed in this study. To achieve this goal, CMO/HfO x ReRAM devices, integrated into the BEOL of a $\mathrm{130\,nm}$ complementary metal-oxide-semiconductor (CMOS) technology node with copper interconnects (see âMethodsâ section âDevice fabricationâ for details), are arranged in an array architecture using a 1T1R unit cell. Compared to implementations that use multiple transistors to control the resistive switching, the 1T1R unit cell maximizes memory density, which is crucial for storing large AI models on a single chip. Fig. 1 b shows an image of the all-in-one analog ReRAM-based AI core used in this work, with the corresponding 8x4 array architecture and the schematic of the BEOL integrated 1T1R cells. The CMO/HfO x ReRAM array is first studied in a quasi-static regime by statistically characterizing the devicesâ electro-forming step and quasi-static switching response. A physical 3D finite-element model (FEM) is developed to represent the geometry of the conductive filament and analytically describe the charge transport mechanism within these cells. Subsequently, the weight transfer accuracy and conductance relaxation are experimentally characterized on the 8x4 array. These measurements enable the demonstration of the coreâs inference capabilities, validated through representative MVM accuracy simulations on a 64Ă64 array. After demonstrating the MVM accuracy of the CMO/HfO x ReRAM core, analog switching experiments using an open-loop identical pulse scheme demonstrated the suitability of the same core for analog on-chip training acceleration. To assess the training performance, a realistic device model was used in the simulation, accounting for measured characteristics such as non-linear and asymmetric switching behavior, as well as inter- and intra-device variabilities. The training performance was validated using AGAD on fully connected and long short-term memory (LSTM) neural networks, demonstrating scalability from small to large-scale neural networks.
<details>
<summary>x1.png Details</summary>
### Visual Description
\n
## Diagram: AIMC Training and Inference Acceleration & All-in-one Analog ReRAM-based AI Core
### Overview
The image presents a diagram illustrating the architecture and process flow for AIMC (Analog In-Memory Computing) training and inference acceleration using an all-in-one analog ReRAM-based AI core. It is divided into two main sections: (a) depicting the training and inference process, and (b) detailing the core's structure.
### Components/Axes
Section (a) shows a flow diagram with labeled blocks representing stages of the process. Section (b) displays a schematic of the ReRAM array and a 3D representation of a single ReRAM unit cell. Key labels include: "Forward (F) pass (short term)", "Backward (B) pass", "Gradient accumulation & Parallel Weight Update", "Forward (F) pass (long term)", "BEOL-integrated Analog ReRAM array", "1T1R unit cell", "Analog ReRAM", "BEOL", "FEOL", "130-nm n-MOSFET", "WL1âŠWL4", "BL1âŠBL8", "SL1âŠSL4", "WL", "BL", "SL", "Tin", "CMO", "HfO2", "G", "B", "S", "I1", "I2", "V1", "V2", "V3".
### Detailed Analysis or Content Details
**Section (a): AIMC Training and Inference Acceleration**
This section illustrates a process flow.
1. **In-situ Training:** This block is divided into two sub-processes:
* **Forward (F) pass (short term):** Depicted with a schematic showing a ReRAM device with input voltages V1 and V2, and output currents I1 and I2. Arrows indicate the flow of current.
* **Backward (B) pass:** Also depicted with a ReRAM device schematic, similar to the forward pass.
2. **Gradient accumulation & Parallel Weight Update:** A rectangular block connecting the training stages.
3. **In-situ Inference:** This block is divided into:
* **Forward (F) pass (long term):** Depicted with a ReRAM device schematic, similar to the forward pass.
**Section (b): All-in-one Analog ReRAM-based AI Core**
This section shows the physical structure of the core.
1. **BEOL-integrated Analog ReRAM array:** A schematic representation of a ReRAM array with multiple word lines (WL1-WL4) and bit lines (BL1-BL8), connected to source lines (SL1-SL4). The array is represented as a grid of cells.
2. **1T1R unit cell:** A 3D representation of a single ReRAM unit cell. The cell consists of:
* **Analog ReRAM:** Layered structure with Tin, CMO, and HfO2 materials.
* **BEOL (Back End of Line):** Indicated as a layer above the ReRAM.
* **FEOL (Front End of Line):** Indicated as a layer below the ReRAM, containing a 130-nm n-MOSFET with gate (G), body (B), and source/drain (S) terminals.
* **M1-M8:** Representing metal layers.
### Key Observations
The diagram highlights the integration of ReRAM devices with standard CMOS technology. The ReRAM array is positioned on top of the MOSFETs, indicating a 3D integration scheme. The training process involves both forward and backward passes, suggesting an in-memory learning approach. The use of analog computation is emphasized by the "Analog ReRAM" label.
### Interpretation
The diagram demonstrates a novel architecture for AI acceleration that leverages the benefits of both analog computation and in-memory processing. By integrating ReRAM devices directly with CMOS circuitry, the design aims to reduce data movement and energy consumption, which are major bottlenecks in traditional AI systems. The in-situ training capability suggests that the system can adapt and learn directly within the memory array, eliminating the need for frequent data transfers to and from the processor. The layered structure of the ReRAM cell (Tin, CMO, HfO2) indicates the materials used to achieve the desired resistive switching characteristics. The diagram suggests a potential pathway for building more efficient and powerful AI hardware. The use of BEOL integration is a key feature, allowing for high-density ReRAM arrays without impacting the performance of the underlying CMOS transistors.
</details>
Figure 1: All-in-one AIMC challenge. a Schematic representation of the key steps required to perform on-chip training and inference with analog acceleration. Each step is executed using a crossbar array of resistive devices. b CMO/HfO x ReRAM AI core used in this work, consisting of an 8Ă4 array of 1T1R unit cells. From a fabrication perspective, each ReRAM cell is integrated into the BEOL of a $\mathrm{130\,nm}$ NMOS transistor with copper interconnects.
## 2 Results
### 2.1 Quasi-static array characterization and modelling
The quasi-static electrical characterization and analytical transport modelling of the 8x4 CMO/HfO x ReRAM array are presented here.
#### 2.1.1 Filament forming
Fig. 2 a shows the current-voltage characteristic of the ReRAM devices in the array, undergoing a soft-dielectric breakdown process, commonly referred to as forming [27]. During this step, a quasi-static voltage sweep up to $\mathrm{3.6\,V}$ is applied to the top electrode of each ReRAM device, while grounding the source and driving the gate of the corresponding NMOS selector with a constant $V_{\mathrm{G}}=\mathrm{1.2\,V}$ ensuring current compliance. This process leads to the formation of a highly defect-rich conductive filament in the HfO x layer. Due to the high oxygen vacancy ( $\rm V_{\rm O}^{\rm\cdot\cdot}$ in KrögerâVink notation [28]) formation energy, ranging from $\mathrm{2.8\,eV}$ to $\mathrm{4.6\,eV}$ in HfO x depending on the stoichiometry [29, 30], defect generation occurs with statistical relevance only during the forming sweep within the HfO x layer [26]. The subsequent application of a negative voltage sweep up to $-1.4\,\mathrm{V}$ , with a constant $V_{\mathrm{G}}=\mathrm{3.3\,V}$ , induces a radial redistribution of the defects within the CMO layer, consistent with findings in literature [26]. This process leads to an increase of the ReRAM conductance and is modelled by considering a constant average radius of the conductive filament, with a local electrical conductivity increase of the CMO layer on top of the filament. Refer to the âMethodsâ section âReRAM forming modellingâ for details. To determine the experimental ReRAM forming voltage, the voltage drop across the NMOS selector must be subtracted from the voltage applied to the 1T1R cell. Fig. 2 b shows the experimental transistor output characteristic, from which the resistance in the triode region at $V_{\mathrm{G}}=\mathrm{1.2\,V}$ is measured and used to extract the distribution of $V_{\mathrm{forming}}^{\mathrm{ReRAM}}$ within the CMO/HfO x ReRAM array (reported in Fig. 2 c). Refer to the âMethodsâ section âReRAM forming voltage extractionâ for details. The highly reproducible CMO/HfO x ReRAM forming step exhibits a 100% yield with a narrow distribution ( $\sigma=\mathrm{75\,mV}$ ) around $V_{\mathrm{forming}}^{\mathrm{ReRAM}}\approx\mathrm{3.2\,V}$ , making it suitable for integration with $\mathrm{130\,nm}$ NMOS transistors rated for $\mathrm{3.3\,V}$ operation.
#### 2.1.2 Resistive switching and polarity optimization
The underlying physical mechanism behind the resistive switching in analog CMO/HfO x ReRAM devices has been recently unveiled [26, 31, 32]. The current transport is explained by a trap-to-trap tunneling process, and the resistive switching by a modulation of the defect density within the conductive sub-band of the CMO that behaves as electric field and temperature confinement layer. In these works, the analog CMO/HfO x ReRAM device shows a counter-eightwise (C8W) switching polarity, according to the definition proposed in literature [33]. The intrinsically gradual reset (from low to high resistance) process, marked by a temperature decrease, occurs during the positive voltage sweep on the ReRAM top electrode, while the exponential set (from high to low resistance) process, involving a rapid temperature increase, occurs on the negative side [26]. However, when arranged in a 1T1R cell configuration based on an NMOS selector, the C8W switching polarity prevents direct control of the transistorâs $V_{\mathrm{GS}}$ during the exponential set process. This results in reduced switching uniformity, which is critical for the array-level adoption of analog CMO/HfO x ReRAM devices. For this reason, in this work the analog CMO/HfO x ReRAM devices within the 1T1R cells are optimized to exhibit the desirable 8W switching polarity by extending the current switching model in literature [26]. To achieve this, following the positive forming and the initial negative voltage sweep, each device in the array is subjected to a forward and backward voltage sweep from 0 to $-1.5\,\mathrm{V}$ . During this process, oxygen vacancies in the CMO layer radially spread outward, depleting the CMO defect sub-band within a half-spherical volume at the interface with the conductive filament, leading to a reset process (Fig. S3 in Supplementary Information shows the experimental arrayâs response). Conversely, a voltage sweep from 0 to $1.3\,\mathrm{V}$ enables the migration of oxygen vacancies in the CMO layer in the reverse direction, resulting in a set transition, controlled by the transistor gate. For each 1T1R cell within the 8x4 array, Fig. 2 d shows 5 quasi-static I-V cycling sweeps to experimentally assess the reproducibility of the optimized 8W switching polarity. The electronic transport in both the low-resistive state (LRS) and high-resistive state (HRS) is modelled as a trap-to-trap tunneling process, described by the Mott and Gurney analytical formulation. The physical parameters characterizing the transport in both LRS and HRS ( $N_{\rm e}$ , $\Delta E_{\rm e}$ , $a_{\rm e}$ , $\sigma_{\rm CMO}$ and $r_{\rm CF}$ ) are shown in Fig. 2 d. Refer to the âMethodsâ section âAnalytical ReRAM transport modellingâ for details on the LRS and HRS modelling. Fig. 2 e illustrates the cumulative probability distribution of the experimental LRS and HRS within the array, demonstrating device-to-device uniformity and a resistance ratio HRS/LRS of approximately 15, with absolute switching voltages $\leq\mathrm{1.5\,V}$ . The excellent uniformity of the forming and the optimized 8W-cycling characteristics set the groundwork for AIMC-based inference and training AI-accelerators using the CMO/HfO x ReRAM technology.
<details>
<summary>x2.png Details</summary>
### Visual Description
## Chart/Diagram Type: ReRAM Device Characteristics & Modeling
### Overview
This image presents a series of charts and diagrams detailing the formation, modeling, and characteristics of a ReRAM (Resistive Random Access Memory) device based on a CMO/HfOx/TiN stack. It includes current-voltage characteristics during array forming, transistor output characteristics, array forming distribution, quasi-static cycling behavior, and high/low resistance state (HRS/LRS) distributions.
### Components/Axes
* **a) CMO/HfOx ReRAM array forming and modeling:**
* X-axis: Voltage<sub>1T1R</sub> [V] (ranging from -1.4 to 3.6 V)
* Y-axis: Current<sub>1T1R</sub> [A] (logarithmic scale, from 10<sup>-9</sup> to 10<sup>-3</sup> A)
* Color scale: Devices (ranging from 1 to 32)
* Labels: Ï<sub>CMO</sub> = 37 S/cm, t<sub>CF</sub> = 11 nm (associated with device 1), Ï<sub>CMO</sub> = 5 S/cm, t<sub>CF</sub> = 11 nm (associated with device 2), V<sub>G</sub> = 1.2V
* **b) Transistor output characteristic:**
* X-axis: V<sub>DS</sub> [V] (ranging from 0 to 4 V)
* Y-axis: I<sub>DS</sub> [mA] (left) and V<sub>G</sub> [V] (right, ranging from 0 to 3 V)
* **c) Array forming distribution:**
* X-axis: V<sub>ReRAM forming</sub> [V] (ranging from 3.0 to 3.3 V)
* Y-axis: Normalized Probability (ranging from 0 to 1.0)
* Error bar: Â±Ï = 75 mV
* **d) Array quasi-static cycling and modeling:**
* X-axis: Voltage<sub>1T1R</sub> [V] (ranging from -1.5 to 1.3 V)
* Y-axis: Current<sub>1T1R</sub> [A] (logarithmic scale, from 10<sup>-7</sup> to 10<sup>-4</sup> A)
* Lines: Model (orange), 32 cycles (blue)
* Labels: N<sub>LRS</sub> = 5 x 10<sup>19</sup> cm<sup>-3</sup>, ÎE<sub>LRS</sub> = 65 meV, ÎŽ<sub>LRS</sub> = 2.1 nm, Ï<sub>CMO</sub> = 9 S/cm, t<sub>CF</sub> = 11 nm, N<sub>HRS</sub> = 1.2 x 10<sup>18</sup> cm<sup>-3</sup>, ÎE<sub>HRS</sub> = 80 meV, ÎŽ<sub>HRS</sub> = 3.6 nm, Ï<sub>CMO</sub> = 0.45 S/cm, t<sub>CF</sub> = 11 nm
* **e) Array HRS-LRS distributions:**
* X-axis: Resistance [Ω] (logarithmic scale, from 10<sup>4</sup> to 10<sup>6</sup> Ω)
* Y-axis: Cumulative Probability [%] (ranging from 0 to 100%)
* Lines: HRS (red), LRS (blue), Mean (green)
* Labels: ÎŒ = 18 kΩ, Ï = 0.25, ÎŒ = 256 kΩ, Ï = 0.25, read @ +0.2 V
### Detailed Analysis or Content Details
* **a) CMO/HfOx ReRAM array forming and modeling:** The plot shows multiple current-voltage curves for different devices during the forming process. Device 1 (Ï<sub>CMO</sub> = 37 S/cm, t<sub>CF</sub> = 11 nm) exhibits a sharp increase in current around 2.5V. Device 2 (Ï<sub>CMO</sub> = 5 S/cm, t<sub>CF</sub> = 11 nm) shows a similar, but less pronounced, increase in current around 2.8V. The color gradient indicates the variation in forming voltage across the array.
* **b) Transistor output characteristic:** This is a standard transistor characteristic curve showing I<sub>DS</sub> as a function of V<sub>DS</sub> for different gate voltages V<sub>G</sub>. The curve shows a typical saturation region for the transistor.
* **c) Array forming distribution:** The distribution of forming voltages is centered around approximately 3.17 V, with a standard deviation of 75 mV. The experimental data (black dots) is relatively tightly distributed.
* **d) Array quasi-static cycling and modeling:** The plot shows the current-voltage behavior of the ReRAM device during repeated switching cycles (32 cycles). The modeled behavior (orange line) closely matches the experimental data (blue line). The device exhibits hysteresis.
* **e) Array HRS-LRS distributions:** The cumulative probability distributions for the high resistance state (HRS) and low resistance state (LRS) are shown. The HRS distribution is centered around 18 kΩ with a standard deviation of 0.25. The LRS distribution is centered around 256 kΩ with a standard deviation of 0.25. The read voltage is specified as +0.2 V.
### Key Observations
* The forming voltage varies across the ReRAM array, as indicated by the color gradient in (a).
* The modeled behavior in (d) closely matches the experimental data, suggesting the model accurately captures the device's switching characteristics.
* There is a significant difference in resistance between the HRS and LRS, enabling clear distinction between the two states.
* The distributions in (e) are relatively narrow, indicating good uniformity in the device's resistance states.
### Interpretation
The data demonstrates the successful fabrication and modeling of a CMO/HfOx/TiN ReRAM device. The array forming process exhibits some variation in forming voltage, which could be attributed to variations in the CMO layer or the HfOx film. The quasi-static cycling data confirms the device's ability to switch between HRS and LRS repeatedly. The narrow resistance distributions suggest good device uniformity, which is crucial for reliable memory operation. The model accurately predicts the device's behavior, providing a valuable tool for optimizing device design and performance. The read voltage of +0.2V is likely chosen to maximize the margin between the HRS and LRS, ensuring accurate data reading. The parameters like Ï<sub>CMO</sub> and t<sub>CF</sub> are critical in determining the device's performance, and their values are provided for different devices to highlight the impact of material properties on the forming process.
</details>
Figure 2: ReRAM array quasi-static electrical characterization and modelling. a (1) Experimental positive forming sweeps (with $V_{\mathrm{G}}=\mathrm{1.2\,V}$ ) of the 8x4 CMO/HfO x ReRAM devices in the array. This process results in an average filament radius of $11\,\mathrm{nm}$ in the HfO x layer. (2) Negative voltage sweeps (with $V_{\mathrm{G}}=\mathrm{3.3\,V}$ ) to enable defect redistribution within the CMO layer, resulting in an increase in the conductance of the ReRAM cells. A representative sweep is shown in black. The insets illustrate a schematic representation of the defect arrangement within the stack. b Experimental NMOS transistor output characteristic, with $V_{\mathrm{G}}$ up to $\mathrm{3\,V}$ . c Experimental ReRAM forming voltage distribution measured from the CMO/HfO x ReRAM array. The experimental data used to extract the distribution are represented as green points. d Superposition of 5 I-V quasi-static 8W-cycles (in blue) for each of the 32 devices in the array, using $V_{\mathrm{set}}=\mathrm{1.3\,V}$ , $V_{\mathrm{G}}=\mathrm{1.1\,V}$ and $V_{\mathrm{reset}}=\mathrm{-1.5\,V}$ , $V_{\mathrm{G}}=\mathrm{3.3\,V}$ for set and reset processes, respectively. The analytical trap-to-trap tunneling model effectively captures the electron transport in both the LRS and HRS (yellow dashed lines). The physical parameters characterizing the transport, extracted from the model, and a schematic representation of the defect distribution, are presented for both resistive states. e Cumulative probability distributions for both LRS and HRS. For each array cell, the average resistance over 5 I-V cycles in LRS and HRS is defined at a read voltage of $\mathrm{0.2\,V}$ .
### 2.2 Analog inference with CMO/HfO x ReRAM core
Here, the experimental characterization of the key metrics of the CMO/HfO x ReRAM array relevant to inference performance is presented. Specifically, the continuous conductance tuning capability is demonstrated over a range spanning approximately one order of magnitude. The trade-off between weight transfer programming noise of CMO/HfO x ReRAM devices and number of required iterations for programming convergence is analyzed across different acceptance ranges. Furthermore, conductance relaxationâdefined as the change in conductance over time after programmingâis characterized. Finally, the combined impact of weight transfer, conductance relaxation, limited input/output quantization of the digital-to-analog converter (DAC) and analog-to-digital converter (ADC), and IR drop on the array wires is evaluated with respect to MVM accuracy.
#### 2.2.1 Weight transfer accuracy
In memristor-based AIMC inference accelerators, pre-trained normalized weights are initially mapped into target conductances and subsequently programmed into hardware in an iterative process known as weight transfer. This iterative process, which stops once the programmed conductance converges to the target value within a defined acceptance range, inherently introduces an error due to the analog nature of conductance weights. This error, described by a normal distribution with the standard deviation referred to as programming noise ( $\sigma_{\rm prog}$ ), leads to a drop in MVM accuracy. To quantify this non-ideality, the non-volatile multi-level capability of the CMO/HfO x ReRAM array is characterized. Fig. 3 a shows the experimental cumulative distribution of conductance values for 35 representative levels, with all states sharply separated and without any overlap. Fig. 3 b shows a schematic representation of the closed-loop (i.e., program-verify) scheme, where identical set and reset pulse trains are employed to program each ReRAM cell to its target conductance within a desired acceptance range (see âMethodsâ section âIdentical-pulse closed-loop schemeâ for details). Selecting programming conditions involves a fundamental trade-off: a narrower acceptance range can improve programming precision by reducing programming noise, but it increases the number of iterations required for convergence (see Fig. 3 d). Besides the longer programming time, other non-idealities to consider when choosing the acceptance range are (1) the conductance relaxation immediately after programming, which is characterized in 2.2.2 for CMO/HfO x ReRAM devices, and (2) read noise, which has already been characterized between 0.2% and 2% of G target for CMO/HfO x ReRAM devices [25] within a similar conductance range used in this work. The trade-off between the programming noise and the number of iterations is characterized for two representative acceptance range intervals: 0.2% and 2% of G target, respectively. Fig. 3 c illustrates the experimental number of pulses needed to converge to the G target using the two representative acceptance ranges. On average, each cell requires approximately 11 and 89 set / reset pulses for acceptance ranges of 2% and 0.2% of G target, respectively. Since the acceptance range is defined as a percentage of G target, the number of iterations required for convergence is almost independent of the target conductance value. In the Supplementary Information, Fig. S5 a shows the experimental cumulative distribution of conductance values for the same 35 representative levels presented in Fig. 3 a, but using 2% G target as acceptance range. The standard deviation of the representative conductance levels is extracted and fitted as a linear function of the target conductance (dashed lines), as shown in Fig. 3 e, for both acceptance ranges. For all conductance levels, a standard deviation of less than 0.1 ”S (1 ”S) is achieved considering 0.2% G target (2% G target) as the acceptance range. This is more than one order of magnitude lower compared to other memristive technologies, such as phase-change memory (PCM) arrays, targeting similar conductance ranges [34, 35, 36]. These results demonstrate that CMO/HfO x ReRAM cells achieve an almost ideal weight transfer during programming, enabling the distinction of more than 32 states (5 bits).
<details>
<summary>x3.png Details</summary>
### Visual Description
## Chart Compilation: CMOx-HfOx ReRAM Programming Characteristics
### Overview
This image presents a compilation of charts illustrating the programming characteristics of CMOx-HfOx ReRAM devices. It explores the cumulative distribution function during programming, a closed-loop scheme for voltage application, the relationship between iterations and target conductance, and the impact of programming noise on iterations and target conductance.
### Components/Axes
* **a) CMOx-HfOx ReRAM during programming:**
* X-axis: Target Conductance [”S] (Scale: 0 to 90, with markers at 10, 20, 30, 40, 50, 60, 70, 80, 90)
* Y-axis: Cumulative Distribution Function (Scale: 0.00 to 1.00, with markers at 0.00, 0.25, 0.50, 0.75, 1.00)
* Color Scale: 1 to 35 (representing some property, likely related to the density of successful programming events)
* Label: "Acceptance Range: 0.2% Gtarget"
* **b) Identical-pulse closed-loop scheme:**
* X-axis: time [a.u.] (arbitrary units)
* Y-axis: V [a.u.] (arbitrary units) and States [a.u.]
* Labels: Vset, Vread, Vreset, Gtarget, ±Acc. Range
* **c) Iterations vs Gtarget:**
* X-axis: Target Conductance [”S] (Scale: 10 to 90, with markers at 10, 30, 50, 70, 90)
* Y-axis: Closed-loop iterations (Logarithmic scale, from 10^1 to 10^3)
* Labels: "Acc. Range 0.2%", "Acc. Range 2%"
* Data Series: Iterations (green circles), Avg per G (orange line), -Avg per G (blue line)
* **d) Prog. noise vs iterations:**
* X-axis: Closed-loop iterations (Scale: Low to High)
* Y-axis: Ïprog (programming noise) (Scale: 0.2 to 2%)
* Labels: Ïprog â [0.1, 1]”s, Iterations = 10 (red X), Ïprog â [0.01, 0.1]”s, Iterations = 90 (blue X), "Trade-off"
* **e) Prog. noise vs Gtarget:**
* X-axis: Target Conductance [”S] (Scale: 10 to 90, with markers at 10, 30, 50, 70, 90)
* Y-axis: Ïprog, Range [”S] (Logarithmic scale, from 10^-2 to 10^0)
* Labels: "Acc. Range 0.2%", "Acc. Range 2%"
* Equations: Ïprog = 10^-3 * (11.3 * G + 11.2), Ïprog = 10^-3 * (1.1 * G + 0.8)
### Detailed Analysis or Content Details
* **a) CMOx-HfOx ReRAM during programming:** The chart shows a cumulative distribution function. The color intensity indicates the density of data points. The "Acceptance Range" of 0.2% Gtarget is highlighted. The data suggests that achieving a specific target conductance has a distribution of outcomes, and the color gradient shows how this distribution changes as the target conductance increases. The density of points is highest around 30-50 ”S.
* **b) Identical-pulse closed-loop scheme:** This diagram illustrates a feedback control scheme. Vset, Vread, and Vreset are voltage pulses applied to the ReRAM device. The device state (conductance) is monitored, and the voltage is adjusted to reach the target conductance (Gtarget) within the acceptable range (±Acc. Range).
* **c) Iterations vs Gtarget:** The green circles representing "Iterations" show a generally increasing trend, leveling off at higher target conductances. The orange line ("Avg per G") shows a decreasing trend, while the blue line ("-Avg per G") is relatively flat. The "Acc. Range 0.2%" and "Acc. Range 2%" regions are highlighted in blue and light blue, respectively. At a target conductance of 50 ”S, the number of iterations is approximately 300.
* **d) Prog. noise vs iterations:** This chart shows a trade-off between programming noise (Ïprog) and the number of iterations. Higher iterations (90) correspond to lower programming noise (around 0.2%), while lower iterations (10) correspond to higher programming noise (around 1.5%).
* **e) Prog. noise vs Gtarget:** The chart shows that programming noise decreases with increasing target conductance. Two equations are provided to model this relationship. At a target conductance of 10 ”S, the programming noise is approximately 0.011 ”S, and at 90 ”S, it is approximately 0.091 ”S.
### Key Observations
* The cumulative distribution function (a) indicates that achieving precise target conductance is challenging, with a spread of outcomes.
* The closed-loop scheme (b) is designed to mitigate this challenge by dynamically adjusting the voltage based on the device's state.
* There is a trade-off between the number of iterations and programming noise (d).
* Programming noise decreases with increasing target conductance (e).
* The number of iterations required to reach the target conductance increases with the target conductance, but plateaus at higher values (c).
### Interpretation
The data suggests that programming CMOx-HfOx ReRAM devices requires careful control of the voltage pulses and iterations to achieve the desired conductance with acceptable precision. The closed-loop scheme is a promising approach to address the inherent variability in the programming process. The trade-off between iterations and programming noise highlights the need for optimization. The decreasing programming noise with increasing target conductance may be related to the physics of the ReRAM device, such as the formation and rupture of conductive filaments. The equations provided in (e) offer a quantitative model for this relationship. The acceptance range of 0.2% Gtarget is a critical parameter for reliable device operation, and the charts demonstrate how different factors influence the ability to meet this requirement. The data presented provides valuable insights for designing and optimizing ReRAM-based memory devices.
</details>
Figure 3: Weight transfer characterization. a Cumulative distributions of 35 conductance states obtained using an identical-pulse closed-loop scheme with a 0.2% G target acceptance range. For each distribution, the entire CMO/HfO x ReRAM array was programmed to the corresponding G target, and the conductance values measured during the final closed-loop iteration (during programming) is reported. Each dot represents a 1T1R cell. b An example sequence of the identical-pulse closed-loop programming scheme utilized in this work. c Experimental number of closed-loop iterations as a function of G target for the two representative acceptance ranges. Each semitransparent point represents a 1T1R cell, the opaque points represent the average number of iterations per G target, and the horizontal dashed line indicates the overall average of the opaque points. d Graphical representation of the trade-off between programming noise and the number of iterations required for convergence, as a function of the acceptance range. e Experimental programming noise as a function of G target for the two representative acceptance ranges. Each point represents the standard deviation of the normal distribution measured across the entire array. The dashed lines in black indicate the corresponding linear fits.
#### 2.2.2 Conductance relaxation and matrix-vector multiplication accuracy
In addition to the excellent weight transfer accuracy during programming as presented in the previous section, the characterization of temporal conductance relaxation is critical to estimate the MVM accuracy over time. In analog ReRAM devices, a significant conductance relaxation has been observed immediately after programming (within 1 second) [9]. Following this initial abrupt conductance change, the relaxation process slows considerably [37, 9]. The physical cause of retention degradation is attributed to the Brownian motion of defects in the resistive switching layer [37]. In this section, the conductance relaxation of the CMO/HfO x ReRAM array after programming is characterized. Fig. 4 a shows the relaxation of the distributions previously reported in Fig. 3 a, approximately 10 minutes after programming. The 35 levels remain distinguishable 10 minutes after programming, with an average overlap of 9.6% between adjacent states gaussians, while the average standard deviation of the distributions increases to 0.6 ”S, showing almost independence from the G target (see Fig. 4 b). The stability of the CMO/HfO x ReRAM conductance states is further assessed on a longer time-scale, up to 1 hour. To achieve so, a linearly spaced G target vector within the experimental conductance range of 10 ”S to 90 ”S is defined, with a fine step of 0.2 ”S (400 points). Each G target value is programmed into a single ReRAM device within the array. Due to the size mismatch between the array (32 devices) and the G target vector (size 400), multiple measurement batches are needed. Fig. 4 c shows the experimental relaxation of the 400 programmed states within the entire conductance window, 1 second and 1 hour after programming, executed with the closed-loop scheme (see âMethodsâ section âIdentical-pulse closed-loop schemeâ for details) and with a 0.2% G target acceptance range. The exhibited conductance error induced by the relaxation process after 1 hour, computed as $G_{\mathrm{1h}}-G_{\mathrm{prog.}}$ , is plotted as a function of the programmed conductances in Fig. 4 d. After 1 hour, although both positive and negative relaxation errors are recorded, an average decrease in conductance is observed across all programmed states, with a relaxation error averaging around -0.7 ”S. This highlights that the relaxation process in CMO/HfO x ReRAM devices leads, on average, to a decrease in the mean and an increase in the standard deviation of the Gaussian distributions regardless of the initial conductance state. Since the absolute magnitudes of the mean decrease and the standard deviation increase are independent of G target, an extended characterization of the relaxation process up to 1 week is conducted for a representative conductance state (50 ”S). To achieve this, the arrayâs CMO/HfO x ReRAM devices are programmed using the identical-pulse closed-loop scheme to G target of 50 ”S, with a 0.2% G target acceptance range. Fig. 4 e illustrates the experimental array relaxation over 1 week. The insets display the evolution of both the mean and standard deviation as a function of the logarithm of time after programming (in seconds), using a linear fit to predict the conductance distribution over a 10-year period. To assess the accuracy of analog MVM, a comprehensive set of non-idealitiesâboth intrinsic to CMO/HfO x ReRAM devices and at the architecture levelâis considered, including finite programming resolution with 0.2% G target acceptance range, conductance relaxation, limited ADC and DAC quantization, and IR-drop across array wires. Fig. 4 f shows the hardware-aware simulation results of the analog MVM using CMO/HfO x ReRAM cells, projected for up to 10 years from programming, compared to the expected floating-point (FP) result. The results are generated using a single 64Ă64 normally distributed random weight matrix and 100 normally distributed input vectors within the range [-1, 1] (see âMethodsâ section âHW-aware simulation of analog MVMâ for details). Considering the input and output quantization of 6-bit and 8-bit respectively, the inset illustrates the time evolution of the root-mean-square error (RMSE) of the simulated analog MVM compared to the FP expected result. These results show that the CMO/HfO x ReRAM core enables accurate MVM operations, achieving an RMSE ranging from 0.03 at 1 second to 0.2 at 10 years after programming, compared to the ideal FP case. Fig. S6 in the Supplementary Information illustrates the impact of IR-drop and input/output quantization on the RMSE of an MVM performed on a 64Ă64 array. Over short time scales (within 1 hour), the primary accuracy bottleneck is the limited input/output quantization of 6-bit and 8-bit, respectively. Over longer periods, relaxation effects become the dominant source of non-ideality. In a larger 512Ă512 array, IR-drop emerges as the main accuracy bottleneck for analog MVM. Compared to the analog ReRAMs studied by Wan et al. [9], who report an experimentally determined RMSE of approximately 0.58 under conditions similar to those of this work, CMO/HfO x ReRAMs demonstrate a potential improvement in MVM accuracy by a factor of 20 and 3, 1 second and 10 years after programming, respectively. The excellent MVM accuracy results demonstrate the suitability of CMO/HfO x ReRAM devices for long-term AI inference applications, and lay the foundation for AI training acceleration, where short-term forward and backward MVMs are key steps.
<details>
<summary>x4.png Details</summary>
### Visual Description
\n
## Charts/Graphs: Array Relaxation and Hardware-Aware MVM Simulations
### Overview
The image presents six charts (a-f) detailing the relaxation behavior of an array after programming, and simulations of hardware-aware Matrix-Vector Multiplication (MVM). The charts explore conductance relaxation over various timescales (10 minutes, 1 hour, 50 microseconds, and up to 1 week), standard deviation of conductance, and the performance of MVM simulations.
### Components/Axes
* **a) Array relaxation after 10min:**
* X-axis: Programmed Conductance \[”S] (Scale: 0 to 90, increments of 10)
* Y-axis: Probability Density (Scale: 0 to 1.0, increments of 0.25)
* Title: "Array relaxation after 10min"
* Annotation: "Adjacent Gaussian Overlay (10min): 9.6%"
* **b) Std. dev. conductance relaxation:**
* X-axis: Programmed Conductance \[”S] (Scale: 0 to 90, increments of 10)
* Y-axis: Standard deviation \[”S] (Scale: 0 to 32, increments of 8)
* Title: "Std. dev. conductance relaxation"
* Legend:
* Green circles: "During Programming (Acc. Range 0.2%)"
* Black circles: "10 min After Programming"
* Annotation: "Array Relaxation After 10 min" with an arrow pointing to the data.
* **c) G-state relaxation after 1h:**
* X-axis: Time after programming \[s] (Logarithmic scale: 10â° to 10Âł, increments are not clearly marked)
* Y-axis: Programmed Conductance \[”S] (Scale: 0 to 90, increments of 20)
* Title: "G-state relaxation after 1h"
* **d) 1h-relaxation error:**
* X-axis: Programmed Conductance \[”S] (Scale: 0 to 90, increments of 10)
* Y-axis: G<sub>fin</sub> - G<sub>prog</sub> \[”S] (Scale: -3 to 3, increments of 1)
* Title: "1h-relaxation error"
* Annotation: "Avg. 1h-Relaxation Error = -0.68 ”S"
* **e) Extended array relaxation at 50”s:**
* X-axis: Programmed Conductance \[”S] (Scale: 0 to 60, increments of 10)
* Y-axis: Normalized Probability Density (Scale: 0 to 1.0, increments of 0.25)
* Title: "Extended array relaxation at 50”s"
* Legend:
* Prog.: Solid line
* 1s: Dashed line
* 1d: Dotted line
* 2d: Dashed-dotted line
* 1w: Long dashed-dotted line
* Inset Chart:
* X-axis: Log(Time[s])
* Y-axis: Mean \[”S] (Top) and Std Dev. \[”S] (Bottom)
* **f) HW-aware MVM simulations:**
* X-axis: Expected inner product output (Scale: -5 to 2.5, increments of 1)
* Y-axis: ReRAM inner product output (Scale: 0 to 5, increments of 1)
* Title: "HW-aware MVM simulations"
* Annotation: "64x64 Forward MVM, 6b input, 8b output"
* Legend:
* Prog.: Solid line
* 1s: Dashed line
* 1h: Dotted line
* 1d: Dashed-dotted line
* 10y: Long dashed-dotted line
* Inset Chart:
* X-axis: Log(Time[s])
* Y-axis: RMSE (Logarithmic scale: 10â»Âč to 10â°, increments of 0.2)
### Detailed Analysis or Content Details
* **a)** The probability density distribution shows a peak around 30-40 ”S, with a tail extending towards higher conductance values. The adjacent Gaussian overlay suggests a 9.6% overlap.
* **b)** The standard deviation of conductance is relatively stable during programming (green circles), with values ranging from approximately 2 to 8 ”S. After 10 minutes, the standard deviation increases slightly, with values ranging from approximately 4 to 12 ”S.
* **c)** The programmed conductance decreases over time after programming, with a rapid initial drop followed by a slower decay. At t=1s, conductance is approximately 80 ”S, decreasing to approximately 40 ”S at t=100s, and leveling off around 30 ”S at t=1000s.
* **d)** The 1h-relaxation error (G<sub>fin</sub> - G<sub>prog</sub>) is generally negative, indicating that the final conductance is lower than the programmed conductance. The average 1h-relaxation error is -0.68 ”S. The data points are scattered around the zero line, with a slight concentration of points below the line.
* **e)** The normalized probability density distribution shifts towards lower conductance values as time increases. At Prog., the peak is around 40 ”S. After 1s, the peak shifts to approximately 35 ”S. After 1 week (1w), the peak shifts further to approximately 30 ”S. The inset chart shows that the mean conductance decreases over time, while the standard deviation remains relatively constant.
* **f)** The ReRAM inner product output closely follows the expected inner product output, especially for shorter times (Prog., 1s, 1h). As time increases (1d, 10y), the ReRAM output deviates from the expected output. The RMSE increases with time, indicating a decrease in accuracy. The RMSE is approximately 0.01 at Prog., increasing to approximately 0.1 at 10y.
### Key Observations
* Conductance relaxation is a significant phenomenon, with conductance decreasing over time after programming.
* The standard deviation of conductance increases slightly after relaxation.
* The 1h-relaxation error is consistently negative, suggesting a systematic underestimation of the final conductance.
* MVM simulations show good accuracy for short times, but accuracy degrades over time due to conductance drift.
* The inset charts in (e) and (f) provide a more detailed view of the trends observed in the main charts.
### Interpretation
The data suggests that the ReRAM array exhibits conductance relaxation, which is a critical factor to consider for long-term reliability and accuracy. The relaxation process leads to a decrease in conductance over time, which can affect the performance of MVM operations. The simulations in (f) demonstrate that the accuracy of MVM operations degrades as the conductance drifts, highlighting the need for calibration or compensation techniques to mitigate the effects of relaxation. The adjacent Gaussian overlay in (a) suggests that the relaxation process is not uniform across the array, and that some devices may relax more quickly than others. The consistent negative relaxation error in (d) indicates a systematic bias in the relaxation process, which could be due to device-specific characteristics or programming conditions. The logarithmic scales used in (c) and (f) emphasize the importance of long-term stability and the potential for significant degradation over extended periods. Overall, the data provides valuable insights into the behavior of ReRAM arrays and the challenges associated with implementing reliable and accurate MVM operations.
</details>
Figure 4: Conductance relaxation and MVM accuracy. a Probability density distributions of 35 conductance states approximately 10 minutes after programming. The black areas between adjacent Gaussian distributions represent the overlap of their tails. On average, an overlap of 9.6% is observed after 10 minutes. b The standard deviations of the 35 conductance states during programming (in purple) and 10 minutes after it (light blue). c Relaxation of 400 conductance states, with one device per G-state, measured 1 second and 1 hour after programming. d Relaxation error 1 hour after programming. A negative and nearly G-independent average error (dashed line) indicates that relaxation in CMO/HfO x ReRAMs tends toward a slight conductance decrease and is state-independent. e Experimental array relaxation of a representative 50 ”S state, up to 1 week after programming with 0.2% G target acceptance range. Each probability density distribution is normalized to its maximum for graphical representation. The experimental data used to extract the distributions are represented as points aligned to the y=0 horizontal axis. Insets show the time dependence of the mean and standard deviation. Dashed blue lines represent the conditions during programming, once the convergence to G target is reached, while a linear fit (green dashed line) extrapolates the distribution 10 years after programming (dashed black line). f Analog MVM accuracy simulations using a 64x64 CMO/HfO x ReRAM array as a function of time after programming (indicated by different colors). The inset shows the expected RMSE compared to the ideal FP result. Experimental programming noise, conductance relaxation, limited input/output quantization and IR-drop are considered in this assessment.
### 2.3 Analog training with CMO/HfO x ReRAM core
To efficiently tackle deep learning workloads, the analog AI accelerator must not only perform forward and backward passes (MVMs), but most importantly, allow for weight updates [38]. During backpropagation, the synaptic weights are modified according to the gradient of the corresponding layer. Therefore, the device conductance must be gradually modified in both positive and negative directions to represent analog weight changes. Analog CMO/HfO x ReRAM arrays not only allow for bidirectional conductance updates, but additionally enable parallel weight updating by following a stochastic open-loop pulse scheme [20, 21]. Remarkably, the parallel and open-loop update scheme significantly accelerates training compared to serial and closed-loop methods, providing efficiency gains of several orders of magnitude and advantages in system design complexity [39]. In this section, the bidirectional open-loop response of the CMO/HfO x ReRAM array, required during Tiki-Taka training, is characterized. Specifically, the analog conductance potentiation, depression and symmetry point are measured. Subsequently, the devicesâ responses are statistically reproduced in the open-source âaihwkitâ simulation platform developed by IBM [38]. Finally, this hardware-aware device model, which includes device variabilities, is used to simulate the training of representative neural networks using the AGAD learning algorithm. This novel analog training algorithm relaxes the symmetry requirements of previous Tiki-Taka versions by incorporating additional digital computations on-the-fly [23].
#### 2.3.1 Open-loop ReRAM array characterization
Fig. 5 a shows the experimental conductance change of a representative CMO/HfO x ReRAM device within the array upon applying identical-voltage pulse trains with alternating polarity in batches of 400. Subsequently, a sequence of 500 pulses with alternating polarity, consisting of 1-pulse-up followed by 1-pulse-down, is applied to experimentally determine the symmetry point. The same open-loop programming scheme, with $V_{\rm set}=1.35\,\mathrm{V}$ ( $V_{\rm G}=1.4\,\mathrm{V}$ ) and $V_{\rm reset}=-1.3\,\mathrm{V}$ ( $V_{\rm G}=3.3\,\mathrm{V}$ ), each lasting 2.5 ”s, is applied to all devices in the 8x4 array. The set / reset pulse width is limited by the experimental setup, although previous work has demonstrated CMO/HfO x ReRAM switching with pulses as short as $60\,\mathrm{ns}$ [25]. Due to inter-device (device-to-device) and intra-device (cycle-to-cycle) variabilities, the experimental response of each device to a given number of identical pulses exhibits some level of variability (see Fig. S7 in the Supplementary Information). Therefore, for each pulse, a Gaussian distribution of the measured conductance states among the devices is extracted. For statistical relevance, Fig. 5 b shows the experimental standard deviation of the array response to the open-loop scheme as a function of the pulse number, represented in grey. To realistically assess the accuracy of analog training with CMO/HfO x ReRAM devices, the key figures of merit of the device training characterizationâsuch as the number of states, the symmetry point skew, and the noise-to-signal ratio (NSR)âare first extracted from experimental data, as defined below.
$$
\displaystyle\mathrm{N}_{\rm states}=\frac{G_{\rm max}-G_{\rm min}}{\overline{
\Delta G_{\rm sp}}} \tag{1}
$$
$$
\displaystyle\mathrm{SP}_{\rm skew}=\frac{G_{\rm max}-\overline{G_{\rm sp}}}{G
_{\rm max}-G_{\rm min}} \tag{2}
$$
$$
\displaystyle\mathrm{NSR}=\frac{\sigma_{\Delta G_{\rm sp}}}{\overline{\Delta G
_{\rm sp}}} \tag{3}
$$
$G_{\rm max}$ and $G_{\rm min}$ represent the maximum and minimum values extracted from the full conductance swings, while $\overline{G_{\rm sp}}$ , $\overline{\Delta G_{\rm sp}}$ and $\sigma_{\Delta G_{sp}}$ denote the values of the mean conductance, mean conductance update and standard deviation of the conductance update at the symmetry point during the 1-pulse-up, 1-pulse-down procedure, respectively. Fig. 5 c shows the experimental Gaussian distributions of these metrics for the 32 devices within the array. The results indicate an average of 22 states, with a range from 16 to 33. A shift in the $G_{\rm sp}$ (or SP skew) of 61% is measured, reflecting a negative trend in the device asymmetry where the down response is steeper than the up response. An average NSR of 90% among the devices is obtained, demonstrating the capability to discriminate between pulses up and down around the symmetry point. This parameter reflects the intrinsic noise on the deviceâs response under identical conditions, highlighting an intra-device variation [38]. Previous studies on similar CMO/HfO x ReRAM systems [24] extracted these metrics from isolated 1R devices using an optimized open-loop scheme tailored to each device. In contrast, this work demonstrates for the first time that a single open-loop identical pulse scheme enables reliable operation of the entire CMO/HfO x 1T1R array, ensuring consistent performance across the array.
<details>
<summary>x5.png Details</summary>
### Visual Description
\n
## Chart: Analog Switching Characteristics of a ReRAM Device & Experimental Array Metrics
### Overview
The image presents a series of charts illustrating the analog switching characteristics of a Resistive Random Access Memory (ReRAM) device, along with experimental array metrics for a "Tiki-Taka" training process. The charts explore conductance changes with pulse number, statistical response, and distributions of key parameters. The image is divided into three main sections: (a) a conductance vs. pulse number plot with inset voltage waveform, (b) an array-level open-loop statistical response plot with a probability distribution inset, and (c) three histograms showing distributions of number of states, symmetry, and noise.
### Components/Axes
**Section (a):**
* **Title:** Analog switching characteristics of a ReRAM device (open-loop)
* **X-axis:** Pulse Number (0 to 2100)
* **Y-axis:** Conductance (logarithmic scale, approximately 1 to 100 [”S])
* **Voltage Waveform (top):** Shows a series of pulses labeled 2.5”s, x400, x400, x500. The voltage is indicated as V<sub>set</sub>.
* **Annotations:** G<sub>max</sub>, G<sub>min</sub>, ÎG<sub>sp</sub>.
* **Sample Device:** Potentiation: 1.35V, 3.5”s; Depression: 1.3V, 2.5”s
**Section (b):**
* **Title:** Array-level open-loop statistical response
* **X-axis:** Pulse Number (0 to 2100)
* **Y-axis:** Conductance (logarithmic scale, approximately 1 to 100 [”S])
* **Inset:** Probability distribution of conductance after 1200 pulses (”S). Labeled with ±Ï.
* **Annotation:** Array exp. data
**Section (c):**
* **Title:** Experimental array metrics for Tiki-Taka training
* **Histogram 1:**
* **X-axis:** Number of States (0 to 31)
* **Y-axis:** Normalized Probability Density (0 to 1.0)
* **Annotations:** Mean = 22, n<sub>states</sub>, Exp.
* **Histogram 2:**
* **X-axis:** Symmetry (0 to 1.0)
* **Y-axis:** Normalized Probability Density (0 to 1.0)
* **Annotations:** Mean = 61%, SP<sub>skew</sub> = Ï<sub>max</sub>/ÎG<sub>0</sub>, Exp.
* **Histogram 3:**
* **X-axis:** Noise (0 to 1.0)
* **Y-axis:** Normalized Probability Density (0 to 1.0)
* **Annotations:** Mean = 90%, NSR = Ï<sub>noise</sub>/ÎG<sub>0</sub>, Exp.
**Additional Element:**
* **8x4 ReRAM array:** A grid labeled W<sub>(i,j)</sub> where i ranges from 1 to 8 and j ranges from 1 to 4.
### Detailed Analysis or Content Details
**Section (a):**
The conductance vs. pulse number plot shows a fluctuating conductance value. The line starts at approximately 20 ”S and fluctuates between approximately 20 ”S and 80 ”S for the first 800 pulses. After 800 pulses, the conductance generally decreases, reaching a minimum of approximately 10 ”S around pulse number 1600. The conductance then increases again, reaching approximately 40 ”S at pulse number 2100. G<sub>max</sub> is approximately 80 ”S and G<sub>min</sub> is approximately 10 ”S. ÎG<sub>sp</sub> is approximately 60 ”S.
**Section (b):**
The array-level response shows a similar trend to (a), but with a wider spread of data points. The data points are clustered around a central line that decreases from approximately 60 ”S to 20 ”S between pulse numbers 0 and 1600, then increases slightly to approximately 30 ”S at pulse number 2100. The inset shows a probability distribution centered around approximately 70 ”S with a standard deviation of approximately 10 ”S.
**Section (c):**
* **Histogram 1 (Number of States):** The distribution is centered around 22 states, with a peak probability density of approximately 0.25.
* **Histogram 2 (Symmetry):** The distribution is centered around 61% symmetry, with a peak probability density of approximately 0.3.
* **Histogram 3 (Noise):** The distribution is centered around 90% noise, with a peak probability density of approximately 0.4.
### Key Observations
* The conductance of the ReRAM device exhibits significant fluctuations with each pulse.
* The array-level response shows a wider distribution of conductance values compared to the single device response.
* The Tiki-Taka training process results in an average of 22 states, 61% symmetry, and 90% noise.
* The inset in (b) shows a relatively narrow distribution of conductance values after 1200 pulses, indicating some degree of convergence.
### Interpretation
The data suggests that the ReRAM device exhibits analog switching behavior, with conductance values that can be modulated by applying a series of pulses. The fluctuations in conductance may be due to stochastic effects or variations in the device characteristics. The array-level response shows that these fluctuations are amplified when multiple devices are connected in an array. The Tiki-Taka training process appears to be effective in achieving a reasonable number of states, but it also introduces a significant amount of noise. The symmetry metric indicates the balance between potentiation and depression during the training process. The high noise level may limit the precision of the analog storage. The inset in (b) suggests that the conductance distribution converges after a certain number of pulses, indicating that the device is approaching a stable state. The ReRAM array is labeled with W<sub>(i,j)</sub>, indicating a matrix of resistive elements, likely used for parallel processing or memory storage. The "Tiki-Taka" training likely refers to a specific algorithm or method used to tune the conductance levels of the ReRAM array for a particular application, potentially related to neural network training.
</details>
Figure 5: Open-loop array characterization for on-chip training. a Bidirectional accumulative response and symmetry point of a representative device in the array. The top inset shows the open-loop identical pulse scheme used for the synaptic potentiation (red) and depression (blue). A conceptual illustration of the 8x4 CMO/HfO x ReRAM array is depicted on the left. b Array statistical open-loop response to identical pulses. The grey area represents the standard deviation of the experimental Gaussian distributions, each corresponding to a specific pulse number. The inset shows a representative example of the experimental G-distribution at pulse number 1200. The raw data can be found in Figure S9 of the Supporting Information. c The experimental probability densities of N states, SP skew and NSR, respectively. The experimental data used to extract the distributions are represented as points aligned along the y=0 horizontal axis.
#### 2.3.2 Tiki-Taka training simulations
To perform realistic hardware-aware training simulations, the experimental device response is reproduced on software using the generalized soft bounds model implemented in the âaihwkitâ [40], which better captures the bidirectional resistive switching behavior (see Fig. S8 in Supplementary Information) and accounts for intra- and inter-device variabilities (see cycle-to-cycle and device-to-device variations in Fig. 6 a). Additionally, Gaussian distributions are modelled based on parameters extracted from device characterization ( $G_{\rm max}$ , $G_{\rm min}$ , $\Delta G_{\rm sp}$ , NSR, SP skew) to account for device-to-device variability observed in the experimental characterization (see âMethodsâ section âIntra and inter-device variabilityâ for details). This Gaussian fitting approach allows defining various device presetsâcharacterized by the same model but with different parameter settingsâto represent the synapses across the neural network. A realistic simulation setup is obtained by exclusively considering experimentally obtained parameters to reproduce the device trace (see âMethodsâ section âGeneralized soft bounds modelâ for details). The device model is defined based on the observed conductance window and number of states, without assuming asymptotic behavior for an infinite number of pulses. This prevents overestimation of both the conductance window and the number of states (material states), enhancing the fidelity of the simulation. To validate analog training with CMO/HfO x ReRAM technology, a 3-layer fully connected (FC) neural network was trained on the MNIST dataset for image classification. In addition, the impact of the deviceâs number of states, asymmetry, and noise-to-signal ratio on accuracy and convergence time is evaluated by simulating identical networks in which each property is individually enhanced, while keeping the others fixed at the experimentally derived values. Literature has shown that these device characteristics critically influence the convergence of analog training algorithms [23]. Therefore, this method assesses the deviation of the current CMO/HfO x ReRAM device properties from the ideal analog resistive device scenario. Moreover, to show the scalability of the CMO/HfO x ReRAM technology to more computationally-intensive tasks, such as time series processing, a 2-layer long short-term memory (LSTM) network was trained on War and Peace text sequences to predict the next token. Each network is initially trained using conventional stochastic gradient descent (SGD) based backpropagation with 32-bit FP precision, serving as the baseline performance. Fig. 6 b illustrates the accuracy per epoch for the FP-baseline trained with SGD (in green) and the analog network trained using AGAD, evaluated under four different parameter settings: (1) properties extracted from the experimental array (in yellow), (2) reduced NSR to 20% (in red), (3) average of N states = 100 states (in blue), and (4) zero average device asymmetry (in orange). Using symmetrical device presets, i.e. with an average SP skew of 50%, improves accuracy by 0.7% with respect to analog training with CMO/HfO x ReRAM experimentally derived configuration (96.9%), landing an accuracy of 97.6%, a 0.7% lower than the FP-SGD baseline (98.3%). The other two configurations show less performance improvement, indicating more resilience of the AGAD-training to deviceâs N states and NSR. Additionally, a 2-layer LSTM network with 64 memory states each (see Fig. 6 c), is trained with the experimentally obtained configuration. The performance is measured using the exponential of the cross-entropy loss, i.e. the test perplexity metric, which quantifies the certainty of the token prediction. Results in Fig. 6 d demonstrate the capabilities of the CMO/HfO x ReRAM technology on more complex network architectures, such as LSTMs, and computationally demanding tasks, exhibiting performance comparable to the FP-equivalent, with an approximate 0.7% difference in test perplexity.
<details>
<summary>x6.png Details</summary>
### Visual Description
## Chart Compilation: Model Training and Performance Analysis
### Overview
The image presents a compilation of four charts (a, b, c, d) detailing the training and performance of various models, including a generalized soft bounds device model, a 3FC MNIST training model, and an LSTM network. The charts showcase weight distributions, test accuracy, network architecture, and test perplexity.
### Components/Axes
**Chart a: Generalized soft bounds device model**
* **Title:** Generalized soft bounds device model
* **X-axis:** Pulse Number (0 to 2100, approximately)
* **Y-axis:** Weight (-1 to 2, approximately)
* **Data Series:**
* Model (CâC and DâD): Dark teal scatter plot.
* Devâ: Red diamonds.
* Devâ: Green triangles.
* Devâ: Blue squares.
* Devâ: Purple crosses.
**Chart b: 3FC MNIST training**
* **Title:** 3FC MNIST training
* **X-axis:** Epochs [a.u.] (0 to 80, approximately)
* **Y-axis:** Test Accuracy [%] (90 to 100, approximately)
* **Legend (top-left):**
* AGAD: Orange line with diamond markers.
* CMO/HfOâ exp. array: Red line with circle markers.
* NSR down to 20%: Blue line with square markers.
* Nstates up to 100: Purple line with triangle markers.
* Symmetry (SP skew 50%): Green line with cross markers.
* FP-baseline: Yellow line with plus markers.
**Chart c: LSTM network trained using CMO/HfOâ statistical array data**
* **Title:** LSTM network trained using CMO/HfOâ statistical array data
* **Components:** Input Tokens, LSTM 1, LSTM 2, FC (Fully Connected), Output.
* **Input Tokens:** "The", "man", "walks", "down", "the", "street".
* **LSTM Layers:** Two LSTM layers, each with 64 hidden units. Input is 87xN, output is 64xN.
* **FC Layer:** Fully connected layer with 87x87 dimensions.
* **Output:** "street".
**Chart d: LSTM training**
* **Title:** LSTM training
* **X-axis:** Epochs [a.u.] (0 to 100, approximately)
* **Y-axis:** Test Perplexity (1 to 5, approximately)
* **Legend (top-left):**
* AGAD: Orange line with diamond markers.
* CMO/HfOâ exp. array: Red line with circle markers.
* FP-baseline: Green line with plus markers.
### Detailed Analysis or Content Details
**Chart a:** The teal scatter plot representing the model weights fluctuates around zero, with a generally decreasing trend in amplitude as the pulse number increases. The Dev series (red, green, blue, purple) show relatively stable values, with some fluctuations. Devâ is consistently around 98.5, Devâ around 97.5, Devâ around 96.5, and Devâ around 95.5.
**Chart b:** The AGAD line (orange) shows a slight upward trend, starting around 92% and reaching approximately 98.5% accuracy. The CMO/HfOâ exp. array (red) line starts at approximately 96% and plateaus around 98%. The NSR down to 20% (blue) line shows a similar trend, starting around 92% and reaching approximately 97.5%. The Nstates up to 100 (purple) line starts around 92% and reaches approximately 97%. The Symmetry (SP skew 50%) (green) line starts around 92% and reaches approximately 97.5%. The FP-baseline (yellow) line starts around 92% and reaches approximately 96.5%.
**Chart c:** The diagram illustrates a two-layer LSTM network. Input tokens are converted to one-hot vectors. Each LSTM layer has 64 hidden units. The output layer is a fully connected layer that predicts the next token ("street").
**Chart d:** The AGAD line (orange) shows a decreasing trend in test perplexity, starting around 4.5 and reaching approximately 2.5. The CMO/HfOâ exp. array (red) line shows a similar decreasing trend, starting around 4.5 and reaching approximately 2. The FP-baseline (green) line shows a decreasing trend, starting around 4.5 and reaching approximately 3.
### Key Observations
* All models in Chart b demonstrate increasing test accuracy with increasing epochs, but the rate of improvement varies.
* The LSTM network in Chart d shows a clear reduction in test perplexity with training, indicating improved language modeling performance.
* The weight distribution in Chart a appears to stabilize over time, suggesting the model is converging.
* The LSTM network architecture in Chart c is relatively simple, consisting of two LSTM layers and a fully connected output layer.
### Interpretation
The data suggests that the models are successfully learning from the training data. The increasing test accuracy in Chart b and decreasing test perplexity in Chart d indicate that the models are improving their ability to generalize to unseen data. The weight distribution in Chart a suggests that the model is converging to a stable state. The LSTM network architecture in Chart c is a standard configuration for language modeling tasks.
The different data series in Chart b (AGAD, CMO/HfOâ, etc.) represent different training configurations or model architectures. Comparing their performance can provide insights into the effectiveness of different techniques. The fact that all models achieve high accuracy suggests that the MNIST dataset is relatively easy to learn.
The LSTM network in Chart d demonstrates the effectiveness of recurrent neural networks for sequence modeling tasks. The decreasing test perplexity indicates that the model is learning to predict the next token in a sequence with increasing accuracy. The choice of input tokens ("The man walks down the street") suggests that the model is being trained to understand simple sentences.
The overall compilation of charts provides a comprehensive overview of the training and performance of various models, highlighting the strengths and weaknesses of different approaches. The data suggests that the models are performing well, but further analysis may be needed to identify areas for improvement.
</details>
Figure 6: Device model and on-chip training simulations. a Device presets generated using the generalized soft bounds model with experimentally extracted parameters of CMO/HfO x devices, including inter- and intra-device variabilities. b Training simulations of a 3-layer fully-connected neural network on MNIST (235K parameters), using 32-bit FP precision trained on SGD (in green). Analog training simulations were performed using AGAD considering the empirical distribution of the parameters (in yellow), enhanced NSR (in red), increased N states (in blue), and symmetrical device configurations (in orange). c LSTM network architecture for text forecasting on the War and Peace dataset (79K parameters). The architecture considers a sequence length of 100 tokens and accounts for 2 layers with 64 hidden units. d Training results of the FP baseline (in green) and the analog training with AGAD on the experimental device configuration (in yellow). The training setup can be found in the Supporting Information.
## 3 Discussion
An all-in-one technology platform based on analog filamentary CMO/HfO x ReRAM devices is presented. This platform addresses critical challenges in modern digital AI accelerators by overcoming the physical separation between memory and compute units. It enables the execution of forward and backward MVMs, along with weight updates and gradient computations, directly on a unified analog in-memory platform with $O(1)$ time complexity. This all-in-one approach fundamentally differs from DNN inference-only [9] and training-only [24, 41] analog accelerators. In inference-only accelerators, DNN weights are trained in software (i.e., off-chip) using traditional digital CPUs or GPUs and then programmed once onto the analog AI hardware accelerator. In training-only accelerators, the long-term retention capabilities and overall MVM accuracy for large array tiles are not assessed. In this work, a novel all-in-one analog computing platform, capable of both on-chip training and inference acceleration, is unveiled. The CMO/HfO x ReRAM devices are integrated in the BEOL of a NMOS transistor platform in a scalable 1T1R array architecture. The highly reproducible forming step demonstrates compatibility with NMOS rated for $\mathrm{3.3\,V}$ operation, while the uniform quasi-static 8W-cycling characteristics, achieved with voltage amplitudes of less than $\pm$ $\mathrm{1.5\,V}$ , exhibit a significant conductance window and a low off-state. The multi-bit capability of more than 32 states (5 bits), distinguishable after 10 minutes with less than 10% overlap error, is experimentally demonstrated using an identical-pulse closed-loop scheme. The characterization of the weight transfer reveals record-low programming noise ranging from $\mathrm{10\,nS}$ to $\mathrm{100\,nS}$ , more than one order of magnitude lower than that of other memristive technologies targeting similar conductance ranges [34, 35, 36]. Each conductance distribution exhibits a state-independent relaxation process over time, characterized by a slight shift of the mean toward lower conductance and an increase in the standard deviation. This independence of the relaxation process from the target conductance is advantageous for implementing effective compensation schemes in the future. Realistic MVM simulations on a 64x64 array tile, considering CMO/HfO x ReRAM device non-idealities such as finite weight transfer resolution, conductance relaxation, limited input/output quantization, and IR-drop across array wires, show an RMSE as low as 0.2 compared to the ideal FP-case, even 10 years after programming. This demonstrates that the CMO/HfO x ReRAM devices improve analog MVM accuracy by a factor of 20 and 3 compared to the state of the art [9], 1 second and 10 years after programming, respectively. Although this study was performed at room temperature, previous characterization of a similar CMO/HfO x ReRAM stack demonstrated the thermal stability of the analog states at high temperature (less than 4% drift after 72 hours at 85 °C) [24]. Future studies will focus on incorporating the experimental read noise of CMO/HfO x ReRAM devices, characterized between 0.2% and 2% of G target within a similar conductance range as used in this work [25], into MVM accuracy simulations. Although read noise is not included in the MVM simulations of this study, no significant additional drop in MVM accuracy is anticipated. In fact, the magnitude of read noise is much smaller than that of the relaxation process and of the effect of reduced input/output quantization, which dominate the RMSE on different timescales. Furthermore, simulation results demonstrate the suitability of CMO/HfO x ReRAM technology for large 512x512 array, with the IR-drop expected to become the primary accuracy bottleneck in this case. Finally, the electrical response of the CMO/HfO x ReRAM array to an open-loop scheme with identical pulses demonstrates the viability of this technology for on-chip training applications. A realistic device model, accounting for both inter- and intra-device variability, is derived from experimental data. Table 1 benchmarks the representative device model used in this work on the MNIST dataset against other approaches, highlighting its high fidelity in reproducing experimental device responses.
Table 1: Device model benchmarking: from simplified approaches to realistic non-ideality modeling
| Ti/HfO x [41] | Not-included | exp. states Measured number of analog states during open-loop device characterization. | BEOL array | TTv2 | Medium | 90.5 % |
| --- | --- | --- | --- | --- | --- | --- |
| Ta/TaO x [41] | Not-included | exp. states Measured number of analog states during open-loop device characterization. | BEOL array | TTv2 | Medium | 96.4 % |
| TaO x /HfO x [24] | included | material states The asymptotic number of states under an infinite number of pulses. | Single ReRAMs | TTv2 | Medium | 97.4 % |
| CMO x /HfO x This work. | included | exp. states Measured number of analog states during open-loop device characterization. | BEOL array | AGAD | High | 96.9 % |
The impact of the deviceâs number of states, asymmetry and noise-to-signal ratio on training accuracy using the AGAD algorithm on MNIST is evaluated. This analysis demonstrates that, with the current deviceâs experimental properties, AGAD analog training achieves 96.9% accuracy, comparable to the ideal FP-baseline of 98.3%. To further improve analog training performance and bring results closer to the software equivalent, the key metric to enhance in the device is the symmetry. Finally, the on-chip analog training capabilities of the CMO/HfO x ReRAM technology are demonstrated on a more complex 2-layer LSTM network, showing comparable performance to its floating-point equivalent. In conclusion, the novel CMO/HfO x ReRAM all-in-one technology platform presented in this work lays the foundation for efficient and versatile analog chips capable of combining both training and inference capabilities, enabling autonomous, energy-efficient, and adaptable AI systems.
## 4 Methods
### 4.1 Device fabrication
The CMO/HfO x ReRAM array is based on 1T1R unit cells. In this configuration, the bottom electrode of the ReRAM device is connected in series to the drain of an n-type metalâoxideâsemiconductor (NMOS) selector transistor. The transistor blocks sneak paths and ensures current compliance during electro-forming and programming of the ReRAM device. The NMOS transistors, rated for $\mathrm{3.3\,V}$ operation, are fabricated using a standard $\mathrm{130\,nm}$ foundry process with copper BEOL interconnects. The ReRAM devices are integrated on metal-8 layer. To prevent the oxidation of the copper vias during the ReRAM stack deposition, the $\mathrm{70\,nm}$ thick silicon nitride (SiN x) passivation layer from the foundry is used as a protective layer. On top of that, a $\mathrm{20\,nm}$ thick titanium nitride (TiN) bottom electrode and a $\mathrm{4\,nm}$ thick hafnium oxide (HfO x) layers are deposited by Plasma-Enhanced Atomic Layer Deposition (PEALD) process at 300 °C, while maintaining vacuum conditions to avoid oxidation of the TiN layer. Subsequently, a stack of layers consisting of a $\mathrm{20\,nm}$ thick conductive metal-oxide (CMO), a $\mathrm{20\,nm}$ thick titanium nitride (TiN), and a $\mathrm{50\,nm}$ thick tungsten (W) is deposited by sputtering and patterned through a lithography step. A $\mathrm{100\,nm}$ thick silicon oxide (SiO x) layer is sputtered as passivation. The passivation layer is then patterned to expose the W top electrode and the copper via in the metal-8 layer beneath the bottom electrode. The ReRAM fabrication is completed using a titanium/gold lift-off process. In this approach, the TiN bottom electrode is connected to the metal-8 via through its vertical sidewalls using gold. The ReRAM BEOL patterning steps are performed through mask-based photolithography performed on a 6 $\times$ 6 mm 2 die issued from a Multi Project Wafer (MPW). The area of the CMO/HfO x ReRAM devices presented in this work is 12 $\times$ 12 ”m 2. Previous studies on CMO/HfO x ReRAM devices have demonstrated scalability down to 200 $\times$ 200 nm 2 [24, 26, 25]. Due to their filament-type nature, the performance of the ReRAM devices presented in this work is expected to remain similar for smaller areas.
### 4.2 ReRAM forming modelling
A 3D FEM of the CMO/HfO x ReRAM device, after the forming event, is used to simulate electronic transport by solving the continuity (4) and the Joule-heating (5) equations in steady state:
$$
\displaystyle\nabla\cdot J_{\rm e}=\nabla\cdot(\sigma(-\nabla V)=0 \tag{4}
$$
$$
\displaystyle\nabla\cdot(-k\nabla T)=J_{\rm e}\cdot E=Q_{\rm e} \tag{5}
$$
where $J_{\rm e}$ is the electric current density, $\sigma$ the electrical conductivity, $V$ the electric potential, $k$ the thermal conductivity and $Q_{\rm e}$ the heat source due to Joule heating. From the fit of the experimental array forming data in the low-voltage linear regime (from 0 to $0.2\,\mathrm{V}$ ), an average filament radius of $11\,\mathrm{nm}$ is extracted. The electrical and thermal conductivities of the materials in the ReRAM stack are taken from literature [26], by considering $\sigma_{\mathrm{CMO}}=5\,\mathrm{S/cm}$ and $k_{\mathrm{CMO}}=4\,\mathrm{W/mK}$ for the CMO layer used in this work. During the subsequent negative voltage sweep, the electrical conductivity of the CMO layer was used as a fitting parameter to model the radial redistribution of defects within the layer. Using experimental array data in the low-voltage linear regime (from 0 to $\mathrm{-0.2\,V}$ ), the resulting CMO electrical conductivity is $37\,\mathrm{S/cm}$ . Fig. S1 in Supplementary Information shows the results of the simulations.
### 4.3 ReRAM forming voltage extraction
The forming voltage of each 1T1R cell ( $V_{\mathrm{forming}}^{\mathrm{1T1R}}$ ) is defined as the voltage required to trigger the highest current increase ( $\max\left(\frac{dI}{dV}\right)$ ) during the quasi-static voltage sweep from 0 to $3.6\,\mathrm{V}$ (see Supplementary Information Fig. S2 a). The corresponding current is defined as the forming current ( $I_{\mathrm{forming}}^{\mathrm{1T1R}}$ ) (see Supplementary Information Fig. S2 b). Being the transistor driven by a constant $V_{\mathrm{G}}=1.2\,\mathrm{V}$ , it acts as a series resistor in the triode region before the forming event, when the ReRAM stack is highly insulating. After the forming event, when a conductive filament is created in the ReRAM device, the transistor ensures current compliance in the saturation region. The resistance of the transistor in the triode region at $V_{\mathrm{G}}=1.2\,\mathrm{V}$ is measured to be $R_{\mathrm{DS}}\approx 0.8\,\mathrm{k\Omega}$ (see Supplementary Information Fig. S2 c). Therefore, for each 1T1R cell, the actual ReRAM forming voltage is computed as $V_{\mathrm{forming}}^{\mathrm{ReRAM}}=V_{\mathrm{forming}}^{\mathrm{1T1R}}-R_{ \mathrm{DS}}^{\mathrm{triode}}\cdot I_{\mathrm{forming}}^{\mathrm{1T1R}}$ and reported in Fig. 2 c.
### 4.4 Analytical ReRAM transport modelling
In the 1T1R cell, the electronic current $I_{\rm e}$ is modelled as a trap-to-trap tunneling process within the CMO layer, as described in equation (6), following the model proposed by Mott and Gurney [42]. This model accounts for electron-hopping conduction across an energy barrier $\Delta E_{\rm e}$ , which remains uniform in all directions when there is no electric field applied. However, when an electric field is introduced, it modifies the energy barrier by $\mp$ $ea_{\rm e}E$ /2 for forward (backward) jumps, leading to a reduction (increase) in the barrier height.
$$
\displaystyle I_{\rm e}^{\rm Mott-Gurney}=2Aea_{\rm e}\nu_{\rm 0,e}N_{\rm e}
\exp{(\frac{-\Delta E_{\rm e}}{k_{\rm B}T})}\sinh{(\frac{a_{\rm e}eE}{2k_{\rm B
}T})} \tag{6}
$$
In equation (6), $e$ is the elementary charge, $k_{\rm B}$ is the Boltzmannâs constant, $a_{\rm e}$ is the hopping distance, $\nu_{\rm 0,e}$ is the electron attempt frequency, $N_{\rm e}$ is the density of electronic defect states in the sub-band of the CMO layer, $\Delta E_{\rm e}$ is the zero-field hopping energy barrier, $T$ and $E$ are the local temperature and electric field, respectively, and $A=\rm\pi\it r_{\rm CF}^{\rm 2}$ , $r_{\rm CF}$ being the filament radius, is the cross-sectional area of the filament at the interface with the CMO layer. The temperature and electric field in the CMO layer, for both LRS and HRS, are simulated by solving equations (4) and (5), while accounting for the experimental I-V non-linearity (see Supplementary Fig. S4 for details). The trap-to-trap tunneling parameters ( $N_{\rm e}$ , $\Delta E_{\rm e}$ , $a_{\rm e}$ ) are extracted from the fit using the same approach as described in previous works [26, 31].
### 4.5 Identical-pulse closed-loop scheme
The procedure begins with a quasi-static voltage sweep from 0 to $-1.5\,\mathrm{V}$ to reset each cell within the array to the HRS. Subsequently, a closed-loop scheme is initiated, which iteratively repeats the following two steps until convergence to G target within an acceptance range: (1) read the conductance of the ReRAM cell, and (2) if the measured value is below (above) the target conductance, apply a set (reset) programming pulse. During this iterative process, the cell conductance may fluctuate multiple times before eventually reaching the acceptance range. Starting from the HRS, this procedure is applied to the CMO/HfO x ReRAM array to sequentially program 35 representative conductance levels, ranging from approximately 10 ”S to 90 ”S, using acceptance ranges of both 0.2% G target and 2% G target. Unlike the conventional incremental-pulse closed-loop technique previously used for ReRAM [9, 43], where the amplitudes of set and reset pulses are gradually increased to achieve convergence, this work employs an identical-pulse closed-loop scheme to simplify the pulse generation circuitry design, using only two fixed amplitude values for the set ( $1.35\,\mathrm{V}$ or $1.5\,\mathrm{V}$ ) and two for the reset ( $-1.3\,\mathrm{V}$ or $-1.5\,\mathrm{V}$ ) pulses. Specifically, depending on G target, three ranges are used: from approximately 10 ”S to 30 ”S with $V_{\rm set}=1.35\,\mathrm{V}$ and $V_{\rm reset}=-1.5\,\mathrm{V}$ ; from 30 ”S to 60 ”S $V_{\rm set}=1.35\,\mathrm{V}$ and $V_{\rm reset}=-1.3\,\mathrm{V}$ ; and from 60 ”S to 90 ”S $V_{\rm set}=1.5\,\mathrm{V}$ and $V_{\rm reset}=-1.3\,\mathrm{V}$ . Fig. S5 b in Supplementary Information shows the flowchart of the identical-pulse closed-loop technique used in this work. The set / reset pulse width is fixed at 2.5 ”s due to setup limitations, even though previous work has demonstrated CMO/HfO x ReRAM switching with pulse width as short as $60\,\mathrm{ns}$ [25]. The reading pulse amplitude and width are $V_{\rm read}=0.2\,\mathrm{V}$ and 300 ”s, respectively. During the set, reset, and read operations of each 1T1R cell, the transistorâs gate voltage is controlled with constant values of $V_{\rm G}$ equal to $1.4\,\mathrm{V}$ , $3.3\,\mathrm{V}$ , and $3.3\,\mathrm{V}$ , respectively.
### 4.6 HW-aware simulation of analog MVM
The âaihwkitâ [44] simulation tool was used to perform MVM assessments including non-ideal behaviors and noise, and their effect on the computation accuracy with respect to floating-point operations. The MVM simulation included the exhibited programming noise, conductance relaxation, input and output quantization, and IR-drop across array wires. The âaihwkitâ allows to configure such noisy effects for dedicated memristive devices such as PCM by Nandakumar et al. [45] and ReRAM by Wan et al. [9]. Therefore, a unique phenomenological noise model for CMO/HfO x ReRAM devices for inference is developed to incorporate into the simulation both the characterized programming noise and conductance relaxation. Additionally, input and output are quantized with 6-bit and 8-bit resolution, respectively, and the IR-drop is considered, with 100 ”S as the maximum ReRAM conductance level and a default segment wire resistance of 0.35 $\Omega$ .
#### 4.6.1 Modelling the programming noise
For a target conductance G target, the deviceâs programmed conductance is defined as the target value plus normally distributed noise with a standard deviation $\sigma_{\rm prog}$ , which is a function of G target. As depicted in Fig. 3 e, the programming noise ( $\sigma_{\rm prog}$ ) of the CMO/HfO x ReRAM devices is statistically described by a first-order polynomial equation for a given acceptance range. The polynomial coefficients for acceptance ranges of 2% and 0.2% of G target are extracted from the characterization and introduced into the simulation environment. To assess the effects of the programming noise, each weight in the normalized matrix (ranging from [-1, 1]) is mapped to its corresponding conductance value (within the range [9, 89] ”S from Fig. 3 a), and is then further adjusted by the programming noise described by the extracted linear functions. Therefore, the MVM accuracy can be assessed immediately after programming ( $t=0$ ), see Fig. 4 f.
#### 4.6.2 Modelling the conductance relaxation
After programming, the conductance levels exhibit relaxation over time, as shown in Fig. 4. Unlike previous ReRAM drift characterizations reported by Wan et al. [9] the observed relaxation in CMO/HfO x ReRAM is approximately independent of the initial programmed conductance. Consequently, a new modelling approach in the âaihwkitâ is needed to accurately simulate the conductance relaxation effect, which differs from the methods derived from previous literature on ReRAM [9]. The conductance relaxation mean and standard deviation are modelled independently of G target and solely as a function of time after programming. The coefficients of the first-order polynomials describing the time dependence of both the mean and standard deviation of the programmed conductance are incorporated into the simulation environment to estimate conductance variations at any given inference time. By doing so, the MVM accuracy can be estimated after a period of time up to 10 years.
### 4.7 HW-aware simulation of analog training
#### 4.7.1 Generalized soft bounds model
The generalized soft bounds model (SBM) selection was based on the observed characteristics of the potentiation and depression since the devices did not strictly exhibit thorough saturation at the upper and lower boundaries (see Fig. S8 in Supplementary Information). The generalized SBM incorporates a tunable scale exponent ( $\gamma$ ) that describes abrupt and gradual trends toward the maximum and minimum conductance levels. This exponent parameter also varies depending on the conductance update direction. Therefore, the analytical expression of the generalized SBM implemented in the âaihwkitâ includes an asymmetry factor ( $\gamma_{\rm up\_down}$ ) to account for this behavior [38]. However, these two parameters do not have a direct physical equivalence, and therefore, cannot be derived from experimental traces. Hereby, $\gamma$ and $\gamma_{\rm up\_down}$ are obtained for each device through an independent linear fitting of the generalized SBM to the experimental response. In addition to the analytical parameters of the generalized SBM, devices in the âaihwkitâ are defined by a set of parameters that can be extracted from experimental traces. More precisely, the empirical maximum and minimum conductance, minimum conductance step size and its standard deviation, and the asymmetry between up and down response are considered ( $G_{\rm max}$ , $G_{\rm min}$ , $\Delta G_{\rm sp}$ , $\sigma_{\Delta G_{\rm sp}}$ , and $up\_down$ ). More details on the $up\_down$ parameter are provided in the Supplementary Information. In this regard, each simulated device is defined by 6 parameters: four empirically obtained ( $G_{\rm max}$ , $G_{\rm min}$ , $\Delta G_{\rm sp}$ and $up\_down$ ) and two analytically modelled from SBM linear fitting ( $\gamma$ and $\gamma_{\rm up\_down}$ ).
#### 4.7.2 Intra and inter-device variability
By extracting the standard deviation of the minimum conductance step size ( $\sigma_{\Delta G_{\rm sp}}$ ) from the experimental traces and incorporating it into the simulationâs device model, the device response intrinsically includes noise from cycle to cycle. This provides a realistic device behavior with intra-device variability. Furthermore, the network devices shall include inter-device variabilities to perform physically accurate simulations. To achieve this, two multi-variate Gaussian distributions, G 1 and G 2, are created (see Fig. S9 in Supplementary Information). G 1 is extracted from the experimentally obtained parameters: N states (which accounts for variations across devices in the G-range and step) and SP in the normalized G-range, whereas G 2 is fitted to the analytical model parameters obtained from the fitted generalized SBM ( $\gamma$ and $\gamma_{\rm up\_down}$ ). Therefore, variables from G 1 showed statistical independence from those of G 2. New device instances are independently sampled from the two Gaussian distributions to represent synapses on the DNN layers. The instantiated CMO/HfO x ReRAM devices include variations in the device response, conductance ranges, and asymmetrical behavior, thus providing a more hardware-aware and realistic scenario for analog training simulation.
## References
- Erdil [2024] Erdil, E.: Data Movement Bottlenecks to Large-Scale Model Training: Scaling Past 1e28 FLOP. Accessed: 2024-12-06 (2024). https://epoch.ai/blog/data-movement-bottlenecks-scaling-past-1e28-flop
- Jouppi et al. [2017] Jouppi, N.P., Young, C., Patil, N., Patterson, D., Agrawal, G., Bajwa, R., Bates, S., Bhatia, S., Boden, N., Borchers, A., Boyle, R., Cantin, P.-l., Chao, C., Clark, C., Coriell, J., Daley, M., Dau, M., Dean, J., Gelb, B., Ghaemmaghami, T.V., Gottipati, R., Gulland, W., Hagmann, R., Ho, C.R., Hogberg, D., Hu, J., Hundt, R., Hurt, D., Ibarz, J., Jaffey, A., Jaworski, A., Kaplan, A., Khaitan, H., Killebrew, D., Koch, A., Kumar, N., Lacy, S., Laudon, J., Law, J., Le, D., Leary, C., Liu, Z., Lucke, K., Lundin, A., MacKean, G., Maggiore, A., Mahony, M., Miller, K., Nagarajan, R., Narayanaswami, R., Ni, R., Nix, K., Norrie, T., Omernick, M., Penukonda, N., Phelps, A., Ross, J., Ross, M., Salek, A., Samadiani, E., Severn, C., Sizikov, G., Snelham, M., Souter, J., Steinberg, D., Swing, A., Tan, M., Thorson, G., Tian, B., Toma, H., Tuttle, E., Vasudevan, V., Walter, R., Wang, W., Wilcox, E., Yoon, D.H.: In-datacenter performance analysis of a tensor processing unit. ACM SIGARCH Computer Architecture News 45 (2017) https://doi.org/10.1145/3140659.3080246
- Sze et al. [2017] Sze, V., Chen, Y.H., Yang, T.J., Emer, J.S.: Efficient Processing of Deep Neural Networks: A Tutorial and Survey (2017). https://doi.org/10.1109/JPROC.2017.2761740
- Haensch et al. [2019] Haensch, W., Gokmen, T., Puri, R.: The next generation of deep learning hardware: Analog computing. Proceedings of the IEEE 107 (2019) https://doi.org/10.1109/JPROC.2018.2871057
- Sebastian et al. [2020] Sebastian, A., Le Gallo, M., Khaddam-Aljameh, R., Eleftheriou, E.: Memory devices and applications for in-memory computing (2020). https://doi.org/10.1038/s41565-020-0655-z
- Mutlu et al. [2019] Mutlu, O., Ghose, S., GĂłmez-Luna, J., Ausavarungnirun, R.: Processing data where it makes sense: Enabling in-memory computation. Microprocessors and Microsystems 67 (2019) https://doi.org/10.1016/j.micpro.2019.01.009
- Tsai et al. [2023] Tsai, H., Narayanan, P., Jain, S., Ambrogio, S., Hosokawa, K., Ishii, M., MacKin, C., Chen, C.T., Okazaki, A., Nomura, A., Boybat, I., Muralidhar, R., Frank, M.M., Yasuda, T., Friz, A., Kohda, Y., Chen, A., Fasoli, A., Rasch, M.J., Wozniak, S., Luquin, J., Narayanan, V., Burr, G.W.: Architectures and circuits for analog-memory-based hardware accelerators for deep neural networks (invited). In: Proceedings - IEEE International Symposium on Circuits and Systems, vol. 2023-May (2023). https://doi.org/10.1109/ISCAS46773.2023.10181650
- Burr et al. [2017] Burr, G.W., Shelby, R.M., Sebastian, A., Kim, S., Kim, S., Sidler, S., Virwani, K., Ishii, M., Narayanan, P., Fumarola, A., Sanches, L.L., Boybat, I., Le Gallo, M., Moon, K., Woo, J., Hwang, H., Leblebici, Y.: Neuromorphic computing using non-volatile memory (2017). https://doi.org/10.1080/23746149.2016.1259585
- Wan et al. [2022] Wan, W., Kubendran, R., Schaefer, C., Eryilmaz, S.B., Zhang, W., Wu, D., Deiss, S., Raina, P., Qian, H., Gao, B., Joshi, S., Wu, H., Wong, H.-S.P., Cauwenberghs, G.: A compute-in-memory chip based on resistive random-access memory. Nature 608 (7923), 504â512 (2022) https://doi.org/10.1038/s41586-022-04992-8
- Yao et al. [2020] Yao, P., Wu, H., Gao, B., Tang, J., Zhang, Q., Zhang, W., Yang, J.J., Qian, H.: Fully hardware-implemented memristor convolutional neural network. Nature 577 (2020) https://doi.org/10.1038/s41586-020-1942-4
- Ambrogio et al. [2023] Ambrogio, S., Narayanan, P., Okazaki, A., Fasoli, A., Mackin, C., Hosokawa, K., Nomura, A., Yasuda, T., Chen, A., Friz, A., Ishii, M., Luquin, J., Kohda, Y., Saulnier, N., Brew, K., Choi, S., Ok, I., Philip, T., Chan, V., Silvestre, C., Ahsan, I., Narayanan, V., Tsai, H., Burr, G.W.: An analog-ai chip for energy-efficient speech recognition and transcription. Nature 620 (2023) https://doi.org/10.1038/s41586-023-06337-5
- Le Gallo et al. [2023] Le Gallo, M., Khaddam-Aljameh, R., Stanisavljevic, M., Vasilopoulos, A., Kersting, B., Dazzi, M., Karunaratne, G., BrĂ€ndli, M., Singh, A., MĂŒller, S.M., BĂŒchel, J., Timoneda, X., Joshi, V., Rasch, M.J., Egger, U., Garofalo, A., Petropoulos, A., Antonakopoulos, T., Brew, K., Choi, S., Ok, I., Philip, T., Chan, V., Silvestre, C., Ahsan, I., Saulnier, N., Narayanan, V., Francese, P.A., Eleftheriou, E., Sebastian, A.: A 64-core mixed-signal in-memory compute chip based on phase-change memory for deep neural network inference. Nature Electronics 6 (2023) https://doi.org/10.1038/s41928-023-01010-1
- [13] Gemini: A Family of Highly Capable Multimodal Models (2024). https://arxiv.org/abs/2312.11805
- Woo and Yu [2018] Woo, J., Yu, S.: Resistive memory-based analog synapse: The pursuit for linear and symmetric weight update. IEEE Nanotechnology Magazine 12 (2018) https://doi.org/10.1109/MNANO.2018.2844902
- Yin et al. [2020] Yin, S., Sun, X., Yu, S., Seo, J.S.: High-throughput in-memory computing for binary deep neural networks with monolithically integrated rram and 90-nm cmos. IEEE Transactions on Electron Devices 67 (2020) https://doi.org/10.1109/TED.2020.3015178
- Zahoor et al. [2020] Zahoor, F., Zulkifli, T.Z.A., Khanday, F.A.: Resistive Random Access Memory (RRAM): an Overview of Materials, Switching Mechanism, Performance, Multilevel Cell (mlc) Storage, Modeling, and Applications (2020). https://doi.org/10.1186/s11671-020-03299-9
- Tang et al. [2019] Tang, J., Bishop, D., Kim, S., Copel, M., Gokmen, T., Todorov, T., Shin, S., Lee, K.T., Solomon, P., Chan, K., Haensch, W., Rozen, J.: Ecram as scalable synaptic cell for high-speed, low-power neuromorphic computing. In: Technical Digest - International Electron Devices Meeting, IEDM, vol. 2018-December (2019). https://doi.org/10.1109/IEDM.2018.8614551
- Li et al. [2018] Li, Y., Kim, S., Sun, X., Solomon, P., Gokmen, T., Tsai, H., Koswatta, S., Ren, Z., Mo, R., Yeh, C.C., Haensch, W., Leobandung, E.: Capacitor-based cross-point array for analog neural network with record symmetry and linearity. In: Digest of Technical Papers - Symposium on VLSI Technology, vol. 2018-June (2018). https://doi.org/10.1109/VLSIT.2018.8510648
- Ielmini [2016] Ielmini, D.: Resistive switching memories based on metal oxides: Mechanisms, reliability and scaling (2016). https://doi.org/10.1088/0268-1242/31/6/063002
- Gokmen and Vlasov [2016] Gokmen, T., Vlasov, Y.: Acceleration of deep neural network training with resistive cross-point devices: Design considerations. Frontiers in Neuroscience 10 (2016) https://doi.org/10.3389/fnins.2016.00333
- Gokmen and Haensch [2020] Gokmen, T., Haensch, W.: Algorithm for training neural networks on resistive device arrays. Frontiers in Neuroscience 14 (2020) https://doi.org/10.3389/fnins.2020.00103
- Gong et al. [2022] Gong, N., Rasch, M.J., Seo, S.C., Gasasira, A., Solomon, P., Bragaglia, V., Consiglio, S., Higuchi, H., Park, C., Brew, K., Jamison, P., Catano, C., Saraf, I., Athena, F.F., Silvestre, C., Liu, X., Khan, B., Jain, N., McDermott, S., Johnson, R., Estrada-Raygoza, I., Li, J., Gokmen, T., Li, N., Pujari, R., Carta, F., Miyazoe, H., Frank, M.M., Koty, D., Yang, Q., Clark, R., Tapily, K., Wajda, C., Mosden, A., Shearer, J., Metz, A., Teehan, S., Saulnier, N., Offrein, B.J., Tsunomura, T., Leusink, G., Narayanan, V., Ando, T.: Deep learning acceleration in 14nm cmos compatible reram array: device, material and algorithm co-optimization. In: Technical Digest - International Electron Devices Meeting, IEDM, vol. 2022-December (2022). https://doi.org/10.1109/IEDM45625.2022.10019569
- Rasch et al. [2024] Rasch, M.J., Carta, F., Fagbohungbe, O., Gokmen, T.: Fast and robust analog in-memory deep neural network training. Nature Communications 15 (1), 7133 (2024) https://doi.org/10.1038/s41467-024-51221-z
- Stecconi et al. [2024] Stecconi, T., Bragaglia, V., Rasch, M.J., Carta, F., Horst, F., Falcone, D.F., Kate, S.C., Gong, N., Ando, T., Olziersky, A., Offrein, B.: Analog resistive switching devices for training deep neural networks with the novel tiki-taka algorithm. Nano Letters 24 (2024) https://doi.org/10.1021/acs.nanolett.3c03697
- Lombardo et al. [2024] Lombardo, D.G.F., Ram, M.S., Stecconi, T., Choi, W., La Porta, A., Falcone, D.F., Offrein, B., Bragaglia, V.: Read noise analysis in analog conductive-metal-oxide/hfox reram devices. In: 2024 Device Research Conference (DRC), pp. 1â2 (2024). https://doi.org/10.1109/DRC61706.2024.10643760
- Falcone et al. [2024] Falcone, D.F., Menzel, S., Stecconi, T., Galetta, M., La Porta, A., Offrein, B.J., Bragaglia, V.: Analytical modelling of the transport in analog filamentary conductive-metal-oxide/hfox reram devices. Nanoscale Horiz. 9, 775â784 (2024) https://doi.org/10.1039/D4NH00072B
- Padovani et al. [2015] Padovani, A., Larcher, L., Pirrotta, O., Vandelli, L., Bersuker, G.: Microscopic modeling of hfox rram operations: From forming to switching. IEEE Transactions on Electron Devices 62 (2015) https://doi.org/10.1109/TED.2015.2418114
- Kröger and Vink [1958] Kröger, F.A., Vink, H.J.: Relations between the concentrations of imperfections in solids. Journal of Physics and Chemistry of Solids 5 (1958) https://doi.org/10.1016/0022-3697(58)90069-6
- Padovani et al. [2012] Padovani, A., Larcher, L., Padovani, P., Cagli, C., Salvo, B.D.: Understanding the role of the ti metal electrode on the forming of hfo 2-based rrams. 2012 4th IEEE International Memory Workshop, IMW 2012, 1â4 (2012) https://doi.org/10.1109/IMW.2012.6213667
- Padovani et al. [2013] Padovani, A., Larcher, L., Bersuker, G., Pavan, P.: Charge transport and degradation in hfo2 and hfox dielectrics. IEEE Electron Device Letters 34, 680â82 (2013) https://doi.org/10.1109/LED.2013.2251602
- Falcone et al. [2023] Falcone, D.F., Menzel, S., Stecconi, T., La Porta, A., Carraria-Martinotti, L., Offrein, B.J., Bragaglia, V.: Physical modeling and design rules of analog conductive metal oxide-hfo2reram. In: 2023 IEEE International Memory Workshop, IMW 2023 - Proceedings (2023). https://doi.org/10.1109/IMW56887.2023.10145936
- Galetta et al. [2024] Galetta, M., Falcone, D.F., Menzel, S., La Porta, A., Stecconi, T., Choi, W., Offrein, B.J., Bragaglia, V.: Compact model of conductive-metal-oxide/hfox analog filamentary reram devices. In: 2024 IEEE European Solid-State Electronics Research Conference (ESSERC), pp. 749â752 (2024). https://doi.org/10.1109/ESSERC62670.2024.10719489
- Dittmann et al. [2021] Dittmann, R., Menzel, S., Waser, R.: Nanoionic memristive phenomena in metal oxides: the valence change mechanism. Advances in Physics 70 (2021) https://doi.org/10.1080/00018732.2022.2084006
- Joshi et al. [2020] Joshi, V., Le Gallo, M., Haefeli, S., Boybat, I., Nandakumar, S.R., Piveteau, C., Dazzi, M., Rajendran, B., Sebastian, A., Eleftheriou, E.: Accurate deep neural network inference using computational phase-change memory. Nature Communications 11 (2020) https://doi.org/10.1038/s41467-020-16108-9
- Tsai et al. [2019] Tsai, H., Ambrogio, S., MacKin, C., Narayanan, P., Shelby, R.M., Rocki, K., Chen, A., Burr, G.W.: Inference of long-short term memory networks at software-equivalent accuracy using 2.5m analog phase change memory devices. In: Digest of Technical Papers - Symposium on VLSI Technology, vol. 2019-June (2019). https://doi.org/10.23919/VLSIT.2019.8776519
- Le Gallo et al. [2018] Le Gallo, M., Sebastian, A., Cherubini, G., Giefers, H., Eleftheriou, E.: Compressed sensing with approximate message passing using in-memory computing. IEEE Transactions on Electron Devices 65 (2018) https://doi.org/%****âŁmain.bblâŁLineâŁ825âŁ****10.1109/TED.2018.2865352
- Zhao et al. [2018] Zhao, M., Wu, H., Gao, B., Zhang, Q., Wu, W., Wang, S., Xi, Y., Wu, D., Deng, N., Yu, S., Chen, H.Y., Qian, H.: Investigation of statistical retention of filamentary analog rram for neuromophic computing. In: Technical Digest - International Electron Devices Meeting, IEDM (2018). https://doi.org/10.1109/IEDM.2017.8268522
- Rasch et al. [2021] Rasch, M.J., Moreda, D., Gokmen, T., Le Gallo, M., Carta, F., Goldberg, C., El Maghraoui, K., Sebastian, A., Narayanan, V.: A flexible and fast pytorch toolkit for simulating training and inference on analog crossbar arrays. In: 2021 IEEE 3rd International Conference on Artificial Intelligence Circuits and Systems (AICAS), pp. 1â4 (2021). https://doi.org/10.1109/AICAS51828.2021.9458494
- Chen et al. [2023] Chen, P., Liu, F., Lin, P., Li, P., Xiao, Y., Zhang, B., Pan, G.: Open-loop analog programmable electrochemical memory array. Nature Communications 14 (2023) https://doi.org/10.1038/s41467-023-41958-4
- Frascaroli et al. [2018] Frascaroli, J., Brivio, S., Covi, E., Spiga, S.: Evidence of soft bound behaviour in analogue memristive devices for neuromorphic computing. Scientific Reports 8 (1), 7178 (2018) https://doi.org/10.1038/s41598-018-25376-x
- Abedin et al. [2023] Abedin, M., Gong, N., Beckmann, K., Liehr, M., Saraf, I., Straten, O.V., Ando, T., Cady, N.: Material to system-level benchmarking of cmos-integrated rram with ultra-fast switching for low power on-chip learning. Scientific Reports 13 (2023) https://doi.org/%****âŁmain.bblâŁLineâŁ925âŁ****10.1038/s41598-023-42214-x
- Mott and Gurney [1950] Mott, N.F., Gurney, R.W.: Electronic processes in ionic crystals. Oxford at the Clarendon Press, 2 ed. (1950)
- Alibart et al. [2012] Alibart, F., Gao, L., Hoskins, B.D., Strukov, D.B.: High precision tuning of state for memristive devices by adaptable variation-tolerant algorithm. Nanotechnology 23 (2012) https://doi.org/10.1088/0957-4484/23/7/075201
- Rasch et al. [2021] Rasch, M.J., Moreda, D., Gokmen, T., Le Gallo, M., Carta, F., Goldberg, C., El Maghraoui, K., Sebastian, A., Narayanan, V.: A flexible and fast pytorch toolkit for simulating training and inference on analog crossbar arrays. In: 2021 IEEE 3rd International Conference on Artificial Intelligence Circuits and Systems (AICAS), pp. 1â4 (2021). https://doi.org/10.1109/AICAS51828.2021.9458494
- Nandakumar et al. [2019] Nandakumar, S.R., Boybat, I., Joshi, V., Piveteau, C., Le Gallo, M., Rajendran, B., Sebastian, A., Eleftheriou, E.: Phase-change memory models for deep learning training and inference. In: 2019 26th IEEE International Conference on Electronics, Circuits and Systems (ICECS), pp. 727â730 (2019). https://doi.org/10.1109/ICECS46596.2019.8964852
Supplementary information This manuscript is supported by additional supplementary information provided in a separate document.
Acknowledgements The authors acknowledge the Binnig and Rohrer Nanotechnology Center (BRNC) at IBM Research Europe - Zurich. Special thanks go to Jean-Michel Portal, Eloi Muhr and Dominique Drouin for their contributions to the design of the NMOS transistors used in this work. The authors also extend their gratitude to Stephan Menzel for the fruitful discussions and to Ralph Heller for his assistance in wire-bonding the chip. This work is funded by SNSF ALMOND (grantID: 198612), by the European Union and Swiss state secretariat SERI within the H2020 MeM-Scales (grantID: 871371), MANIC (grantID: 861153), PHASTRAC (grantID: 101092096) and CHIST-ERA UNICO (20CH21-186952) projects.
Author contributions Conceptualization: D. F. F. and V. B.; hardware fabrication: D. F. F. and L. B. L.; electrical characterization: D. F. F, W. C., T. S., F. H., physical simulations: D. F. F.; inference and training simulations: V. C., D. F. F.; NMOS transistor design : N. G., F. A.; result interpretation: D. F. F., V. C., W. C., V. B., M. G., A. L. P. and B. J. O., supervision: V. B. and B. J. O.; manuscript writing: D. F. F., V. C.; data curation: D. F. F., V. C. and V. B.; manuscript review and editing: all authors; funding acquisition: B. J. O. and V. B.
Competing interests The authors declare no competing interests.
Data availability The data that support the plots within this paper and other findings of this study are available from the corresponding authors upon reasonable request.
Code availability The repositories containing the source codes used in this work for analog inference simulations and CMO/HfO x ReRAM noise model can be found at this link and this link, respectively.
## Supplementary Information
<details>
<summary>x7.png Details</summary>
### Visual Description
## Scientific Diagram: Device Structure and Filament Characteristics
### Overview
The image presents a series of diagrams illustrating the structure and electrical characteristics of a memristive device. It includes a bird's-eye view and a y-z cross-section of the device, followed by two graphs depicting filament radius fit and CMO layer defect redistribution under varying voltage conditions. The graphs utilize a color map to represent device count.
### Components/Axes
**a) Device bird's-eye view:**
- Dimensions: 200nm x 280nm
- Shows a rectangular device with circular electrodes.
**b) Device y-z view:**
- Layers: SiO<sub>x</sub> (50nm), TiN (20nm), CMO (20nm), HfO<sub>x</sub> (4nm), TiN (20nm).
- CF (presumably a conductive filament) is indicated.
**c) Filament radius fit:**
- X-axis: Voltage<sub>ITIR</sub> [V] (Scale: 0.0 to 3.6)
- Y-axis: Current<sub>ITIR</sub> [A] (Logarithmic scale: 10<sup>-9</sup> to 10<sup>-3</sup>)
- Color Map: Represents the number of devices (Scale: 1 to 32)
- Legend:
- Black dashed line: "Model"
- Circles: Ï<sub>CMO</sub> = 5 S/cm, r<sub>CF</sub> = 11 nm
- Symbols: V<sub>o</sub>, CMO, 2r<sub>CF</sub>, TiN, HfO<sub>x</sub>
- Annotation: â
**d) CMO layer defect redistribution:**
- X-axis: Voltage<sub>ITIR</sub> [V] (Scale: -1.4 to 0.0)
- Y-axis: Current<sub>ITIR</sub> [A] (Logarithmic scale: 10<sup>-7</sup> to 10<sup>-3</sup>)
- Color Map: Represents the number of devices (Scale: 1 to 32)
- Legend:
- Black dashed line: "Model"
- Circles: Ï<sub>CMO</sub> = 37 S/cm, r<sub>CF</sub> = 11 nm
- Symbols: V<sub>o</sub>, CMO, 2r<sub>CF</sub>, TiN, HfO<sub>x</sub>
- Annotation: âĄ
### Detailed Analysis or Content Details
**a) Device bird's-eye view:**
- The device is a rectangular structure with dimensions approximately 200nm x 280nm.
- Two circular electrodes are visible.
**b) Device y-z view:**
- The device consists of several layers: SiO<sub>x</sub>, TiN, CMO, HfO<sub>x</sub>, and TiN.
- The thicknesses of each layer are indicated: SiO<sub>x</sub> (50nm), TiN (20nm), CMO (20nm), HfO<sub>x</sub> (4nm), TiN (20nm).
- A conductive filament (CF) is shown extending through the layers.
**c) Filament radius fit:**
- The graph shows multiple curves representing the current-voltage (I-V) characteristics of the device.
- The curves generally exhibit a sharp increase in current at a specific voltage, followed by saturation.
- The color map indicates that the highest device counts (darker green, around 32) are concentrated at lower voltages (around 0.5-1.5V) and moderate current levels (around 10<sup>-6</sup> to 10<sup>-5</sup> A).
- The "Model" (dashed black line) shows a similar trend to the experimental data.
- The circles represent a specific set of parameters: Ï<sub>CMO</sub> = 5 S/cm and r<sub>CF</sub> = 11 nm.
- The annotation â points to the initial rise in current.
**d) CMO layer defect redistribution:**
- Similar to (c), this graph displays I-V characteristics.
- The curves are shifted towards negative voltages compared to graph (c).
- The highest device counts (darker green, around 32) are concentrated at negative voltages (around -0.5 to -1.0V) and moderate current levels (around 10<sup>-6</sup> to 10<sup>-5</sup> A).
- The "Model" (dashed black line) shows a similar trend to the experimental data.
- The circles represent a different set of parameters: Ï<sub>CMO</sub> = 37 S/cm and r<sub>CF</sub> = 11 nm.
- The annotation ⥠points to the initial rise in current.
### Key Observations
- The I-V curves in both graphs (c) and (d) demonstrate memristive behavior, characterized by a non-linear relationship between current and voltage.
- The color maps reveal that the device performance is sensitive to voltage and current levels.
- The two graphs show different voltage ranges and parameter sets, suggesting that the device characteristics can be tuned by adjusting material properties (Ï<sub>CMO</sub>) and filament radius (r<sub>CF</sub>).
- The "Model" closely matches the experimental data in both graphs, indicating the model's validity.
- The shift in I-V curves between (c) and (d) suggests a change in the dominant switching mechanism or defect distribution within the CMO layer.
### Interpretation
The diagrams illustrate the structure and electrical behavior of a resistive switching memory device. The device consists of a stack of thin films, including SiO<sub>x</sub>, TiN, CMO, and HfO<sub>x</sub>, with a conductive filament (CF) forming the switching element.
The I-V characteristics shown in graphs (c) and (d) demonstrate the memristive effect, where the resistance of the device changes depending on the history of applied voltage. The color maps provide insights into the distribution of device performance, revealing that certain voltage and current ranges yield higher device counts.
The differences between graphs (c) and (d) suggest that the CMO layer plays a crucial role in the switching mechanism. The change in conductivity (Ï<sub>CMO</sub>) and the filament radius (r<sub>CF</sub>) influence the I-V characteristics and device performance. The annotations â and ⥠highlight the initial switching events, indicating the voltage thresholds for filament formation or rupture.
The close agreement between the experimental data and the "Model" suggests that the underlying physics of the device is well understood. This information can be used to optimize the device structure and material properties for improved performance and reliability. The defect redistribution in the CMO layer, as indicated in graph (d), is likely a key factor in the switching process and warrants further investigation.
</details>
Figure S1: ReRAM forming modelling. The CMO/HfO x ReRAM device is simulated using a 3D FEM in COMSOL Multiphysics 5.2 software. a The birdâs-eye view and b the lateral y-z view of the deviceâs geometry and material stack are shown. Due to the temperature and electric field confinement, an effective device area of 200 $\times$ 200 nm 2 is considered for the simulation to reduce computational resource demands. c The experimental array forming data in the low-voltage linear regime (from 0 to $0.2\,\mathrm{V}$ ) are fitted to extract the average filament radius. d The increase in experimental conductance resulting from a negative voltage sweep after the forming event is modelled as an effective increase in the electrical conductivity of the CMO layer, due to a radial redistribution of defects.
<details>
<summary>x8.png Details</summary>
### Visual Description
## Cumulative Distribution Plots & IV Curve: Device Forming Characteristics
### Overview
The image presents four separate plots characterizing the forming behavior of different resistive switching devices. Plots (a) and (d) show cumulative probability distributions of forming voltages for 1T1R and ReRAM devices, respectively. Plot (b) shows the cumulative probability distribution of forming current for 1T1R devices. Plot (c) displays an IV curve demonstrating the triode resistance of a device at a fixed gate voltage.
### Components/Axes
* **Plot a: 1T1R forming voltage**
* X-axis: V<sup>1T1R</sup><sub>forming</sub> [V] (Forming Voltage for 1T1R device) - Scale: 2 to 4 V
* Y-axis: Cumulative Probability [%] - Scale: 0 to 100%
* Data Series: Blue dots representing the cumulative distribution. Dashed black line representing the mean.
* **Plot b: 1T1R forming current**
* X-axis: I<sup>1T1R</sup><sub>forming</sub> [”A] (Forming Current for 1T1R device) - Scale: 100 to 500 ”A
* Y-axis: Cumulative Probability [%] - Scale: 0 to 100%
* Data Series: Red dots representing the cumulative distribution. Dashed black line representing the mean.
* **Plot c: Triode resistance at V<sub>G</sub>=1.2V**
* X-axis: V<sub>DS</sub> [V] (Drain-Source Voltage) - Scale: 0 to 4 V
* Y-axis: I<sub>DS</sub> [mA] (Drain-Source Current) - Scale: 0 to 0.8 mA
* Data Series: Blue line representing the IV curve.
* **Plot d: ReRAM forming voltage**
* X-axis: V<sup>ReRAM</sup><sub>forming</sub> [V] (Forming Voltage for ReRAM device) - Scale: 2 to 4 V
* Y-axis: Cumulative Probability [%] - Scale: 0 to 100%
* Data Series: Green dots representing the cumulative distribution. Dashed black line representing the mean.
### Detailed Analysis or Content Details
* **Plot a: 1T1R forming voltage**
* The blue data points show a steep increase in cumulative probability between approximately 3.1 V and 3.5 V.
* The mean forming voltage is indicated by a dashed black line at approximately 3.38 V.
* **Plot b: 1T1R forming current**
* The red data points show a steep increase in cumulative probability between approximately 200 ”A and 300 ”A.
* The mean forming current is indicated by a dashed black line at approximately 258 ”A.
* **Plot c: Triode resistance at V<sub>G</sub>=1.2V**
* The blue line shows a non-linear relationship between V<sub>DS</sub> and I<sub>DS</sub>.
* The current increases rapidly from 0 mA to approximately 0.4 mA as V<sub>DS</sub> increases from 0 V to 3 V.
* The triode resistance is labeled as approximately 0.8 kΩ.
* **Plot d: ReRAM forming voltage**
* The green data points show a steep increase in cumulative probability between approximately 3.0 V and 3.3 V.
* The mean forming voltage is indicated by a dashed black line at approximately 3.17 V.
### Key Observations
* The 1T1R devices exhibit forming voltages clustered around 3.38 V and forming currents clustered around 258 ”A.
* The ReRAM devices exhibit forming voltages clustered around 3.17 V.
* The IV curve in plot (c) demonstrates a clear triode region with a resistance of 0.8 kΩ at V<sub>G</sub> = 1.2 V.
* All cumulative distribution plots show a relatively narrow distribution of forming voltages/currents, suggesting a consistent forming process.
### Interpretation
The data suggests that the 1T1R and ReRAM devices have different forming voltage requirements. The 1T1R devices require slightly higher forming voltages (mean of 3.38 V) compared to the ReRAM devices (mean of 3.17 V). The forming current distribution for 1T1R devices is also relatively narrow, indicating a consistent forming process. The IV curve in plot (c) provides insight into the device's behavior in the triode region, which is crucial for understanding its switching characteristics. The consistent forming characteristics, as indicated by the steep cumulative distributions, suggest a reliable and repeatable fabrication process. The difference in forming voltages between 1T1R and ReRAM could be attributed to differences in their material properties and device structures. The triode resistance value provides a key parameter for modeling and simulating the device's performance.
</details>
Figure S2: Experimental CMO/HfO x ReRAM array forming data. a The forming voltage distribution of the 1T1R cells within the array, defined as the voltage required to trigger the highest current increase during the quasi-static voltage sweep from 0 to $3.6\,\mathrm{V}$ in Fig. 2 a of the manuscript. b The array forming current distribution corresponding to $V=V_{\mathrm{forming}}^{\mathrm{1T1R}}$ . c The experimental resistance of the transistor in the triode region at $V_{\mathrm{G}}=\mathrm{1.2\,V}$ , extracted from a linear fit from 0 to $0.2\,\mathrm{V}$ of the transistor output characteristic. d The forming voltage distribution of the ReRAM array, shown in Fig. 2 c of the manuscript, computed as $V_{\mathrm{forming}}^{\mathrm{ReRAM}}$ = $V_{\mathrm{forming}}^{\mathrm{1T1R}}$ - $R_{\mathrm{DS}}^{\mathrm{triode}}$ * $I_{\mathrm{forming}}^{\mathrm{1T1R}}$ .
<details>
<summary>x9.png Details</summary>
### Visual Description
## Chart: RESET: CMO defect depletion
### Overview
The image presents a chart illustrating the relationship between current (Current<sub>1T1R</sub>) and voltage (Voltage<sub>1T1R</sub>) during a reset process, specifically focusing on CMO defect depletion. The chart displays numerous data curves, with a color gradient representing the density of devices. Below the chart are two diagrams illustrating the device structure before and after defect depletion.
### Components/Axes
* **Title:** RESET: CMO defect depletion (Top-center)
* **X-axis:** Voltage<sub>1T1R</sub> [V] (Bottom-center), ranging from approximately -1.5 to 0.0.
* **Y-axis:** Current<sub>1T1R</sub> [A] (Left-center), on a logarithmic scale ranging from 10<sup>-9</sup> to 10<sup>-3</sup>.
* **Colorbar/Legend:** Located on the right side of the chart, representing the number of devices. The color gradient ranges from dark purple (1 device) to light green (32 devices).
* **Data Series:** Numerous curves representing individual device responses.
* **Annotations:** Two circles with arrows indicating trends. A number "3" is also present with an arrow.
* **Diagrams:** Two schematic diagrams illustrating the device structure before and after defect depletion.
### Detailed Analysis or Content Details
The chart displays a large number of curves, each representing the current-voltage characteristics of a single device. The curves generally show a decreasing current as voltage increases from negative to positive values. The color of each curve corresponds to the density of devices exhibiting similar behavior, as indicated by the colorbar.
**Trend Analysis:**
* **Overall Trend:** The majority of curves exhibit a steep decrease in current as the voltage approaches 0.0 V.
* **High-Density Region:** The region around Voltage<sub>1T1R</sub> = 0.0 V and Current<sub>1T1R</sub> = 10<sup>-7</sup> A to 10<sup>-6</sup> A shows a high density of devices (green color).
* **Low-Density Region:** The region around Voltage<sub>1T1R</sub> = -1.5 V and Current<sub>1T1R</sub> = 10<sup>-3</sup> A shows a low density of devices (dark purple color).
* **Arrow 1 (Top-Left):** Points to a region of curves that initially show a relatively high current at negative voltages, then decrease rapidly as voltage approaches 0.
* **Arrow 2 (Bottom-Right):** Points to the region where most curves converge, indicating a common reset behavior.
* **Annotation "3":** Points to a region of curves that exhibit a more gradual decrease in current.
**Diagram Analysis:**
* **Left Diagram (Before Depletion):** Shows a layered structure: TiN / HfO<sub>x</sub> / CMO / HfO<sub>x</sub> / TiN. The CMO layer contains several "V<sub>O</sub>" defects (oxygen vacancies) represented as white circles.
* **Right Diagram (After Depletion):** Shows a similar layered structure, but the CMO layer has significantly fewer V<sub>O</sub> defects. The defects appear to have been reduced or eliminated.
### Key Observations
* The chart demonstrates a clear correlation between voltage and current during the reset process.
* The color gradient indicates that the reset behavior is relatively consistent across a large number of devices.
* The diagrams suggest that the reset process involves the depletion of oxygen vacancies (V<sub>O</sub>) in the CMO layer.
* The curves show a wide range of current values at a given voltage, indicating some variability in device characteristics.
* The number "3" annotation suggests a subset of devices exhibiting a different reset behavior.
### Interpretation
The data suggests that the reset process in these devices is driven by the depletion of oxygen vacancies in the CMO layer. Applying a voltage causes the defects to be eliminated, leading to a decrease in current. The color gradient on the chart indicates that this process is relatively consistent across the device population. The diagrams visually represent this defect depletion mechanism. The variability in the curves suggests that the initial distribution of defects and other device parameters may influence the reset behavior. The annotation "3" could indicate devices with a different defect structure or a different reset mechanism. The chart provides valuable insights into the physics of the reset process and could be used to optimize device performance. The logarithmic scale on the Y-axis is crucial for visualizing the wide range of current values.
</details>
Figure S3: The experimental arrayâs response to the voltage sweep from 0 to $-1.5\,\mathrm{V}$ , following the positive forming and the initial negative voltage sweep (denoted as step (1) and (2) in Fig. 2 a of the manuscript, respectively). The oxygen vacancies in the CMO layer radially spread outward, depleting the CMO defect sub-band within a half-spherical volume at the interface with the conductive filament, leading to a reset process.
<details>
<summary>x10.png Details</summary>
### Visual Description
## Charts: Average Temperature and Electric Field in the CMO Layer
### Overview
The image presents two charts, labeled 'a' and 'b', both depicting the relationship between Voltage and either Temperature or Electric Field within a CMO layer. Chart 'a' shows Average Temperature (T) versus Voltage, while chart 'b' shows Average Electric Field (E) versus Voltage. Both charts compare two states: Low Resistance State (LRS) and High Resistance State (HRS).
### Components/Axes
**Chart a: Average T in the CMO layer**
* **X-axis:** Voltage [V], ranging from approximately -1.0 to 0.9.
* **Y-axis:** Temperature [K], ranging from approximately 250 to 500.
* **Legend:**
* LRS (Low Resistance State) - Red line
* HRS (High Resistance State) - Blue line
**Chart b: Average E in the CMO layer**
* **X-axis:** Voltage [V], ranging from approximately -1.0 to 0.9.
* **Y-axis:** Electric Field [V/m], displayed on a logarithmic scale from approximately 10<sup>6</sup> to 10<sup>8</sup>.
* **Legend:**
* LRS (Low Resistance State) - Red line
* HRS (High Resistance State) - Blue line
The legends are positioned in the top-left corner of each chart.
### Detailed Analysis or Content Details
**Chart a: Average T in the CMO layer**
* **LRS (Red Line):** The temperature starts at approximately 480 K at -1.0 V, rapidly decreases to a minimum of approximately 260 K at around 0.1 V, and then increases again to approximately 320 K at 0.9 V. The trend is a sharp dip followed by a gradual rise.
* **HRS (Blue Line):** The temperature starts at approximately 280 K at -1.0 V, gradually decreases to a minimum of approximately 260 K at 0.0 V, and then rapidly increases to approximately 450 K at 0.9 V. The trend is a slow decrease followed by a sharp rise.
**Chart b: Average E in the CMO layer**
* **LRS (Red Line):** The electric field starts at approximately 9.0 x 10<sup>7</sup> V/m at -1.0 V, remains relatively constant around 8.0 x 10<sup>7</sup> V/m until approximately 0.0 V, and then rapidly increases to approximately 9.5 x 10<sup>7</sup> V/m at 0.9 V. The trend is relatively flat with a slight increase.
* **HRS (Blue Line):** The electric field starts at approximately 1.0 x 10<sup>7</sup> V/m at -1.0 V, rapidly increases to approximately 8.0 x 10<sup>7</sup> V/m at 0.0 V, and then continues to increase to approximately 9.0 x 10<sup>7</sup> V/m at 0.9 V. The trend is a sharp increase followed by a more gradual increase.
### Key Observations
* In Chart a, the LRS exhibits a much more pronounced temperature dip around 0.1 V compared to the HRS.
* In Chart b, the HRS experiences a much more significant increase in electric field around 0.0 V compared to the LRS.
* Both charts show a divergence in behavior between LRS and HRS as the voltage approaches 0.9 V.
* The Y-axis for Chart b is logarithmic, highlighting the large changes in electric field.
### Interpretation
These charts demonstrate the differing thermal and electrical responses of the CMO layer in Low Resistance State (LRS) and High Resistance State (HRS) as a function of applied voltage. The temperature dip in LRS (Chart a) suggests a phase change or energy dissipation mechanism occurring at low voltages. The rapid increase in electric field in HRS (Chart b) around 0.0 V indicates a breakdown or switching event.
The contrasting behaviors suggest that the LRS and HRS represent distinct physical states of the CMO layer, likely related to the material's resistive switching properties. The data suggests that applying a voltage near 0.0 V can induce a transition between these states, as evidenced by the sharp changes in both temperature and electric field. The logarithmic scale on Chart b emphasizes the substantial electric field changes associated with the HRS, indicating a significant change in the material's conductivity. The divergence at 0.9V could indicate a saturation point or a limit to the material's response.
</details>
Figure S4: The voltage-dependent evolution of a the average temperature and b electric field within a 3D half-spherical volume of the CMO layer situated atop the conductive filament in both HRS and LRS is presented. These trends serve as inputs for equation (6) of the manuscript.
<details>
<summary>x11.png Details</summary>
### Visual Description
\n
## Heatmap & Flowchart: CMO-HfOx ReRAM Programming & Closed-Loop Scheme
### Overview
The image presents two distinct visualizations. The left side (a) is a heatmap illustrating the cumulative distribution function of CMO-HfOx ReRAM during programming, plotted against target conductance. The right side (b) is a flowchart detailing the closed-loop scheme used for programming the ReRAM.
### Components/Axes
**Heatmap (a):**
* **Title:** CMO-HfOx ReRAM during programming
* **X-axis:** Target Conductance [”S] (Scale: 0 to 90, with markers at 10, 20, 30, 40, 50, 60, 70, 80)
* **Y-axis:** Cumulative Distribution Function (Scale: 0.00 to 1.00, with markers at 0.00, 0.25, 0.50, 0.75, 1.00)
* **Color Scale:** Ranges from 1 (blue) to 35 (red). The color scale is positioned vertically on the right side of the heatmap.
* **Annotation:** "Acceptance Range: 2% Gtarget" is placed in the top-left region of the heatmap.
**Flowchart (b):**
* **Title:** Flowchart of the closed-loop scheme
* **Nodes:** "Calculate acceptance range (AR) based on Gtarget", "Gtarget â [10,30]”S", "Gtarget â [30,60]”S", "Gtarget â [60,90]”S", "Apply SET pulse (Vset)", "Measure G", "Apply RESET pulse (Vreset)", "Write Succeeds".
* **Decision Points:** G < Gtarget - AR, G > Gtarget + AR, G â (Gtarget ± AR)
* **Voltage Values:** Vset = 1.35V (appears three times), Vreset = -1.5V (appears twice), Vreset = -1.3V (appears once).
* **Arrows:** Indicate the flow of the scheme.
### Detailed Analysis or Content Details
**Heatmap (a):**
The heatmap shows a generally increasing trend in cumulative distribution function with increasing target conductance. The color transitions from blue (low cumulative distribution) to red (high cumulative distribution).
* At a Target Conductance of 10 ”S, the Cumulative Distribution Function is approximately 0.05.
* At a Target Conductance of 20 ”S, the Cumulative Distribution Function is approximately 0.20.
* At a Target Conductance of 30 ”S, the Cumulative Distribution Function is approximately 0.35.
* At a Target Conductance of 40 ”S, the Cumulative Distribution Function is approximately 0.50.
* At a Target Conductance of 50 ”S, the Cumulative Distribution Function is approximately 0.60.
* At a Target Conductance of 60 ”S, the Cumulative Distribution Function is approximately 0.70.
* At a Target Conductance of 70 ”S, the Cumulative Distribution Function is approximately 0.80.
* At a Target Conductance of 80 ”S, the Cumulative Distribution Function is approximately 0.90.
* At a Target Conductance of 90 ”S, the Cumulative Distribution Function is approximately 0.95.
**Flowchart (b):**
The flowchart describes a closed-loop programming scheme.
1. The process begins with calculating an acceptance range (AR) based on the target conductance (Gtarget).
2. Based on the Gtarget value, different voltage values are applied.
* If Gtarget is between 10 and 30 ”S, Vset = 1.35V and Vreset = -1.5V.
* If Gtarget is between 30 and 60 ”S, Vset = 1.35V and Vreset = -1.3V.
* If Gtarget is between 60 and 90 ”S, Vset = 1.5V and Vreset = -1.3V.
3. A SET pulse (Vset) is applied.
4. The conductance (G) is measured.
5. Decision points:
* If G < Gtarget - AR, a RESET pulse (Vreset) is applied.
* If G > Gtarget + AR, a SET pulse (Vset) is applied.
* If G is within the acceptance range (G â (Gtarget ± AR)), the write succeeds.
### Key Observations
* The heatmap shows a relatively smooth increase in the cumulative distribution function with increasing target conductance, suggesting a consistent programming behavior.
* The acceptance range annotation on the heatmap indicates a tolerance of 2% around the target conductance.
* The flowchart demonstrates a feedback mechanism where the conductance is measured after each pulse and the next pulse is determined based on the measured value.
* The voltage values for SET and RESET pulses vary depending on the target conductance range.
### Interpretation
The data suggests a controlled programming process for CMO-HfOx ReRAM. The heatmap provides insight into the distribution of programmed states for different target conductances, indicating the reliability and uniformity of the programming process. The closed-loop scheme, as depicted in the flowchart, aims to precisely control the conductance of the ReRAM by iteratively applying SET and RESET pulses and adjusting the voltage based on the measured conductance. The varying voltage values for different conductance ranges suggest an optimization strategy to achieve accurate and efficient programming. The acceptance range indicates a tolerance level for the programmed conductance, allowing for some variation while still achieving the desired functionality. The flowchart's iterative nature highlights the importance of feedback in achieving precise control over the ReRAM's resistive state.
</details>
Figure S5: a The experimental cumulative distribution of conductance values for 35 representative programmed levels using 2% of G target as acceptance range. The closed-loop scheme based on identical pulses shown in Fig. 3 b of the manuscript and detailed in Methods is used. b Flowchart illustrating the identical-pulse closed-loop technique used for programming the ReRAM array into target analog conductance range.
<details>
<summary>x12.png Details</summary>
### Visual Description
\n
## Charts: I/O Quantization and Scaling Performance
### Overview
The image presents two charts (a and b) comparing the Root Mean Squared Error (RMSE) against the logarithm of time for different I/O configurations. Chart 'a' focuses on a 64x64 configuration with varying I/O quantization levels and IR drop considerations. Chart 'b' demonstrates scaling performance up to a 512x512 configuration. Both charts use a logarithmic scale for both the x and y axes.
### Components/Axes
**Common Elements:**
* **X-axis:** Log(Time[s]) - Logarithmic scale representing time in seconds. Ranges from approximately 0 to 20.
* **Y-axis:** RMSE - Root Mean Squared Error, representing the error magnitude. Logarithmic scale ranging from 10<sup>-2</sup> to 10<sup>0</sup>.
* **Annotation:** "10y" - Appears in the top-right corner of both charts, likely indicating a time scale or a reference point.
* **Annotation:** "Prog." - Appears in both charts, likely indicating a progress marker or a point of interest.
**Chart a (64x64: I/O quantization and IR drop):**
* **Title:** "64x64: I/O quantization and IR drop"
* **Legend:** Located in the top-left corner.
* Black Circle: "64x64: IRdrop, 6/8bit I/O (Manuscript)"
* Gray Square: "64x64: IRdrop, 32/32bit I/O"
* Orange Circle: "64x64: NO_IRdrop, 32/32bit I/O"
**Chart b (Scaling up to 512x512):**
* **Title:** "Scaling up to 512x512"
* **Legend:** Located in the top-left corner.
* Green Diamond: "512x512: IRdrop, 6/8bit I/O"
* Green Cross: "64x64: IRdrop, 6/8bit I/O (Manuscript)"
### Detailed Analysis or Content Details
**Chart a (64x64):**
* **Black Circle (IRdrop, 6/8bit I/O):** The line starts at approximately RMSE = 0.02 (2x10<sup>-2</sup>) at Log(Time) = 0, remains relatively stable until Log(Time) â 10, then increases to approximately RMSE = 0.1 (1x10<sup>-1</sup>) at Log(Time) = 20.
* **Gray Square (IRdrop, 32/32bit I/O):** The line starts at approximately RMSE = 0.03 (3x10<sup>-2</sup>) at Log(Time) = 0, increases steadily to approximately RMSE = 0.12 (1.2x10<sup>-1</sup>) at Log(Time) = 10, and continues to increase to approximately RMSE = 0.2 (2x10<sup>-1</sup>) at Log(Time) = 20.
* **Orange Circle (NO_IRdrop, 32/32bit I/O):** The line starts at approximately RMSE = 0.01 (1x10<sup>-2</sup>) at Log(Time) = 0, increases to approximately RMSE = 0.03 (3x10<sup>-2</sup>) at Log(Time) = 10, and reaches approximately RMSE = 0.08 (8x10<sup>-2</sup>) at Log(Time) = 20.
**Chart b (Scaling up to 512x512):**
* **Green Diamond (512x512: IRdrop, 6/8bit I/O):** The line starts at approximately RMSE = 0.03 (3x10<sup>-2</sup>) at Log(Time) = 0, increases to approximately RMSE = 0.08 (8x10<sup>-2</sup>) at Log(Time) = 10, and reaches approximately RMSE = 0.15 (1.5x10<sup>-1</sup>) at Log(Time) = 20.
* **Green Cross (64x64: IRdrop, 6/8bit I/O):** The line starts at approximately RMSE = 0.02 (2x10<sup>-2</sup>) at Log(Time) = 0, increases to approximately RMSE = 0.05 (5x10<sup>-2</sup>) at Log(Time) = 10, and reaches approximately RMSE = 0.1 (1x10<sup>-1</sup>) at Log(Time) = 20.
### Key Observations
* In Chart a, the 32/32bit I/O configurations (both with and without IR drop) exhibit higher RMSE values compared to the 6/8bit I/O configuration. The configuration *without* IR drop performs best.
* In Chart b, the 512x512 configuration shows a higher RMSE than the 64x64 configuration across all time scales, indicating a performance degradation with scaling.
* Both charts show an increasing RMSE with increasing Log(Time), suggesting that the error accumulates over time.
* The "Prog." annotations appear at similar RMSE values in both charts, potentially indicating a performance threshold or a point of interest in the error behavior.
### Interpretation
The data suggests that I/O quantization and IR drop significantly impact the accuracy (as measured by RMSE) of the system. Lower bit-width I/O (6/8bit) generally results in lower RMSE values, especially when IR drop is considered. The absence of IR drop further improves performance.
Scaling the system from 64x64 to 512x512 introduces increased RMSE, indicating that the performance benefits of scaling are offset by increased error. This could be due to increased complexity, signal interference, or other factors associated with larger system sizes.
The consistent upward trend of RMSE with time suggests that the error is not static but accumulates over time, potentially due to drift or other time-dependent effects. The "Prog." annotations might represent a point where the error reaches an unacceptable level, triggering a corrective action or indicating a system limitation.
The charts provide valuable insights into the trade-offs between I/O configuration, system size, and accuracy. Optimizing I/O quantization and mitigating IR drop are crucial for maintaining low error rates, while scaling requires careful consideration of potential performance degradation.
</details>
Figure S6: The individual impact of IR-drop across array wires and input/output bit quantization on MVM accuracy. a Simulated RMSE compared to FP ideal results using 64x64 analog CMO/HfOx ReRAM array, shown as a function of the time after programming. Dashed horizontal lines represent the RMSE during programming, considering programming noise (with 0.2% G target as the acceptance range) but excluding relaxation effects. With 32-bit input/output quantization and no IR-drop (orange dots), an RMSE as low as 6 $10^{-3}$ is achieved during programming, which immediately increases (see the arrow) after relaxation (within $\mathrm{1\,s}$ ). Including the realistic IR-drop results in an overall RMSE increase (blue squares). Finally, reducing input/output quantization to 6/8 bits, respectively, leads to a further accuracy loss (green crosses), demonstrating that at short timescales (within 1 hour), the main analog MVM accuracy bottleneck is the reduced input/output quantization. After 1 hour, all cases converge, showing that the accuracy bottleneck is then dominated by the relaxation process. b By scaling up to a 512x512 array size (grey diamonds) and considering input/output quantization of 6/8 bits, IR-drop emerges as the primary bottleneck for analog MVM accuracy.
<details>
<summary>x13.png Details</summary>
### Visual Description
\n
## Chart: Open-loop pulsed programming of the CMO-HfOx ReRAM array
### Overview
The image presents a 4x5 grid of charts, each depicting the conductance (in ”S) of a ReRAM device as a function of pulse number. Each chart shows three curves representing different pulse parameters: Gmin (blue), Gmax (red), and Gsp (green). The title indicates the data relates to open-loop pulsed programming of a CMO-HfOx ReRAM array.
### Components/Axes
* **Title:** Open-loop pulsed programming of the CMO-HfOx ReRAM array (top-center)
* **X-axis Label:** Pulse Number (appears on all charts, ranging from 0 to 2100)
* **Y-axis Label:** Conductance [”S] (appears on all charts, ranging from 0 to 10)
* **Legend:** Located in the top-left corner of each chart, with the following labels and corresponding colors:
* Gmin (Blue, dashed line)
* Gmax (Red, dashed line)
* Gsp (Green, dashed line)
* **Grid:** Each chart has a grid with vertical lines at intervals of 400 along the x-axis.
* **Chart Arrangement:** 4 rows and 5 columns.
### Detailed Analysis or Content Details
Each of the 20 individual charts displays similar trends, but with varying magnitudes and slight differences in curve behavior. Here's a breakdown of the general trends observed, followed by approximate data points extracted from a representative chart (the one in the top-left corner).
**General Trends:**
* **Gmin (Blue):** Generally remains relatively stable at a low conductance value throughout the pulse sequence. There is a slight initial increase, followed by stabilization.
* **Gmax (Red):** Starts at a higher conductance than Gmin and exhibits a more pronounced initial decrease, followed by a period of fluctuation and eventual stabilization at a lower conductance level.
* **Gsp (Green):** Shows a more dynamic behavior, with initial fluctuations and a tendency to settle between the conductance levels of Gmin and Gmax.
**Data Points (Top-Left Chart - Approximate):**
* **Gmin (Blue):**
* Pulse 0: ~0.6 ”S
* Pulse 400: ~0.8 ”S
* Pulse 800: ~0.8 ”S
* Pulse 1200: ~0.8 ”S
* Pulse 1600: ~0.8 ”S
* Pulse 2000: ~0.8 ”S
* **Gmax (Red):**
* Pulse 0: ~3.2 ”S
* Pulse 400: ~2.0 ”S
* Pulse 800: ~1.2 ”S
* Pulse 1200: ~1.0 ”S
* Pulse 1600: ~1.0 ”S
* Pulse 2000: ~1.0 ”S
* **Gsp (Green):**
* Pulse 0: ~1.6 ”S
* Pulse 400: ~1.2 ”S
* Pulse 800: ~1.0 ”S
* Pulse 1200: ~0.8 ”S
* Pulse 1600: ~0.8 ”S
* Pulse 2000: ~0.8 ”S
**Variations Across Charts:**
The initial conductance values for all three curves (Gmin, Gmax, Gsp) vary slightly across the 20 charts. Some charts show a more pronounced decrease in Gmax, while others exhibit more stable curves. The settling point of Gsp also varies, sometimes closer to Gmin and sometimes closer to Gmax.
### Key Observations
* The Gmin curve consistently represents the lowest conductance state.
* The Gmax curve consistently represents the highest initial conductance state, but decreases significantly with pulse application.
* The Gsp curve appears to mediate between Gmin and Gmax, potentially representing a switching behavior.
* There is a clear trend of conductance change with pulse number, indicating the ReRAM devices are being programmed.
* The variations between the charts suggest device-to-device variability in the ReRAM array.
### Interpretation
The data demonstrates the open-loop pulsed programming behavior of a CMO-HfOx ReRAM array. The three curves (Gmin, Gmax, Gsp) likely represent different programming conditions or measurement points during the pulse sequence. Gmax represents the initial high resistance state, which is driven to a lower resistance state with each pulse. Gmin represents the low resistance state. Gsp is likely a measurement of the conductance during the pulse, or a specific point after the pulse.
The consistent decrease in Gmax suggests that the ReRAM devices are being switched from a high resistance state to a low resistance state. The variations between the charts indicate that the switching process is not uniform across the array, likely due to inherent device-to-device variability in the material properties or fabrication process. The stabilization of the curves at higher pulse numbers suggests that the devices are reaching a stable resistance state.
The data suggests that the CMO-HfOx ReRAM array is capable of being programmed using pulsed voltage, but further optimization may be needed to improve the uniformity of the switching process across the array. The observed device-to-device variability could be a limiting factor for the performance and reliability of the ReRAM array.
</details>
Figure S7: The experimental response of the 8x4 CMO/HfO x ReRAM devices within the array to the open-loop programming pulse scheme (shown in Fig. 5 b of the manuscript) is shown. The set and reset pulse amplitudes are $1.35\,\mathrm{V}$ ( $V_{\mathrm{G}}=\mathrm{1.4\,V}$ ) and $-1.3\,\mathrm{V}$ ( $V_{\mathrm{G}}=\mathrm{3.3\,V}$ ), respectively, with a constant width of 2.5 ”s due to setup limitations.
<details>
<summary>x14.png Details</summary>
### Visual Description
## Line Chart: Generalized SBM vs SBM
### Overview
The image presents a line chart comparing "Generalized SBM" and "SBM" against "Experimental Data" as a function of "Pulse Number". The y-axis represents "Normalized G". The chart displays three distinct curves, each representing one of the aforementioned data sets.
### Components/Axes
* **Title:** "Generalized SBM vs SBM" (centered at the top)
* **X-axis:** "Pulse Number" (ranging from approximately 0 to 2100)
* **Y-axis:** "Normalized G" (ranging from approximately -1.5 to 1.2)
* **Legend:** Located in the top-right corner.
* "Exp. data" (represented by red circles)
* "Gen SBM" (represented by a yellow line)
* "SBM" (represented by a black line)
* **Gridlines:** Light gray vertical gridlines are present throughout the chart.
### Detailed Analysis
The chart shows three curves plotted against Pulse Number.
* **Exp. data (Red Circles):** The data points are scattered, but generally follow a pattern of rapid increase from approximately 0 to 1 around Pulse Number 200, a rapid decrease to approximately -1 around Pulse Number 800, another increase to 1 around Pulse Number 1600, and then a leveling off around 0. The data points are most densely clustered around the transitions between these states.
* Around Pulse Number 200: Normalized G is approximately 1.
* Around Pulse Number 800: Normalized G is approximately -1.
* Around Pulse Number 1600: Normalized G is approximately 1.
* Around Pulse Number 2100: Normalized G is approximately 0.
* **Gen SBM (Yellow Line):** This line exhibits a step-like function. It remains at approximately 1 until Pulse Number 200, then drops sharply to approximately -1 around Pulse Number 800, rises sharply to approximately 1 around Pulse Number 1600, and then remains relatively stable around 0.
* From 0 to 200: Normalized G is approximately 1.
* From 200 to 800: Normalized G drops from 1 to -1.
* From 800 to 1600: Normalized G rises from -1 to 1.
* From 1600 to 2100: Normalized G is approximately 0.
* **SBM (Black Line):** This line also exhibits a step-like function, similar to "Gen SBM", but with a slightly different shape. It remains at approximately 1 until Pulse Number 200, then drops sharply to approximately -1 around Pulse Number 800, rises sharply to approximately 1 around Pulse Number 1600, and then remains relatively stable around 0.
* From 0 to 200: Normalized G is approximately 1.
* From 200 to 800: Normalized G drops from 1 to -1.
* From 800 to 1600: Normalized G rises from -1 to 1.
* From 1600 to 2100: Normalized G is approximately 0.
### Key Observations
* The "Gen SBM" and "SBM" lines are very similar in shape, suggesting that the "Generalized SBM" is a close approximation of the "SBM".
* The "Exp. data" shows more variability and a smoother transition between states compared to the step-like behavior of the models.
* The experimental data appears to lag slightly behind the model predictions, particularly during the transitions.
* The chart shows a repeating pattern, suggesting a cyclical behavior related to the "Pulse Number".
### Interpretation
The chart compares a model ("SBM" and "Generalized SBM") to experimental data ("Exp. data"). The "Normalized G" value appears to represent a state variable that oscillates between approximately -1 and 1 in response to the "Pulse Number". The models attempt to capture this behavior, and the chart demonstrates how well they align with the experimental observations. The differences between the models and the experimental data suggest that the models may not fully capture the complexity of the underlying system. The smoother transitions in the experimental data indicate that the state changes are not instantaneous, as assumed by the step-like models. The repeating pattern suggests that the system is driven by a periodic input (the "Pulse"). The slight lag in the experimental data could be due to measurement delays or inherent system response times. The chart is a validation of the model against real-world data, and highlights areas where the model could be improved.
</details>
Figure S8: The experimental open-loop pulsed response of a representative CMO/HfO x ReRAM device within the array shows that the potentiation and depression characteristics do not inherently saturate at the upper and lower boundaries. The generalized soft bounds model (yellow line) better captures this experimental trend compared to the saturated soft bounds model (black line).
<details>
<summary>x15.png Details</summary>
### Visual Description
## Scatter Plots: G1 and G2
### Overview
The image presents two scatter plots, labeled G1 and G2, comparing "Exp. array data" (Experimental array data) and "Gen. SBM" (Generated Stochastic Block Model) data. Both plots visualize the relationship between two variables, with the experimental data represented by black circles and the generated data by yellow diamonds.
### Components/Axes
**Plot G1:**
* **Title:** G1 (top-left)
* **X-axis:** Nstates (logarithmic scale from approximately 8 to 100)
* **Y-axis:** up\_down (scale from -1.0 to 1.0)
* **Legend:** Located in the top-right corner.
* Black circles: Exp. array data
* Yellow diamonds: Gen. SBM
**Plot G2:**
* **Title:** G2 (top-left)
* **X-axis:** Îł (scale from 0.0 to 3.0)
* **Y-axis:** Îłup\_down (scale from -0.75 to 0.75)
* **Legend:** Located in the top-right corner.
* Black circles: Exp. array data
* Yellow diamonds: Gen. SBM
### Detailed Analysis or Content Details
**Plot G1:**
The black circles (Exp. array data) are clustered in a relatively tight group around Nstates â 20-40, with up\_down values ranging from approximately -0.6 to 0.2. The yellow diamonds (Gen. SBM) are more dispersed, covering a wider range of Nstates (8 to 100) and up\_down values (-0.8 to 0.6). The Gen. SBM data appears to have a slight upward trend as Nstates increases.
* **Approximate Data Points (Exp. array data):**
* (10, -0.2)
* (20, 0.0)
* (30, 0.1)
* (40, -0.1)
* (50, -0.3)
* **Approximate Data Points (Gen. SBM):**
* (10, 0.4)
* (20, 0.2)
* (30, -0.1)
* (40, -0.3)
* (50, -0.5)
* (80, 0.3)
* (100, 0.6)
**Plot G2:**
The black circles (Exp. array data) are concentrated around Îł â 1.5, with Îłup\_down values ranging from approximately -0.3 to 0.3. The yellow diamonds (Gen. SBM) are also somewhat clustered around Îł â 1.5, but are more spread out, with Îłup\_down values ranging from approximately -0.7 to 0.6. The Gen. SBM data shows a slight positive correlation between Îł and Îłup\_down.
* **Approximate Data Points (Exp. array data):**
* (0.5, 0.1)
* (1.0, -0.1)
* (1.5, 0.0)
* (2.0, -0.2)
* (2.5, 0.2)
* **Approximate Data Points (Gen. SBM):**
* (0.5, -0.5)
* (1.0, 0.2)
* (1.5, 0.4)
* (2.0, 0.1)
* (2.5, -0.3)
* (3.0, 0.6)
### Key Observations
* In both plots, the experimental data (black circles) is more tightly clustered than the generated data (yellow diamonds).
* In G1, the generated data shows a slight upward trend with increasing Nstates.
* In G2, both datasets appear centered around Îł = 1.5, but the generated data has a wider spread.
* The scales of the axes are different between the two plots.
### Interpretation
These plots likely represent a validation of a stochastic block model (SBM) against experimental data. The SBM is a statistical model used to represent networks with community structure. The plots compare the distribution of certain network properties (up\_down and Îłup\_down) in the experimental data to those generated by the SBM.
The tighter clustering of the experimental data suggests that the real-world network has more constrained properties than the SBM allows for. The differences in distribution between the experimental and generated data indicate that the SBM may not perfectly capture the characteristics of the real-world network. The trends observed in the generated data (e.g., the upward trend in G1) could be artifacts of the model or reflect underlying relationships in the data that are not fully understood. The fact that the generated data covers a wider range of values suggests that the SBM is capable of producing a variety of network structures, but may not be accurately reflecting the specific structure of the experimental data. Further analysis would be needed to determine the extent to which the SBM is a valid representation of the real-world network.
</details>
Figure S9: Multi-variate Gaussian distributions to reproduce the experimental inter-device variability. a Multi-variate gaussian G1 distribution of the experimental number of states and device asymmetry ( $up\_down$ ). b Gaussian G2 distribution of the analytical parameters $\gamma$ and $\gamma_{\rm up\_down}$ extracted from the generalized soft bounds model fitting to the experimental traces.
### Device modelling
#### $up\_down$ parameter
The $up\_down$ parameter is defined for the generalized soft bounds model in the simulation environment of the âaihwkitâ as the directional bias between the up and down update size ( $\Delta G^{+}$ and $\Delta G^{-}$ ). In addition, the minimum step in each direction d is described by the following expression [44].
$$
\displaystyle\Delta G^{d}=\Delta G_{SP}(1+d\beta+\sigma_{d-to-d}) \tag{7}
$$
where d is -1 or 1 depending on the update direction. In contrast, the symmetry point is defined for each device as follows [23]:
$$
\displaystyle SP=[\Delta G^{+}-\Delta G^{-}]/[\Delta G^{+}/(b_{\rm max}-\Delta
G
^{+}/b_{\rm min})] \tag{8}
$$
Where $\Delta G^{+}$ , $\Delta G^{-}$ define the minimum step size in the up and down direction respectively, and $b_{\rm max}$ and $b_{\rm min}$ represent the upper and lower bounds of the conductance. Therefore, considering an independent definition of each device (i.e. zero d-to-d variability) and a normalized conductance range between -1 and 1, the symmetry point device-level characteristic and the $up\_down$ analytical parameter are equivalent.
#### Training setup
For result replicability, the experimental parameters are incorporated into the simulation environment, where the Noise-to-Signal Ratio (NSR) is represented by âdw_min_stdâ, normalized SP by âup_downâ, normalized maximum and minimum conductances by âw_maxâ and âw_minâ and min conductance step by âdw_minâ. From this device model, analog training simulations were performed using AGAD considering a learning rate to update the weights of 1e-2, âfast_lrâ of 0.1 to update matrix, âtransfer_everyâ 3 iterations and batch size of 32. The FP baseline was obtained with SGD training using a learning rate of 1e-3 and batch size of 32.