A CMOS LIF neuron based on a charge-powered oscillator with

time-domain threshold logic

This paper was downloaded from TechRxiv (https://www.techrxiv.org).

LICENSE

CC BY 4.0

SUBMISSION DATE / POSTED DATE

08-02-2023 / 17-02-2023

CITATION

Granizo Cuadrado, Javier; Garvi, Ruben; Garcia, Diego; Hernandez Corporales, Luis (2023): A CMOS LIF
neuron based on a charge-powered oscillator with time-domain threshold logic. TechRxiv. Preprint.
https://doi.org/10.36227/techrxiv.22047044.v1

DOI

10.36227/techrxiv.22047044.v1
 A CMOS LIF neuron based on a charge-powered
oscillator with time-domain threshold logic
Javier Granizo Ruben Garvi Diego Garcia Luis Hernandez
Electronic Tech. Dept. Electronic Tech. Dept. Electronic Tech. Dept. Electronic Tech. Dept.
Carlos III University Carlos III University Carlos III University Carlos III University
Leganes, Spain Leganes, Spain Leganes, Spain Leganes, Spain
[email protected] [email protected] [email protected] [email protected]

Abstract—This paper introduces a new CMOS implementation

of a spiking Leaky Integrate-and-Fire (LIF) neuron. The circuit
does not require any analog circuit block such as comparators
or current sources thanks to the use of a ring-oscillator as an
integrator and a phase delay detector as threshold logic. The
circuit admits both excitatory and inhibitory input spiking signals
whose pulse width does not affect the computation. Instead of
powering the ring oscillators from a power supply, the input
spikes charge a capacitor bank which powers the ring oscillator.
The paper describes the operation of the neuron analytically
for the input integration and the membrane voltage decay. Also,
a circuit-level simulation in 65nm CMOS technology has been
performed, achieving a power efficiency of <40fJ/spike.
Index Terms—Spiking neural network, Integrate-And-Fire
neuron, Ring Oscillators, Time Encoding
Fig. 1. Analog system-level implementation of a spiking LIF neuron [3]
I. I NTRODUCTION
Hardware implementation of Spiking Neural Networks
(SNN) is a rapidly evolving field whose foundations were Synapse coefficients can be programmed by either different
laid out several decades ago [1]. Several SNN architectures current source sizes or by modulating the pulse width of the
seize the simplicity and processing power of Integrate-and- input pulses. This neuron has several implementation problems
Fire (IF) neuron models to pack, in a small neuron population, related to the analog nature of the circuitry. On one hand, if
computations that would require a more power-demanding the coefficient information is encoded in the nominal currents
Deep Neural Network (DNN) [1]. Integrate-and-Fire neurons of a current source array, process variability and temperature
have been implemented in hardware with simple analog cir- may affect the absolute value of the coefficients. On the other
cuits [2]. However, their operation is based in most cases on hand, if coefficient encoding is performed in the pulse width of
programmable current sources or pulse width modulated ex- the spike signals, the required time accuracy forces a circuitry
citation signals [3]. Recently, VCO-based IF hardware neuron bandwidth in excess of the average firing rate, which translates
models have been proposed, with the benefit of digital ring- into extra power. Also, the voltage comparator in Fig. 1 is
oscillator simplicity and low voltage operation [4]–[6]. In this limited by offset and voltage reference stability.
paper, we propose a new IF neuron circuit that can reduce As a difference to Fig. 1, our proposed circuit does not
the power and supply voltage of existing designs. This is use current mirrors, using instead several small unit capac-
accomplished by moving the integration into the time-domain itors whose charge and discharge power a ring oscillator,
and transforming the voltage threshold required in the analog transforming the analog-domain threshold logic to a time-
IF model into a time delay threshold using the ring oscillator domain equivalent. This principle of evaluating the charge in
circuit. a capacitor by counting the pulses of a ring oscillator was
Figure 1 represents a basic IF neuron circuit with a lossy introduced in [7] to measure a capacitive sensor. With this
integrator (LIF neuron). The excitatory inputs E1 and E2 operating principle, supply voltages for the integrating core
and inhibitory input I1 charge and discharge respectively can be on the order of the MOS threshold voltage, increasing
capacitor Cmem . When the membrane potential Vmem reaches the power efficiency.
threshold voltage Vth , the neuron fires producing a pulse The paper describes in Section II the operating principle of
at the output whose width is controlled by a monostable. the neuron and its governing equations, as well as the threshold
This research was funded by project PID2020-118804RB-I00 of the Spanish logic implementation. Section III shows circuit simulations of
Agency of Research (AEI) a proof-of-concept in TSMC 65nm technology. Finally, section
 Fig. 2. Transistor-level schematic of the proposed time-domain neuron.

IV concludes the paper. to solve this issue is to drive the integrating circuit like in
an asynchronous switched capacitor circuit, where the charge
II. O PERATING PRINCIPLE OF THE PROPOSED between capacitors is transferred under the spike command.
TIME - DOMAIN NEURON
Instead of using a current mirror, this paper uses a
Figure 2 shows the proposed architecture of the time- capacitive-loaded inverter as the input driver, which injects
domain neuron. The input spike trains are integrated into a a charge into the membrane voltage proportional to its output
common membrane potential by means of several inverters capacitor and voltage difference on the inverter (see Fig. 2).
loaded by capacitors. The sum of all capacitors connected The unit capacitors at each inverter are connected in parallel
to the membrane voltage node supplies a single-ended ring when no spikes are applied to the inputs. As such, the big
oscillator, which actuates both as a time-varying resistor and capacitor used in the current mirror integration approaches
as a charge-controlled oscillator. Comparing the delay between is substituted by several smaller ones. When a spike, either
two consecutive phases with respect to a programmable delay, inhibitory or excitatory, reaches the input inverter, one of the
the digital logic detects when the oscillator has exceeded unit capacitors quickly commutes to VDD or GND, receiving a
a frequency threshold. This comparison is time-based and fixed charge independent of the input pulse width. Afterward,
not voltage-based, as such, there is no need for the voltage the charge is transferred to the rest of the unit capacitors,
comparator used typically in IF mixed-signal neurons. Thus, giving a voltage increase (or decrease) which depends on the
the use of the oscillator can be promising in terms of power membrane potential Vmem . Equations (1) and (2) describe
and supply voltage reduction. Once the oscillation frequency the evolution of the membrane voltage for excitatory and
surpasses the threshold, an output spike is generated, and the inhibitory inputs respectively:
membrane voltage is reset to a low-level state.
The neuron can be analytically described in three parts: first,
Ci
the input integration and weighting block using capacitive- Vmem (t+ ) = Vmem (t− ) + (VDD − Vmem (t− )) Pn (1)
loaded inverters; next, the ring oscillator implementing the i=1 Ci
charge loss due to its power consumption; and lastly, a digital
logic time-domain threshold detector. Ci
Vmem (t+ ) = Vmem (t− ) − Vmem (t− ) Pn (2)
i=1 Ci
A. Spike integration and weighting
Usually, IF neurons based on mixed-signal circuits use In these equations, t− and t+ are the instants before and after
switched current mirrors in order to integrate the spikes and the input spike reaches a given inverter and Ci is the value of
multiply them by a weight. This approach is highly dependent the capacitor connected to the i-th input. Note that, in order to
on the width of the input pulse, as it determines the time ensure that all capacitors are connected to the same node when
window in which the current source supplies the integrating no input is present, excitatory spikes are low-level driven and
capacitor. When implemented in an SNN, a single output inhibitory spikes are high-level driven. Charger injection can
can be seen as slightly different inputs in the next layer be modified by either having programmable capacitors at each
due to small changes in the width of the signal. One way input or, having a fixed number of input inverters, connecting
 Fig. 4. Time-domain threshold logic digital signal diagram.

equivalent capacitor made by all capacitive-loaded inverters,

the resulting differential equation that governs the decay rate
can be given by the Bernoulli equation (5):
Fig. 3. Typical LIF neuron decay rate (3) vs our implementation (6) n
X dvmem (t) 2
Ci = −N KV CO Cinv vmem (t) (5)
i=1
dt
more than one inverter to the same input, thus triggering more
than one inverter at the same time. The solution of this equation is displayed at (6):

B. Charge-powered oscillator leakage Vo

vmem (t) = (6)
1 + Vo N Kvco PCninvCi t
In previous state-of-the-art mixed-signal neurons, membrane i=1

voltage leakage has been modeled by using a programmable Figure 3 compares the charge-powered oscillator decaying
resistor in parallel with the integrating capacitor, following the rate with respect to the standard implementation using a
equation: resistor. In section III a simulation of the circuit of Fig. 2 is
−t shown. For the parameters chosen in section III, the fractional
vmem (t) = V0 e τ (3) decaying rate shows a similar decaying rate to the exponential
using a 9.3 MΩ equivalent resistor. Furthermore, changing the
where τ is the time constant of the neuron and V0 is the initial
output capacitance of the inverters it is possible to modify the
value of the membrane voltage, which decays exponentially. In
decaying rate correspondingly.
our design, we need to establish an equation for the equivalent
resistance of an N-tap ring oscillator to model the leakage C. Time-domain threshold logic
current. For low voltage level biasing, the current driving a
ring oscillator can be divided into two types: the short current Figure 4 shows the digital function implemented in order
and the input-driving current. The short current goes from to detect the membrane voltage threshold. Note that signals
the positive to the negative terminal of each inverter when displayed in Fig 4 have their corresponding nodes in Fig. 2.
it changes logic states, due to the short window in which both As seen in the diagram, two of the phases of the ring oscillator
transistors are conducting current. The input driving current are brought out in order to check the delay between them
charges the input gate capacitor of each inverter when it’s in comparison with the threshold delay. On the left side of
changing logic states. If a capacitor Cinv is placed at the the diagram, the delay is higher than the threshold, as such,
input of each inverter in the ring oscillator of Fig. 2, the no spike pulses are produced. However, when looking on
input driving current becomes much higher than the short the right, the oscillation frequency is slightly higher and the
current. As such, an approximation can be made for the current delay between the phases is shorter, producing then a spike.
consumption of the ring oscillator taking only the input driver As the oscillation frequency is proportional to the voltage
current component: membrane vmem , the oscillator will integrate input spikes
until the membrane voltage is sufficiently high, and thus, the
iro (t) = N fo (t)Cinv vmem (t) (4) oscillation frequency is high enough to produce a spike.
The threshold delay can be programmed using several
Taking inspiration from VCO-ADC related works [8], fo circuits, such as a voltage-controlled delay line based on
is proportional to vmem times the oscillator gain (KV CO ). inverters. Furthermore, to ensure that all spike pulses are
For the sake of simplifying the resulting decaying term, and the same width, a rising-edge trigger circuit was also imple-
taking into account that the oscillator does not have a wide mented. Modifying this last stage will allow having a variable
tuning range in this application, we will consider KV CO to refractory period. Nevertheless, variable width in the output
be constant in the desired operating range. As the current spike pulse would not be necessary when using this cell in
consumed by the oscillator is equal to the discharge of the the following layers, as the inputs are not width-dependent.
 Fig. 6. Output spike frequency and current consumption with respect to input
spike frequency.

Fig. 5. Transient simulation of the membrane voltage and resulting output

TABLE I
spikes, using both excitatory and inhibitory inputs.
C OMPARISON WITH OTHER ACCELERATED LIF NEURONS

Reference This work [3] [9] [10] [11]

III. S IMULATIONS Process 65nm 50nm 65nm 28nm 32nm
Operating voltage 0.5V 0.6V 1V 0.2V 2.8V
Threshold Logic Time Voltage Voltage Voltage Voltage
The purposed architecture was implemented using the Energy/spike 39.8 fJ 54fJ 2pJ 1.2fJ 35pJ
TSMC 65nm process in Cadence Virtuoso. The power-supply
voltage is 0.5V, which is sufficient to trigger low-voltage
Figure 6 displays the relationship between input spike
threshold transistors with minimal length in this process node.
frequency and output spike frequency using only one exci-
The input integration is composed of four capacitive-loaded
tatory input. It can be appreciated that the input and output
inverters, two for excitatory signals and two for inhibitory. The
frequencies of the spikes increase linearly, and current scales
parameters of the inverters are 6µm/60nm for the NMOS,
with respect to the output frequency. For the given input
8µm/60nm for the PMOS, and 5fF per unit capacitor. Another
frequency range, the minimum energy per output spike is
capacitor of 40fF was placed in the membrane voltage node to
39.8fJ/spike, also considering the input currents integrated into
ensure that the ring oscillator always has a minimum capacitor
vmem and the discharge of the vmem node when the output
supplying current.
spike is generated. If only the current consumption of the
The ring oscillator is single-ended, using only 3 taps. Its threshold logic is taken into consideration, the energy per spike
minimum biasing voltage is 200mV. The inverter PMOS is drops to 28.77fJ/spike. A comparison with related accelerated
sized at 240nm/60nm, with the NMOS having half the width LIF neurons is displayed on table I. Energy per spike varies
and the same length. The output capacitor of each stage is greatly between pJ and fJ; nevertheless, our design shows
10fF, which is much higher than the equivalent Cg capacitance good power consumption with respect to other state-of-the-
of the inverters. For an operating point of 250mV, the ring art implementations.
oscillator has a gain of 23.96 MHz/V. The digital logic gates
used 200nm/60nm for both NMOS and PMOS transistors. IV. C ONCLUSSION
The threshold delay cell was implemented using one inverter This paper combines two new core ideas to implement a
with Vth at 0.2V. power-efficient spiking neuron. The first idea is using time
As shown in figure 5, the excitatory inputs are integrated encoding to represent the membrane potential and implement
into the membrane voltage by commuting the unit capacitor the threshold function [6] and the second is integration via
between VDD and vmem , and the inhibitory inputs subtract capacitive discharge. The combination of both results in a
charge by commuting from vmem to GND. Note that, as circuit that allows to easily seize the input coefficients and
explained in section II-A, excitatory inputs are low-level that operates power-efficiently. The simulation results show
active, while inhibitory inputs are high-level active. Once the a promising <40fJ/spike initial proof-of-concept. In addition,
membrane voltage reaches a certain voltage level (Vspk ), there refinements in this time-based LIF architecture, such as imple-
is a delay due to the RO oscillation behavior and the logic gates mentation in lower lithography nodes, may bring the proposed
delay, and the output spike is registered. This spike resets the architecture closer to some of the best power-efficient neurons
voltage membrane to its minimum operation level (Vmin ). currently researched [10].
 R EFERENCES
[1] W. Maass and C. Bishop, Pulsed Neural Networks, ser. A Bradford book.
MIT Press, 2001.
[2] L. Khacef, P. Klein, M. Cartiglia, A. Rubino, G. Indiveri, and
E. Chicca, “Spike-based local synaptic plasticity: A survey of
computational models and neuromorphic circuits,” 2022. [Online].
Available: https://arxiv.org/abs/2209.15536
[3] C. Raymond and E. Gutierrez, “A low power and low area mixed-signal
neuronal cell for spiking neural networks,” in 2021 IEEE International
Midwest Symposium on Circuits and Systems (MWSCAS), 2021, pp.
313–316.
[4] B. Datta Sahoo, “Ring oscillator based sub-1v leaky integrate-and-fire
neuron circuit,” in 2017 IEEE International Symposium on Circuits and
Systems (ISCAS), 2017, pp. 1–4.
[5] M. Payvand and G. Indiveri, “Spike-based plasticity circuits for always-
on on-line learning in neuromorphic systems,” in 2019 IEEE Interna-
tional Symposium on Circuits and Systems (ISCAS), 2019, pp. 1–5.
[6] J. Song, J. Shirn, H. Kim, and W.-S. Choi, “Energy-efficient high-
accuracy spiking neural network inference using time-domain neurons,”
in 2022 IEEE 4th International Conference on Artificial Intelligence
Circuits and Systems (AICAS), 2022, pp. 5–8.
[7] W. Jung, S. Jeong, S. Oh, D. Sylvester, and D. Blaauw, “27.6 a
0.7pf-to-10nf fully digital capacitance-to-digital converter using iterative
delay-chain discharge,” in 2015 IEEE International Solid-State Circuits
Conference - (ISSCC) Digest of Technical Papers, 2015, pp. 1–3.
[8] G. G. Gielen, L. Hernandez, and P. Rombouts, “Time-encoding analog-
to-digital converters: Bridging the analog gap to advanced digital cmos-
part 1: Basic principles,” IEEE Solid-State Circuits Magazine, vol. 12,
no. 2, pp. 47–55, 2020.
[9] A. Joubert, B. Belhadj, O. Temam, and R. Héliot, “Hardware spiking
neurons design: Analog or digital?” in The 2012 International Joint
Conference on Neural Networks (IJCNN), 2012, pp. 1–5.
[10] M. Besrour, S. Zitoun, J. Lavoie, T. Omrani, K. Koua, M. Benhouria,
M. Boukadoum, and R. Fontaine, “Analog spiking neuron in 28 nm
cmos,” in 2022 20th IEEE Interregional NEWCAS Conference (NEW-
CAS), 2022, pp. 148–152.
[11] S. Dutta, V. Kumar, A. Shukla, N. R. Mohapatra, and U. Ganguly,
“Leaky integrate and fire neuron by charge-discharge dynamics in
floating-body mosfet,” Scientific Reports, vol. 7, no. 1, p. 8257, Aug
2017. [Online]. Available: https://doi.org/10.1038/s41598-017-07418-y