With the emergence of the Internet of Things (IoT), devices will generate massive datastreams demanding services that pose huge technical challenges due to limited device resources.
Furthermore, IoT systems increasingly need to run complex and energy intensive Machine
Learning (ML) algorithms, but do not have the resources to run many state-of-the-art ML models,
instead opting to send their data to the cloud for computing. This results in insufficient security,
slower moving data, and energy intensive data centers. In order to achieve real-time learning in
IoT systems, we need to redesign the algorithms themselves using strategies that more closely
model the ultimate efficient learning machine: the human brain.
This dissertation focuses on increasing the computing efficiency of machine learning onIoT devices with the application of Hyperdimensional Computing (HDC). HDC mimics several
desirable properties of the human brain, including: robustness to noise, robustness to hardware
failures, and single-pass learning where training happens in one-shot without storing the training
data points or using complex gradient-based algorithms. These features make HDC a promising
solution for today’s embedded devices with limited storage, battery, and resources, and the
potential for noise and variability. Research in the HDC field has targeted improving these key
features of HDC and expanding to include even more features. There are four main paths in
HDC research: (1) Algorithmic changes for faster and more energy efficient learning, (2) Novel
architectures to accelerate HDC, usually targeting lower power IoT devices, (3) Extending HDC
applications beyond classification, (4) Exploiting the robust property of HDC for more efficient
and faster inference, and (5) HDC Theory, its connection to neuroscience and mathematics. This
dissertation contributes to four of these research paths in HDC.
Our contributions include: (1) We introduce the first adaptive bitwidth model for HDC.In this work we propose a new quantization method and during inference we iterate through
the bits along all dimensions taking the hamming distance. At each iteration, we check if the
current hamming distance passes a threshold similarity, if it does, we terminate execution early
to save energy and time. (2) We create a redesign of the entire HDC process with a locality-based
encoding, quantized retraining, and online dimension reduction during inference, all accelerated
by a new novel FPGA design. In this work we our locality-based encoding removes random
memory accesses from HDC encoding as well as adds sparsity for more efficiency. We also
introduce a general method to quantize to any desired model bitwidth. Finally, we propose a
method to find any insignificant dimensions in the HDC model and remove them for more energy
efficiency during inference. (3) We extend HDC to support multi-label classification. We
perform multi-label classification by creating a binary classification model for each label. Upon
inference, our models determine if each label exists independently. This is different than prior
work that took the power set of the labels to reduce the problem to a single label classification as HDC scales poorly with this method. (4) Finally, we experimentally evaluate the robustness
of HDC for the first time and create a new analog PIM architecture with reduced precision
Analog to Digital Converters (ADC), exploiting that robustness. We test HDC robustness in
a federated learning environment where edge devices send encoded hypervectors to a central
server wirelessly. We evaluate the impact of any wireless transmission errors on this data and
show that HDC is 48× more robust than other classifiers. We then use this knowledge that HDC
is robust to create a more efficient analog PIM circuit by reducing the bitwidth of the ADCs.
