ANNUAL REPORT 2012

HTM - Hierarchical Temporal Memory on Manycores
HTM - Hierarchical Temporal Memory on Many-cores
T. Nordström, D. Hammerstrom, Zain-ul-Abdin, J. Duracz, and B. Svensson
Centre for Research on Embedded Systems
Introduction
An increasingly important aspect of embedded computing is
the processing and understanding of noisy real world data,
then making decisions and taking timely actions based on these
data. Consequently, various kinds of intelligent computing
structures are being investigated as important building blocks
in embedded system design.
One very promising algorithm is Hierarchical Temporal Memory
(HTM) which is being developed by Numenta, Inc. and
which is already being used in a number of real applications.
Being based on more biological kinds of models, HTM is massively
parallel, but it is also computationally intensive and as
it is integrated into real applications, it is starting to run into
performance limitations. The goal of this CERES project is to
explore suitable hardware support for the acceleration of the
Hierarchical Temporal Memory
HTM Learning
Cortical Learning Algorithm (CLA) is a memory system that
learns sequences of patterns and makes predictions. When an
HTM model is exposed to a stream of data the CLA predicts
what is likely to happen next, similar to how you predict the
next note in a familiar song or the next word someone is likely
to say in a common phrase. In addition, the CLA modifies
its memory with each new record. Thus the HTM models are
continually adapting to reflect the most recent patterns.
Next Steps
As memory is critical in HTM, as in most artificial neural network
models, we will focus our effort on many-core mapping
strategies towards optimizing memory management.
In our cooperation with Portland State University, PSU has
been focusing on FPGA and GPU implementation and HH
on multi-core and Ambric/Adapteva “many-core” style parallelism.
As a next step we hope to compare these different implementations.
Project Key Data
Partners: Halmstad University, Portland State University, Numenta,
Inc., Nethra Imaging, Inc. , and Adapteva, Inc.
Duration: Sep. 2011 – Dec. 2013,
Funding: CERES+ project, Volume: 820 kSEK
Contact: Tomas Nordström, HH
HTM Structure
The HTM-CLA is a highly detailed model of a layer of cells in
the neocortex. In a typical CLA implementation there are 2000
columns of simulated neurons (one per output bit of the spatial
memory structure) and twenty simulated neurons per column,
giving each CLA over 40,000 neurons. Each neuron has dozens
of non-linear dendrite segments and potentially thousands of
synapses.
HTM on Adapteva
Master students Zhou Xi & Luo Yaoyao, have implemented
CLA on our Adapteva development board.
They have investigated how to map HTM-CLA onto Adapteva’s
Epiphany many-core architecture and have been running
experiments to find out the speedup and efficiency of this
HTM mapping onto this many-core architecture. Preliminary
results show an almost perfect scalability when mapping HTM
onto Adapteva.
CERES Annual Report 2012
23