Instructor guide¶
Why we teach this lesson¶
Alternative forms of computing have gained relevance in the last decade as traditional CPU and GPU architectures reach fabrication and physical limits, and the power consumption of compute clusters sharply increased with the rise of ML/AI algorithms. New computing paradigms such as quantum and analog computing, as well as dedicated hardware architectures offer paths to sidestep the limitations of classical computing. Neuromorphic computing is another paradigm competing in this area: based on replicating the behavior of biological neurons, it combines the advantages of analog and digital computing, resulting in computations that are inherently low power and well suited for solving a large number of problems and implementing a wide variety of algorithms including optimization algorithms, AI/ML, and more. Working in neuromorphic computing requires new ways of thinking and of approaching algorithm design. This module is intended as a light introduction to neuromorphic computing, its basic concepts, advantages and applicability domains, and finally a couple of hands-on practical simulations based on Python code.
Timing¶
Approximated timings:
30 mins for theoretical introduction
30 mins for implementing the simple LIF model
30 mins for the introduction to PyNN
1-1.5 hours for the map coloring model tutorial
Total: Around 3 hours, plus pause, Q&A, etc.
Hardware requirements¶
The exercises can be carried out on any regular computer or compute node with the right environment.
Preparing exercises¶
Environment should be tested beforehand. In particular, the connection between PyNN and the Neuron simulator is sometimes not correctly configured out of the box. Using different version of the Neuron library for Python can help fix this. The same applies if the Jupyter notebooks are used.
Interesting questions you might get¶
How do these simulations compares to realistic biological neuron networks? If the simulation parameters are chosen correctly, it can be run practically in “real time” compared to a biological network, since biological neurons also are active in the milliseconds range (compared to something like molecular simulations where simulated vs wall time is off by orders of magnitude). It is possible to simulate full connectomes of small animals, such as the D. melanogaster nervous system. The limitation here is more the availability of experimental data on large biological networks than limitations on neuromorphic computing itself.
The practical limitations of neuromorphic simulations are not so obvious. One might think the bottleneck is the total number of neurons that can be simulated, but this can easily reach hundreds of thousands of individual neurons. Th e real problem are the total number of connections between neurons (which even in biological networks can reach hundreds or thousands of synapses per neurons), and routing the messages/spikes between them. Many simulation packages will error out if there are too many connections, or too many spikes generated in a given timestep.
Dedicated hardware simulations in addition have extra memory and numerical limitations due to their implementation, for example, limited numerical ranges for the values of the neuron model parameters.
Typical pitfalls¶
The specific terminology used when setting up spiking neuron networks can take some using to. In particular the term “projection” for connecting groups of neurons is not necessarily intuitive. For advanced usage, the difference in PyNN between Populations, Views and Assemblies can be subtle but important.
Projections and neuron connections are not bidirectional by default, they have a specific direction, going from pre- to post-synaptic population. Some projection functions do have an extra option to make bidirectional connections, but this is not the default (and this is also not the default in biological networks! feedback loops are indirect).
It is important that participants understand the behavior of the basic LIF model, and get an intuitive qualitative understanding of how a simulation will run or is expected to run so they can detect and diagnose obvious ill-formed simulations. Related to this, they should be able to get an idea of the influence of the different parameters of the LIF model, and their relative values. For example, if I want a neuron to spike on average after X successive incoming spikes, what should the synapse strength be compared to the threshold voltage? If neurons are too sensitive, how can I change the time constant tau or the threshold voltage to alleviate this?
Related to this, it is particularly important that participants understand that the simpler neuron models naturally tend to a resting state if not continuously excited by an external input. This means a LIF-only network will just do nothing at all. Some external network input is required. This usually takes the form of continuous concrete input if implementing some ML or optimization problem (such as the “clues” in the map coloring problem), or randomly generated Poisson noise in the case of a biologically inspired network, which simulated background neural activity or activity from portions of the nervous system that have been abstracted away.
Some types of simulations, usually those implementing optimization methods, use neurons that are all identical in model and parameters. Others require non-clone neurons with a distribution of parameters, particularly biological simulations, since real neurons are not all identical. PyNN and other simulation libraries and dedicated hardware usually have ways of easily initializing groups of neurons with slightly different model parameters. Simulations with all clone neurons can exhibit very unintuitive behaviors since the whole system will run in lockstep without any source of randomness such as external noise.
When doing the constraint optimization exercise, there are some unintuitive “tricks” that can be used, but become somewhat self-evident after they have been explained once. For example, it is not evident that a neuron population can be connected to itself, so it can excite itself and keep itself in the “on” state.