Skip to main content
Springer Nature Link
Account
Menu
Find a journal Publish with us Track your research
Search
Cart
  1. Home
  2. International Journal of Parallel Programming
  3. Article

Managing Burstiness and Scalability in Event-Driven Models on the SpiNNaker Neuromimetic System

  • Open access
  • Published: 23 July 2011
  • Volume 40, pages 553–582, (2012)
  • Cite this article
Download PDF

You have full access to this open access article

International Journal of Parallel Programming Aims and scope Submit manuscript
Managing Burstiness and Scalability in Event-Driven Models on the SpiNNaker Neuromimetic System
Download PDF
  • Alexander D. Rast1,
  • Javier Navaridas1,
  • Xin Jin1,
  • Francesco Galluppi1,
  • Luis A. Plana1,
  • Jose Miguel-Alonso2,
  • Cameron Patterson1,
  • Mikel Luján1 &
  • …
  • Steve Furber1 

  • 1007 Accesses

  • 6 Citations

  • 3 Altmetric

  • Explore all metrics

Abstract

Neural networks present a fundamentally different model of computation from the conventional sequential digital model, for which conventional hardware is typically poorly matched. However, a combination of model and scalability limitations has meant that neither dedicated neural chips nor FPGA’s have offered an entirely satisfactory solution. SpiNNaker introduces a different approach, the “neuromimetic” architecture, that maintains the neural optimisation of dedicated chips while offering FPGA-like universal configurability. This parallel multiprocessor employs an asynchronous event-driven model that uses interrupt-generating dedicated hardware on the chip to support real-time neural simulation. Nonetheless, event handling, particularly packet servicing, requires careful and innovative design in order to avoid local processor congestion and possible deadlock. We explore the impact that spatial locality, temporal causality and burstiness of traffic have on network performance, using tunable, biologically similar synthetic traffic patterns. Having established the viability of the system for real-time operation, we use two exemplar neural models to illustrate how to implement efficient event-handling service routines that mitigate the problem of burstiness in the traffic. Extending work published in ACM Computing Frontiers 2010 with on-chip testing, simulation results indicate the viability of SpiNNaker for large-scale neural modelling, while emphasizing the need for effective burst management and network mapping. Ultimately, the goal is the creation of a library-based development system that can translate a high-level neural model from any description environment into an efficient SpiNNaker instantiation. The complete system represents a general-purpose platform that can generate an arbitrary neural network and run it with hardware speed and scale.

Article PDF

Download to read the full article text

Similar content being viewed by others

FNS allows efficient event-driven spiking neural network simulations based on a neuron model supporting spike latency

Article Open access 09 June 2021

Development of a Dynamically Extendable SpiNNaker Chip Computing Module

Chapter © 2014

Neuromorphic scaling advantages for energy-efficient random walk computations

Article 14 February 2022
Use our pre-submission checklist

Avoid common mistakes on your manuscript.

References

  1. Abeles M.: Local Cortical Circuits: An Electrophysiological Study. Springer, Berlin (1982)

    Book  Google Scholar 

  2. Ananthanarayanan, R., Modha, D.S.: Anatomy of a cortical simulator. In: Proceedings of 2007 ACM/IEEE International Conference on Supercomputing (SC’07), pp. 1–12 (2007)

  3. Boahen K.A.: Point-to-point connectivity between neuromorphic chips using address events. IEEE Trans. Circuits Syst. 2 Analog Digit. Signal Process. 47(5), 416–434 (2000)

    Article  MATH  Google Scholar 

  4. Cauwenberghs G.: An analog VLSI recurrent neural network learning a continuous-time trajectory. IEEE Trans. Neural Netw. 7(2), 346–361 (1996)

    Article  Google Scholar 

  5. Dally W.J., Seitz C.L.: Deadlock-free message routing in multiprocessor interconnection networks. IEEE Trans. Comput. C-36(5), 547–553 (1987)

    Article  Google Scholar 

  6. Davison, A.P., Brüderle, D., Eppler, J., Kremkow, J., Muller, E., Pecevski, D., Perrinet, L., Yger, P.: PyNN: a common interface for neuronal network simulators. Frontiers Neuroinform. 2(11) (2009)

  7. Delorme A., Thorpe S.J.: SpikeNET: an event-driven simulation package for modelling large networks of spiking neurons. Network Comput. Neural Syst. 14(4), 613–627 (2003)

    Article  Google Scholar 

  8. Fieres, J., Schemmel, J., Meier, K.: Realizing biological spiking network models in a configurable wafer-scale hardware system. In: Proceedings of 2008 International Joint Conference on Neural Networks (IJCNN2008), pp. 969–976. IEEE Press, NJ (2008)

  9. Furman B., White J., Abidi A.A.: CMOS analog IC implementing the back propagation algorithm. Neural Netw. 1(Supplement 1), 381 (1988)

    Google Scholar 

  10. Galluppi, F., Rast, A., Davies, S., Furber, S.: A General-purpose Model Translation System for a Universal Neural Chip. In: Proceedings of 2010 International Conference on Neural Information Processing (ICONIP 2010). Springer, Berlin (2010)

  11. Gewaltig M.O., Diesmann M.: NEST (NEural Simulation Tool). Scholarpedia 2(4), 1430 (2007)

    Article  Google Scholar 

  12. Glackin, B., McGinnity, T.M., Maguire, L.P., Wu, Q.X., Belatreche, A.: A novel approach for the implementation of large scale spiking neural networks on FPGA hardware. In: Proceedings of 8th Internatioal Work Conference on Artificial Neural Networks (IWANN 2005), pp. 552–563. Springer, Berlin (2005)

  13. Goodman, D., Brette, R.: Brian: a simulator for spiking neural networks in Python. Frontiers Neuroinform. 2(5) (2008)

  14. Graf, H.P., Hubbard, W., Jackel, L.D., de Vegvar, P.G.N.: A CMOS Associative Memory Chip. In: Proceedings of IEEE First International Conference on Neural Networks, pp. 461–468 (1987)

  15. Hagmann P., Cammoun L., Gigandet X., Meuli R., Honey C.J., Wedeen V.J., Sporns O.: Mapping the structural core of human cerebral cortex. PLoS Biology 6(7), 1479–1493 (2008)

    Article  Google Scholar 

  16. Harkin, J., Morgan, F., Hall, S., Dudek, P., Dowrick, T., McDaid, L.: Reconfigurable platforms and the challenges for large-scale implementations of spiking neural networks. In: Proceedings of 2008 International Conference on Field Programmable Logic and Applications (FPL 2008), pp. 483–486 (2008)

  17. Hayon G., Abeles M., Lehmann D.: A model for representing the dynamics of a system of synfire chains. J. Comput. Sci. 18(1), 41–53 (2005)

    MathSciNet  Google Scholar 

  18. Hines M.L., Carnevale N.T.: The NEURON simulation environment. Neural Comput. 9(6), 1179–1209 (1997)

    Article  Google Scholar 

  19. Holler, M., Tam, S., Castro, H., Benson, R.: An electrically trainable artificial neural network (ETANN) with 10240 “Floating Gate” Synapses. In: Proceedings of 1989 Internatiional Joint Conference on Neural Networks (IJCNN1989), pp. 191–196 (1989)

  20. Indiveri G., Chicca E., Douglas R.: A VLSI array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity. IEEE Trans. Neural Netw. 17(1), 211–221 (2006)

    Article  Google Scholar 

  21. Izhikevich E.: Simple model of spiking neurons. IEEE Trans. Neural Netw. 14, 1569–1572 (2003)

    Article  Google Scholar 

  22. Izhikevich E., Edelman G.M.: Large-scale model of mammalian thalamocortical systems. Proc. Natl. Acad. Sci. USA 105(9), 3593–3598 (2008)

    Article  Google Scholar 

  23. Izhikevich E.M.: Which model to use for cortical spiking neurons. IEEE Trans. Neural Netw. 15(5), 1063–1070 (2004)

    Article  Google Scholar 

  24. James M., Hoang D.: Design of low-cost, real-time simulation systems for large neural networks. J. Parallel Distrib. Comput. 14(3), 221–235 (1992)

    Article  Google Scholar 

  25. Jin, X., , Luján, M., Khan, M.M., Plana, L.A., Rast, A.D., Welbourne, S.R., Furber, S.B.: Efficient parallel implementation of multilayer backpropagation network on torus-connected CMPs. In: Proceedings of 2010 ACM International Conference on Computing Frontiers (CF’10), pp. 89–90 (2010)

  26. Jin, X., Furber, S., Woods, J.: Efficient modelling of spiking neural networks on a scalable chip multiprocessor. In: Proceedings of 2008 International Joint Conference on Neural Networks (IJCNN2008) (2008)

  27. Jin, X., Rast, A., Galluppi, F., Khan, M.M., Furber, S.: Implementing learning on the SpiNNaker universal neural chip multiprocessor. In: Proceedings of 2009 International Conference on Neural Information Processing (ICONIP 2009). Springer, Berlin (2009)

  28. Johansson C., Lansner A.: Towards cortex sized artificial neural systems. Neural Netw. 20(1), 48–61 (2007)

    Article  MATH  Google Scholar 

  29. Khan, M., Lester, D., Plana, L., Rast, A., Jin, X., Painkras, E., Furber, S.: SpiNNaker: Mapping Neural Networks onto a Massively-Parallel Chip Multiprocessor. In: Proceedings of 2008 International Joint Conference on Neural Networks (IJCNN2008) (2008)

  30. Lazzaro J., Wawrzynek J., Mahowald M., Silviotti M., Gillespie D.: Silicon auditory processors as computer peripherals. IEEE Trans. Neural Netw. 4(3), 523–528 (1993)

    Article  Google Scholar 

  31. Lee B.J., Sheu B.W.: General-purpose neural chips with electrically programmable synapses and gain-adjustable neurons. IEEE J. Solid-State Circuits 27(9), 1299–1302 (1992)

    Article  Google Scholar 

  32. Lytton W.H., Omurtag A., Neymotin S.A., Hines M.L.: Just-in-Time connectivity for large spiking networks. Neural Comput. 20(11), 2745–2756 (2008)

    Article  MATH  Google Scholar 

  33. Maguire L., McGinnity T.M., Glackin B., Ghani A., Belatreche A., Harkin J.: Challenges for large-scale implementations of spiking neural networks on FPGAs. Neurocomputing 71(1–3), 13–29 (2007)

    Article  Google Scholar 

  34. Mattia M., Guidice P.D.: Efficient event-driven simulation of large networks of spiking neurons and dynamical synapses. Neural Comput. 12(10), 2305–2329 (2000)

    Article  Google Scholar 

  35. Mehrtash N., Jung D., Hellmich H., Schönauer T., Lu V.T., Klar H.: Synaptic plasticity in spiking neural networks (SP2INN): a System Approach. IEEE Trans. Neural Netw. 14(5), 980–992 (2003)

    Article  Google Scholar 

  36. Migliore M., Cannia C., Lytton W.W., Markram H., Hines M.L.: Parallel network simulations with NEURON. J. Comput. Neurosci. 21(2), 119–129 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  37. Mouraud, A., Paugam-Moisy, H., Puzenat, D.: A distributed and multithreaded neural event driven simulation framework. In: Proceedings of IASTED International Conference on Parallel and Distributed Computing and Networks, pp. 212–217 (2006)

  38. Nageswaran, J.M., Dutt, N., Krichmar, J.L., Nicolau, A.: A configurable simulation environment for the efficient simulation of large-scale spiking neural networks on graphics processors. Neural Netw. 22(5–6) (2007)

    Google Scholar 

  39. Navaridas, J., Luján, M., Miguel-Alonso, J., Plana, L.A., Furber, S.B.: Understanding the interconnection network of SpiNNaker. In: Proceedings of 23rd International Conference on Supercomputing (ICS’09), pp. 286–295 (2009)

  40. Navaridas J., Miguel-Alonso J., Pascual J.A., Ridruejo F.J.: Simulating and evaluating interconnection networks with INSEE. Simul. Modell. Pract. Theory 19(1), 494–515 (2011)

    Article  Google Scholar 

  41. Navaridas, J., Plana, L.A., Miguel-Alonso, J., Luján, M., Furber, S.B.: SpiNNaker: impact of traffic locality, causality and burstiness on the performance of the interconnection Network. In: Proceedings of 2010 ACM Conference on Computing Frontiers (CF’10), pp. 11–19 (2010)

  42. Orellana, C.G., Caballero, R.G., Velasco, H.M.G., Aligue, F.J.L.: NeuSim: a modular neural networks simulator for Beowulf clusters. In: Proceedings of 6th International Work-Conference on Artifical and Natural Neural Networks (IWANN 2001), Part II, pp. 72–79. Springer, Berlin (2001)

  43. Pelayo F.J., Ros E., Arreguit X., Prieto A.: VLSI implementation of a neural model using spikes. Analog Integr. Circuits Signal Process. 13(1–2), 111–121 (1997)

    Article  Google Scholar 

  44. Plana L., Furber S., Temple S., Khan M., Shi Y., Wu J., Yang S.: A GALS infrastructure for a massively parallel multiprocessor. IEEE Design Test Comput. 24(5), 454–463 (2007)

    Article  Google Scholar 

  45. Porrmann, M., Witkowski, U., Kalte, H., Rückert, U.: Implementation of artificial neural networks on a reconfigurable hardware accelerator. In: Proceedings of 2002 Euromicro Conference on Parallel, Distributed, and Network-based processing, pp. 243–250 (2002)

  46. Rast, A., Jin, X., Khan, M., Furber, S.: The deferred event model for hardware-oriented spiking neural networks. In: Proceedings of 2008 International Conference on Neural Information Processing (ICONIP 2008). Springer, Berlin (2009)

  47. Rast, A., Khan, M.M., Jin, X., Plana, L.A., Furber, S.: A universal abstract-time platform for real-time neural networks. In: Proceedings of 2009 International Joint Conference on Neural Networks (IJCNN2009), pp. 2611–2618 (2009)

  48. Rast, A., Welbourne, S., Jin, X., Furber, S.: Optimal connectivity in hardware-targetted MLP networks. In: Proceedings of 2009 International Joint Conference on Neural Networks (IJCNN2009), pp. 2619–2626 (2009)

  49. Rast, A., Yang, S., Khan, M., Furber, S.: Virtual synaptic interconnect using an asynchronous network-on-chip. In: Proceedings of 2008 International Joint Conference on Neural Networks (IJCNN2008) (2008)

  50. Rast, A.D., Galluppi, F., Jin, X., Furber, S.B.: The Leaky Integrate-and-Fire Neuron: A Platform for Synaptic Model Exploration on the SpiNNaker Chip. In: Proceedings of 2010 International Joint Conference Neural Networks (IJCNN2010), pp. 3959–3966 (2010)

  51. Rast, A.D., Jin, X., Galluppi, F., Plana, L.A., Patterson, C., Furber, S.B.: Scalable event-driven native parallel processing: the SpiNNaker neuromimetic System. In: Proceedings of 2010 ACM Conference on Computing Frontiers (CF’10), pp. 20–29 (2010)

  52. Rice, K.L., Vutsinas, C.N., Taha, T.M.: A preliminary investigation of a neocortex model implementation on the cray XD1. In: Proceedings of 2007 ACM/IEEE International Conference on Supercomputing (SC’07), pp. 1–8 (2007)

  53. Ros E., Carrillo R., Ortigosa E.M.: Event-driven simulation scheme for spiking neural networks using lookup tables to characterize neuronal activity. Neural Comput. 18(12), 2959–2993 (2006)

    Article  MATH  Google Scholar 

  54. Rückert, U.: ULSI architectures for artificial neural networks. In: Proceedings of 9th Euromicro Workshop on Parallel and Distributed Processing, pp. 436–442 (2001)

  55. Steinkraus, D., Buck, I., Simard, P.Y.: Using GPUs for machine learning algorithms. In: Proceedings 8th International Conference on Document Analysis and Recognition, pp. 1115–1120 (2005)

  56. Upegui A., Peña-Reyes C.A., Sanchez E.: An FPGA platform for on-line topology exploration of spiking neural networks. Microprocess. Microsyst. 29(5), 211–223 (2005)

    Article  Google Scholar 

  57. Vogelstein R.J., Mallik U., Vogelstein J.T., Cauwenberghs G.: Dynamically reconfigurable silicon array of spiking neurons with conductance-based synapses. IEEE Trans. Neural Netw. 18(1), 253–265 (2007)

    Article  Google Scholar 

  58. Watts, L.: Event-driven simulation of networks of spiking neurons. In: Advances in Neural Information Processing (NIPS) 6, pp. 927–934. Morgan Kaufmann Publishers, Waltham (1994)

  59. Yasunaga, M., Masuda, N., Yagyu, M., Asai, M., Yamada, M., Masaki, A.: Design, Fabrication and Evaluation of a 5-inch Wafer Scale Neural Network LSI Composed of 576 Digital Neurons. In: Proceedings of 1990 International Joint Conference Neural Networks (IJCNN1990), pp. 527–535 (1990)

  60. Yu T., Cauwenberghs G.: Analog VLSI biophysical neurons and synapses with programmable membrane channel kinetics. IEEE Trans. Biomed. Circuits Syst. 4(3), 139–148 (2010)

    Article  Google Scholar 

Download references

Acknowledgments

The SpiNNaker project is supported by the Engineering and Physical Sciences Research Council, partly through the Advanced Processor Technologies Portfolio Partnership at the University of Manchester, and through Grants EP/D07908X/1 and GR/S61270/01; and also by ARM and Silistix. When this research was performed Dr. Javier Navaridas was supported by a post-doctoral grant of the University of the Basque Country and is now a Newton International Fellow with the University of Manchester. Prof. Jose Miguel-Alonso is supported by the Spanish Ministry of Education and Science, grant TIN2010-14931, and by Basque Government grant IT-242-07. Dr.Mikel Luján holds a Royal Society University Research Fellowship. We appreciate the support of these sponsors and industrial partners.

Open Access

This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.

Author information

Authors and Affiliations

  1. School of Computer Science, University of Manchester, Oxford Road, Manchester, M13 9PL, UK

    Alexander D. Rast, Javier Navaridas, Xin Jin, Francesco Galluppi, Luis A. Plana, Cameron Patterson, Mikel Luján & Steve Furber

  2. School of Computer Science, University of the Basque Country, Manuel Lardizabal, 1, 20018, San Sebastian, Spain

    Jose Miguel-Alonso

Authors
  1. Alexander D. Rast
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Javier Navaridas
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xin Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Francesco Galluppi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Luis A. Plana
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jose Miguel-Alonso
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Cameron Patterson
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Mikel Luján
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Steve Furber
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alexander D. Rast.

Rights and permissions

Open Access This is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License (https://creativecommons.org/licenses/by-nc/2.0), which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.

Reprints and permissions

About this article

Cite this article

Rast, A.D., Navaridas, J., Jin, X. et al. Managing Burstiness and Scalability in Event-Driven Models on the SpiNNaker Neuromimetic System. Int J Parallel Prog 40, 553–582 (2012). https://doi.org/10.1007/s10766-011-0180-7

Download citation

  • Received: 31 October 2010

  • Accepted: 07 July 2011

  • Published: 23 July 2011

  • Issue Date: December 2012

  • DOI: https://doi.org/10.1007/s10766-011-0180-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Keywords

  • Asynchronous
  • Burst
  • Network
  • Event-driven
  • Universal
  • Neural
  • Multiprocessor
  • Interconnection
  • Real-time
  • Traffic
  • Characterisation
Use our pre-submission checklist

Avoid common mistakes on your manuscript.

Advertisement

Search

Navigation

  • Find a journal
  • Publish with us
  • Track your research

Discover content

  • Journals A-Z
  • Books A-Z

Publish with us

  • Journal finder
  • Publish your research
  • Open access publishing

Products and services

  • Our products
  • Librarians
  • Societies
  • Partners and advertisers

Our imprints

  • Springer
  • Nature Portfolio
  • BMC
  • Palgrave Macmillan
  • Apress
  • Your US state privacy rights
  • Accessibility statement
  • Terms and conditions
  • Privacy policy
  • Help and support
  • Cancel contracts here

Not affiliated

Springer Nature

© 2024 Springer Nature