A memristor-based Bayesian machine

7 min read Original article ↗

References

  1. Editorial. Big data needs a hardware revolution. Nature 554, 145–146 (2018).

    Google Scholar 

  2. Zhang, W. et al. Neuro-inspired computing chips. Nat. Electron. 3, 371–382 (2020).

    Article  Google Scholar 

  3. Sebastian, A., Le Gallo, M., Khaddam-Aljameh, R. & Eleftheriou, E. Memory devices and applications for in-memory computing. Nat. Nanotechnol. 15, 529–544 (2020).

    Article  Google Scholar 

  4. Marković, D., Mizrahi, A., Querlioz, D. & Grollier, J. Physics for neuromorphic computing. Nat. Rev. Phys. 2, 499–510 (2020).

    Article  Google Scholar 

  5. Pedram, A., Richardson, S., Horowitz, M., Galal, S. & Kvatinsky, S. Dark memory and accelerator-rich system optimization in the dark silicon era. IEEE Design & Test 34, 39–50 (2017).

    Article  Google Scholar 

  6. Ambrogio, S. et al. Equivalent-accuracy accelerated neural-network training using analogue memory. Nature 558, 60–67 (2018).

    Article  Google Scholar 

  7. Yu, S. Neuro-inspired computing with emerging nonvolatile memorys. Proc. IEEE 106, 260–285 (2018).

    Article  Google Scholar 

  8. Prezioso, M. et al. Training and operation of an integrated neuromorphic network based on metal-oxide memristors. Nature 521, 61–64 (2015).

    Article  Google Scholar 

  9. Ielmini, D. & Wong, H.-S. P. In-memory computing with resistive switching devices. Nat. Electron. 1, 333–343 (2018).

    Article  Google Scholar 

  10. Wang, Z. et al. Fully memristive neural networks for pattern classification with unsupervised learning. Nat. Electron. 1, 137–145 (2018).

    Article  Google Scholar 

  11. Xue, C.-X. et al. A CMOS-integrated compute-in-memory macro based on resistive random-access memory for AI edge devices. Nat. Electron. 4, 81–90 (2021).

    Article  Google Scholar 

  12. Chen, D. et al. Deep learning and alternative learning strategies for retrospective real-world clinical data. npj Digit. Med. 2, 43 (2019).

    Article  Google Scholar 

  13. Ghassemi, M. et al. A review of challenges and opportunities in machine learning for health. AMIA Jt Summits Transl. Sci. Proc. 2020, 191–200 (2020).

    Google Scholar 

  14. Rai, A. Explainable AI: from black box to glass box. J. Acad. Mark. Sci. 48, 137–141 (2020).

    Article  Google Scholar 

  15. Ghahramani, Z. Probabilistic machine learning and artificial intelligence. Nature 521, 452–459 (2015).

    Article  Google Scholar 

  16. Letham, B., Rudin, C., McCormick, T. H. & Madigan, D. Interpretable classifiers using rules and Bayesian analysis: building a better stroke prediction model. Ann. Appl. Stat. 9, 1350–1371 (2015).

    Article  MATH  Google Scholar 

  17. Jaynes, E. T. Probability Theory: The Logic of Science (Cambridge Univ. Press, 2003).

  18. Bessière, P., Mazer, E., Ahuactzin, J. M. & Mekhnacha, K. Bayesian Programming (CRC Press, 2013).

  19. Van de Schoot, R. et al. A gentle introduction to Bayesian analysis: applications to developmental research. Child Dev. 85, 842–860 (2014).

    Article  Google Scholar 

  20. Laurens, J. & Droulez, J. Bayesian processing of vestibular information. Biol. Cybern. 96, 389–404 (2007).

    Article  MATH  Google Scholar 

  21. Lee, T. S. & Mumford, D. Hierarchical Bayesian inference in the visual cortex. J. Opt. Soc. Am. A 20, 1434–1448 (2003).

    Article  Google Scholar 

  22. Maass, W. Noise as a resource for computation and learning in networks of spiking neurons. Proc. IEEE 102, 860–880 (2014).

    Article  Google Scholar 

  23. Knill, D. C. & Pouget, A. The Bayesian brain: the role of uncertainty in neural coding and computation. Trends Neurosci. 27, 712–719 (2004).

    Article  Google Scholar 

  24. Deneve, S. Bayesian spiking neurons I: inference. Neural Comput. 20, 91–117 (2008).

    Article  MATH  Google Scholar 

  25. Houillon, A., Bessière, P. & Droulez, J. The probabilistic cell: implementation of a probabilistic inference by the biochemical mechanisms of phototransduction. Acta Biotheor. 58, 103–120 (2010).

    Article  Google Scholar 

  26. Smith, R. J., Ashton, G., Vajpeyi, A. & Talbot, C. Massively parallel Bayesian inference for transient gravitational-wave astronomy. Mon. Notices Royal Astron. Soc. 498, 4492–4502 (2020).

    Article  Google Scholar 

  27. Leech, C., Raykov, Y. P., Ozer, E. & Merrett, G. V. Real-time room occupancy estimation with Bayesian machine learning using a single PIR sensor and microcontroller. In 2017 IEEE Sensors Applications Symposium (SAS) 1–6 (IEEE, 2017).

  28. Lei, X. & Wu, Y. Research on mechanical vibration monitoring based on wireless sensor network and sparse Bayes. Eurasip J. Wirel. Commun. Netw. 2020, 225 (2020).

    Article  Google Scholar 

  29. Ferreira, J. F., Lobo, J. & Dias, J. Bayesian real-time perception algorithms on GPU. J. Real-Time Image Proc. 6, 171–186 (2011).

    Article  Google Scholar 

  30. Zermani, S., Dezan, C., Chenini, H., Diguet, J.-P. & Euler, R. FPGA implementation of Bayesian network inference for an embedded diagnosis. In 2015 IEEE Conference on Prognostics and Health Management (PHM) 1–10 (IEEE, 2015).

  31. Cai, R. et al. Vibnn: hardware acceleration of Bayesian neural networks. ACM SIGPLAN Not. 53, 476–488 (2018).

    Article  Google Scholar 

  32. Liu, S., Mingas, G. & Bouganis, C.-S. An unbiased MCMC FPGA-based accelerator in the land of custom precision arithmetic. IEEE Trans. Comput. 66, 745–758 (2016).

    Article  MATH  Google Scholar 

  33. Frisch, R. et al. A Bayesian stochastic machine for sound source localization. In 2017 IEEE International Conference on Rebooting Computing (ICRC) 1–8 (IEEE, 2017).

  34. Ko, G. G. et al. A 3mm2 programmable Bayesian inference accelerator for unsupervised machine perception using parallel Gibbs sampling in 16nm. In 2020 IEEE Symposium on VLSI Circuits 1–2 (IEEE, 2020).

  35. Faria, R., Camsari, K. Y. & Datta, S. Implementing Bayesian networks with embedded stochastic MRAM. AIP Adv. 8, 045101 (2018).

    Article  Google Scholar 

  36. Friedman, J. S., Calvet, L. E., Bessière, P., Droulez, J. & Querlioz, D. Bayesian inference with Muller C-elements. IEEE Trans. Circuits Syst. I: Regul. Pap. 63, 895–904 (2016).

    Article  MATH  Google Scholar 

  37. Vodenicarevic, D. et al. Low-energy truly random number generation with superparamagnetic tunnel junctions for unconventional computing. Phys. Rev. Appl. 8, 054045 (2017).

    Article  Google Scholar 

  38. Dalgaty, T. et al. In situ learning using intrinsic memristor variability via Markov chain Monte Carlo sampling. Nat. Electron. 4, 151–161 (2021).

    Article  Google Scholar 

  39. Gao, D. et al. Bayesian inference based robust computing on memristor crossbar. In 2021 58th ACM/IEEE Design Automation Conference (DAC) 121–126 (IEEE, 2021).

  40. Gaines, B. R. Stochastic computing systems. in Advances in Information Systems Science 37–172 (Springer, 1969).

  41. Alaghi, A. & Hayes, J. P. Survey of stochastic computing. ACM Trans. Embed. Comput. Syst. 12, 92 (2013).

    Article  Google Scholar 

  42. Winstead, C. Tutorial on stochastic computing. in Stochastic Computing: Techniques and Applications 39–76 (Springer, 2019).

  43. Chang, Y.-F. et al. eNVM RRAM reliability performance and modeling in 22FFL FinFET technology. In 2020 IEEE International Reliability Physics Symposium (IRPS) 1–4 (IEEE, 2020).

  44. Gregori, S., Cabrini, A., Khouri, O. & Torelli, G. On-chip error correcting techniques for new-generation flash memories. Proc. IEEE 91, 602–616 (2003).

    Article  Google Scholar 

  45. Hirtzlin, T. et al. Digital biologically plausible implementation of binarized neural networks with differential hafnium oxide resistive memory arrays. Front. Neurosci. 13, 1383 (2020).

    Article  Google Scholar 

  46. Gupta, P. K. & Kumaresan, R. Binary multiplication with pn sequences. IEEE Trans. Acoust., Speech, Signal Process. 36, 603–606 (1988).

    Article  MATH  Google Scholar 

  47. Warden, P. & Situnayake, D. Tinyml: Machine Learning with Tensorflow Lite on Arduino and Ultra-Low-Power Microcontrollers (O’Reilly Media, 2019).

  48. Petzold, S. et al. Heavy ion radiation effects on hafnium oxide-based resistive random access memory. IEEE Trans. Nucl. Sci. 66, 1715–1718 (2019).

    Article  Google Scholar 

  49. Borders, W. A. et al. Integer factorization using stochastic magnetic tunnel junctions. Nature 573, 390–393 (2019).

    Article  Google Scholar 

  50. Li, C. et al. CMOS-integrated nanoscale memristive crossbars for CNN and optimization acceleration. In 2020 IEEE International Memory Workshop (IMW) 1–4 (IEEE, 2020).

  51. Yao, P. et al. Fully hardware-implemented memristor convolutional neural network. Nature 577, 641–646 (2020).

    Article  Google Scholar 

  52. Wan, W. et al. 33.1 A 74 TMACS/W CMOS-RRAM neurosynaptic core with dynamically reconfigurable dataflow and in-situ transposable weights for probabilistic graphical models. In 2020 IEEE International Solid-State Circuits Conference—(ISSCC), 498–500 (IEEE, 2020).

  53. Jung, S. et al. A crossbar array of magnetoresistive memory devices for in-memory computing. Nature 601, 211–216 (2022).

    Article  Google Scholar 

  54. Khaddam-Aljameh, R. et al. HERMES-core—a 1.59-TOPS/mm2 PCM on 14-nm CMOS in-memory compute core using 300-ps/LSB linearized CCO-based ADCs. IEEE J. Solid-State Circuits 57, 1027–1038 (2022).

    Article  Google Scholar 

Download references