Neuromorphic Computing: A Signal Processing Perspective

“OK Google, do not transfer any of my data to the network.”

In these days Image result for privacy machine learningof GDPR and AI-enhanced toasters, can we keep our personal data private while still enjoying the convenience of mobile assistants, autonomous navigation systems, smart home appliances, and health-monitoring wearables?

The question is central to many discussions on the future of our societies. While for now “consumers on the whole seem content to beat a little totalitarism for convenience”, as Tim Wu wrote, things may soon change as more and more scandals around the use of personal data are undermining the trust in the tech giants. This is both a cultural and technological problem.

I have written before about neuromorphic computing as a promising technology for the implementation of low-power machine learning algorithms on mobile and embedded devices. Local processing on personal devices contrasts with the edge- or cloud-based learning of standard solutions, including within 5G, and could help developing privacy-minded AI applications.

Since my original post, a lot has happened in the field. Commercial products are now available thanks to the emergence of start-ups such as BrainChip; Intel has launched its own neuromorphic chip and supported a programming platform developed by Applied Brain Research, another start-up; and new applications in computer science and engineering are emerging that take advantage of the energy efficiency of the technology.

My fascination with the field has also not waned. But finding good references to recommend to engineering students and researchers interested in an introduction to the topic is not easy.  Most previous work has been done in the computational neuroscience, where conceptual aspects are often lost amid biological arguments and lengthy asides on relationships with existing hypotheses on the operation of the brain. In the hope of providing a friendlier entry point, I would like to share a tutorial prepared with Hyeryung Jang, Brian Gardner, and André Grüning:

Feedback and comments are as always very welcome.



Before and After Facebook

It’s 2002, Myspace has yet to be founded, the Facebook30342903 we know is still years away, and Google is a private company. With uncanny foresight, a writer in Iceland sees through the following years of social networks creeping into the lives of millions through computers and mobile devices to imagine the ultimate, pervasive and intangible, social web. Tapping into the communication system used by flocks of birds, the LoveStar corporation develops a technology that enables people to communicate directly with one another — and with LoveStar’s employees — via their thoughts, with no need for gadgets or physical proximity. Everyone is seamlessly connected in a gigantic virtual social network.

What is most striking about Andri Snær Magnason’s imagined future is that he also foresaw the main commercial use of a universal platform connecting users and corporation: advertising. Unlike the corporations ruling the web today, LoveStar could bypass the burdensome step of capturing one’s attention in order to sell it to a third-party. Instead, people’s speech centers could be directly rented out and hijacked to automatically relay advertisements or post-purchase praise for targeted passerby, friends, or schoolmates. “A company could have its name substituted for hello, bye, really, yes, no, black, or white”, forcibly infiltrating any conversation. 

Today this sounds more like a business plan than science-fiction. Or in the author’s words:

“when it came out in 2002 it was called a dystopian novel; now it’s being called a parody. We seem to have already reached that dystopia.”

The novel ends on a somewhat positive note, but not before untold suffering is unleashed on the connected humanity. It may be time to start thinking about how to change course in our own timeline.

The New Futurological Congress

In 1971 — writing in what was then the Soviet Union, was previously Poland and would later become Ukraine — Stanisław Lem imagined “The Futurological Congress” Held in Costa Rica, planet Earth, the “first item of business [of the congress] was to address first the world urban crisis, the second — the ecology crisis, the third — the air pollution crisis, the fourth — the energy crisis, the fifth — the food crisis. Then, adjournment. The technology, 24070877._SX540_military and political crises were to be dealt with on the following day”.  The conference room was overcrowded with representatives from many countries, so, to “help expedite the proceedings, […] the lecturer would speak only in numerals, calling attention in this fashion to the salient paragraphs of his work. […] Stan Hazelton of the U.S. delegation immediately threw the hall into a flurry by emphatically repeating: 4, 6, 11, and therefore 22; 5, 9, hence 22; 3, 7, 2, 11, from which it followed that 22 and only 22!! Someone jumped up, saying yes but 5, and what about 6, 18, or 4 for that matter; Hazelton countered this objection with the crushing retort that, either way, 22.”*

What type of futurological congress can we imagine today for the next century? The agenda would conceivably be a carbon copy of that from Lem’s assembly — today’s ecological, energy, political, and military crises differ in details but are no less daunting than in the 70’s — with some added sessions on cyber-security, bio-technology, and education. The scene could, however, look rather different. As the few delegates walk into the room — some virtually, some physically — their local AI platforms would interface with those of their colleagues from other countries.  The low-level discussions would then take place at the speed of the AI algorithms ping-ponging numbers and only occasionally requesting some high-level directives from the human delegates. (Doing A, may cause war with a 10% probability, but may otherwise increase our GDP by 5%, choose A?) Countries without AI capabilities would be excluded from the proceedings — after all, those countries would be mostly depopulated and mined for energy resources to run the personal AI assistants of first-world citizens. The representatives would be creative types with non-technical backgrounds, trained to make decisions quickly based on instinct, while the AI algorithms would take care of all practical aspects.

Or so we could be led to imagine based on a number of popular books and editorials by today’s experts and gurus. The last 30 years of software development may instead conjure up scenes of representatives scrambling to get their software to boot and algorithms to connect, of sessions lost to updates and bugs while trying to contact the lonely person who still remembers what the algorithms do and how they were programmed.

More seriously, at issue here is the understanding of what an algorithm is. As Steven Poole writes  on The Guardian, the “word sounds hi-tech, but in fact it’s very old: imported into English, via French and Latin, from the name of the ninth-century Arab mathematician al-Khwarizmi. Originally algorithm simply meant what is now called the “Arabic” system of numbers (including zero). […] To this day, algorithm is still just a fancy name for a set of rules. If this, then that; if that is true, then do this.” In other words, an algorithm does what it was programmed to do under the circumstances contemplated by the programmer. “If we thought of algorithms as mere humdrum flowcharts, drawn up by humans, we’d be better able to see where responsibility really lies if, or when, they go wrong.”

Using algorithms without understanding what they do and when is a form of  proceduralism, “the rote application of sophisticated techniques, at the expense of qualitative reasoning and subjective and judgment,” which may well lead to illogical, unethical, or even deadly outcomes. The problem is compounded by the fact that the “algorithms” that are considered today as AI are not made of logical sequences of if-then-else statements, but are rather sophisticated pattern recognition mechanisms that operate on large troves of data.

Here is to hoping that, prior to plugging an AI into the next futurological congress network, a representative would have to follow Hannah Fry’s suggestion to address Tony Benn’s five simple questions:

“What power have you got?
Where did you get it from?
In whose interests do you use it?
To whom are you accountable?
How do we get rid of you?”

* 22 meant “the end of the world”.

Make AI Do it

1200px-23rd_St_Park_Av_15_-_Make_Google_do_itWhat shall we do tonight? You ask.  Friend A is flexible — You choose — while Friend B has a strong opinion and lets you know it. Which friend is being kinder to you?

In “Algorithms to Live by” , the authors argue that Friend B is the more generous of the two:

“Seemingly innocuous language like ‘Oh, I’m flexible’ […] has a dark computational underbelly that should make you think twice. It has the veneer of kindness about it, but it does two deeply alarming things. First, it passes the cognitive buck: ‘Here’s a problem, you handle it.’ Second, by not stating your preferences, it invites the others to simulate or imagine them.”

If we allow that deciding on a plan for the evening is mostly a matter of computation, this computational kindness principle has evident merits and I, for one, am nodding in agreement. And so are also the big tech companies, all furiously competing to be your best Friend B. The last campaign by Google — Make Google do it. — makes this plain: The ambition is that of thinking for us — giving us directions, telling us what to buy, what to watch, where to go, whom to date, and so on.

The amount of cognitive offload from humans to AI-powered apps is an evident, and possibly defining, trend of our times. As pointed out by James Bridle in “New Dark Age: Technology and the End of the Future“, this process is accompanied by the spreading “belief that any given problem can be solved by the application of computation”, so that

“Computation replaces conscious thought. We think more and more like machine, or we do not think at all.

A typical way to justify our over-reliance on machines as surrogates for our own cognitive capacities is to point to the complexity of the modern world, which has been compounded by the inter-connectivity brought about by the Internet. This view echoes this prescient 1926 passage by H. P. Lovecraft as cited by Bridle:

The most merciful thing in the world, I think, is the inability of the human mind to correlate all its contents. We live on a placid island of ignorance in the midst of black seas of infinity, and it was not meant that we should voyage far. The sciences, each straining in its own direction, have hitherto harmed us little; but some day the piecing together of dissociated knowledge will open up such terrifying vistas of reality, and of our frightful position therein, that we shall either go mad from the revelation or flee from the light into the peace and safety of a new dark age.

But the effects of this unprecedented cognitive offload, even on our health, are at best unclear. Ominously, Vivienne Ming warns that, as the use of apps is depriving our brains of the exercise they have becomes used to over millions of years, we might see widespread early-onset dementia within a single generation.

Integrating Machine Learning and Communication Engineering

Image result for to be a machineA spectre is haunting conferences on communication and information theory — the spectre of machine learning.

Some find the (relatively) new tools a liberation from the time-consuming task of acquiring domain knowledge and advanced analytical skills. For this camp, machine learning shifts the engineering problem from the traditional model-based optimization of individual functionalities to an overall data-driven end-to-end design.

Others hold the reduction of communication engineering to the choice of objective functions and hyperparameters of general-purpose machine learning algorithms to be disturbing and demeaning. This second camp views machine learning as an excuse to cop out on the engineering responsibilities of ensuring reliable and verifiable performance guarantees.

And yet, to continue with the initial paraphrase: Where is the research group that has not attempted to apply machine learning to well-studied communication problems? Where is the communication researcher who has not tweaked hyperparameters so as to reproduce known engineering solutions?

Some perspective seems needed in order to find a beneficial synthesis between the two extreme viewpoints that may open new opportunities rather than transform communication engineering into a branch of computer science. To this end, I would propose that the use of machine learning in communications is justified when:

1) a good mathematical model exists for the problem at hand, and:

  • an objective function is hard to define or to optimize mathematically (even approximately);
  • it is easy to draw data samples from the model;
  • it is easy to evaluate desired pairs of well-specified input and output pairs (supervised learning), or to provide feedback on sequentially selected outputs (reinforcement learning);
  • the task does not required detailed explanations or performance guarantees;

or 2) a good mathematical model does not exist, and:

  • it is easy to collect real-world data;
  • the phenomenon under study does not change rapidly over time;
  • it is easy to evaluate desired pairs of well-specified input and output pairs (supervised learning), to assess the quality of the generated output (unsupervised learning), or to provide feedback on sequentially selected outputs (reinforcement learning);
  • the task does not required detailed explanations or performance guarantees.

Even when the use of machine learning methods is justified, the adoption of a data-driven approach does not entail the complete rejection of domain knowledge and engineering principles. As a case in point, the decoding of a dense channel code via efficient message passing schemes is known to fail in general. Nevertheless, the structure of a message passing algorithm can still be leveraged to define a parametrized scheme that can be tuned based on input-output examples.

More broadly, engineering a modern communication system is likely to call for a macro-classification of subtasks for which either model- or data-driven approaches are most suitable, followed by a micro-design of individual subproblems and by a possible end-to-end fine-tuning of parameters via learning methods.

Computer scientists and communication engineers of all countries, unite?

Snow Crash

The novel “Snow Crash” published in 1992 by Neal Stephenson, is often invoked as a prime example of fiction that has shaped, and not merely predicted, today’s technological landscape. The following lightly commented list of quotes from the first one hundred pages offer ample justification for the novel’s reputation. [Comments are in brackets.]

… all of the information got converted into machine-readable form, which is to say, ones and zeroes.  And as the number of media grew, the material became more up to date, and the methods for searching the Library became more up to date […] Millions […] are uploading millions of other fragments at the same time. […] clients, mostly large corporations and Sovereigns, rifle through the Library looking for useful information, and if they find a use for something that Hiro put into it, Hiro gets paid.

[Mosaic, the first popular web browser, was introduced only in 1993, and the first widely known search engine, WebCrawler, came out in 1994. Rather than relying on systems of payments as envisaged in the novel, the world wide web was built on an advertisement-based model. This system is currently being challenged by the changing legal and political climate, while new technologies for micro-payments are emerging.]

By drawing a slightly different image in front of each eye, the image can be made three-dimensional. By changing the image seventy-two times a second, it can be made to move. By drawing the moving three-dimensional image at a resolution of 2K pixels on a side, it can be as sharp as the eye can perceive, and by pumping stereo digital sound through the little earphones, the moving 3-D pictures can have a perfectly realistic soundtrack. So Hiro’s not actually here at all. He’s in a computer-generated universe that his computer is drawing onto his goggles and pumping into his earphones. In the lingo, this imaginary place is known as the Metaverse.”

[Metaverse, a term coined in the novel, refers to the idea of a shared virtual space. This concept has been extremely influential, inspiring game designers and VR developers.]

“It was, of course, nothing more than sexism, the especially virulent type espoused by male techies who sincerely believe that they are too smart to be sexists.”

[Sadly still relevant today.]

There is something new: A globe about the size of a grapefruit, a perfectly detailed rendition of Planet Earth, hanging in space at arm’s length in front of his eyes. […] It is a piece of […] software called, simply, Earth. It is the user interface that CIC uses to keep track of every bit of spatial information that it owns — all the maps, weather data, architectural plans, and satellite surveillance stuff.

[This ideas has reportedly inspired the development of Google Earth.]

“For the most part I write myself,” explains an automated online search engine. “That is, I have the innate ability to learn from experience. But this ability was originally coded into me by my creator.

[A good self-definition of a machine learning algorithm.]

Human-Imitative AI vs. Useful AI

Image result for Robot-Proof: Higher Education in the Age of Artificial IntelligenceIn a recent post, Michael I. Jordan distinguishes the following aspirations for AI-related research:

  • Human imitation: AI should behave in a way that is indistinguishable from that of a human being.
  • Intelligence Augmentation (IA): AI should augment human capacity to think, communicate, and create.
  • Intelligence Infrastructure (II): AI should manage a network of computing and communicating agents, markets, and repositories of information in a way that is efficient, supportive of human and societal needs, as well as economically and legally viable.

Human-imitative AI is often singled out by technologists, academicians, journalists and venture capitalists as the real aspiration and end goal of AI research. However, most progress in “AI” to date has not concerned high-level abstract thinking machines, but rather low-level engineering solutions with roots in operations research, statistics, pattern recognition, information theory and control theory.

In fact, as argued by Jordan, the single-minded focus on human-imitative AI has become a distraction from the more useful endeavor of addressing IA and II. To this end, the post calls for the founding of a new engineering discipline building on

ideas that the preceding century gave substance to — ideas such as “information,” “algorithm,” “data,” “uncertainty,” “computing,” “inference,” and “optimization.” Moreover, since much of the focus of the new discipline will be on data from and about humans, its development will require perspectives from the social sciences and humanities.”

The development of such a discipline would call not only for large-scale targeted research efforts, but also for new higher education programs. As proposed in “Robot-proof“, the new programs should impart data literacy, technological literacy, and human literacy:

Students will need data literacy to manage the flow of big data, and technological literacy to know how their machines work, but human literacy–the humanities, communication, and design–to function as a human being. Life-long learning opportunities will support their ability to adapt to change.

Information Theory is…

Image result for strand bookstore

[According to Google’s Talk to Books]

“Information theory is a branch of applied mathematics providing a framework allowing the quantification of the information generated or transmitted through a communication channel.”  from The Manual of Photography  by Elizabeth Allen, Sophie Triantaphillidou

“Information theory is a mathematical theory dealing with highly precise aspects of the communication of information in terms of bits through well-defined channels. The theory of international politics developed in this volume is non-mathematical and non-precise.” from System and Process in International Politics  by Morton A. Kaplan

“Information theory is a branch of science, mathematics, and engineering that studies information in a physical and mathematical context rather than a psychological framework.” from Assessing and Measuring Environmental Impact and Sustainability  by Jiri J Klemes

“Information theory is the science of message transmission developed by Claude Shannon and other engineers at Bell Telephone Laboratories in the late 1940s. It provides a mathematical means of measuring information.” from The Creation Hypothesis: Scientific Evidence for an Intelligent Designer by James Porter Moreland

Information theory is a branch of statistics and probability theory dealing with the study of data, ways to manipulate it (for instance, cryptography and compression) and communicate it (for instance, data transmissions and communication systems)”. from AI Game Development: Synthetic Creatures with Learning and Reactive Behaviors  by Alex J. Champandard

Information theory provides a means to quantify the complexity of information that can be used in the design of communication systems (Shannon 1948). It originated during World War II as a tool for assuring the successful transmission…” from Oceanography and Marine Biology: An Annual Review by R. N. Gibson, R. J. A. Atkinson, J. D. M. Gordon

And so on…