Information, Knowledge, Wisdom and 5G

bestdoc-535x300One of the most compelling conceptual visions for 5G contrasts the user-driven information-centric operation of previous generations with the industry-driven knowledge-centric nature of the upcoming fifth generation. According to this vision, the evolution from 1G to 4G has been marked by the goal of enhancing the efficiency of human communication — with end results that we are still trying to understand and manage. In contrast, 5G will not be aimed at channeling tweets or instantaneous messages for human-to-human communications, but at transferring actionable knowledge for vertical markets catering to the healthcare, transportation, agriculture, manufacturing, education, automation, service and entertainment industries. In other words, rather than carrying only information, future networks will carry knowledge and skills. Whose knowledge and whose skills will be amplified and shared by the 5G network infrastructure?

Two options are typically invoked: learning machines (AI) and human experts. AI is widely assumed to be able to produce actionable knowledge from large data sets solely for tasks that require systematic, possibly real-time, pattern recognition and search operations. Typical examples pertain the realm of the Internet of Thing, with data acquired by sensors feeding control or diagnostic mechanisms. AI is, however, still very far from replicating the skills of human experts when it comes to “instinctive intelligence“, making multi-faceted judgements  based on acquired “wisdom“, innovation, relating to other humans, providing advice, offering arguments, and, more generally, performing complex non-mechanical tasks. Therefore, human experts can complement the knowledge and skills offered by AI. A scenario that is consistently summoned is that of a surgeon operating on a patient remotely thanks to sensors, haptic devices and low-latency communication networks.

By sharing knowledge and skills of AI and human experts, 5G networks are bound to increase the efficiency and productivity of learning machines and top professionals, revolutionizing, e.g., hospitals, transport networks and agriculture. But, as a result, 5G is also likely to become a contributor to the reduction of blue– and white-collar jobs and to the widening income gap between an educated elite and the rest. This effect may be somewhat mitigated if more optimistic visions of a post-capitalist economic system, based on sharing and collaborative commons, will be at least partly realized thanks to the communication substrate brought by 5G.

Tomorrow Never Knows


It is unquestionably a time of uncertainty, and visions of the not-too-far future come in all flavors. Here is a partial list:

  • Friedmans, or techno-optimists: New technologies for communication, computation, energy production and transportation will raise productivity and reduce costs, breaking the logic of scarcity and ushering in a new economic system based on sharing and cooperation;
  • Merkels, or business-as-usuals: Capitalism, liberalism and democracy will naturally prevail and progress will continue unimpeded;
  • Mandibles, or stone-agers: The over-reliance on networked devices will lead to a catastrophic breakdown of the communication and financial infrastructures as a result of cyber-wars, making advanced economies unable to provide for themselves;
  • Hararis, or techno-pessimists: Intelligent machines will take over jobs and functions of “regular” humans, and a new minority of super-human cyborgs will emerge beyond the point of technological singularity;
  • Realists, or climate catastrophists: There is really no future for humankind on Earth;
  • Cixins, or escapists: The future for humanity is in space — at least if you can pay the ticket.

It from Bit

6261055049_26244e9348_bIn most classes on information theory (IT), the relationship between IT and physics is reduced to a remark on the origin of the term “entropy” in Boltzmann’s classical work on thermodynamics. This is possibly accompanied by the anecdote regarding von Neuman’s quip on the advantages of using this terminology. Even leaving aside recent, disputed, attempts, such as constructor theory (see here) and integrated information theory (see here), to use concepts from IT as foundations for new theories of the physical world, it seems useful to provide at least a glimpse of the role of IT in more mainstream discussions on the future of theoretical physics.

As I am admittedly not qualified to provide an original take on this topic, I will rely here on the poetic tour of modern physics by Carlo Rovelli, in which one of the last chapters is tellingly centered on the subject of “information”. Rovelli starts his discussion by describing information as a “specter” that is haunting theoretical physics, arousing at the same time enthusiasm and confusion. He goes on to say that many scientists suspect that the concept of information may be essential to make progress in theoretical physics, providing the correct language to describe reality.

At a fundamental level, information refers to a correlation between the states of two physical systems. A physical system, e.g., one’s brain, has information about another physical system, e.g., a tea cup, if the state of the tea cup is not independent of that of the neurons in the brain. This happens if a state of the tea cup, say that of being hot, is only compatible with a subset of states of the brain, namely those in which the brain has memorized the information that the tea cup is hot. Reality can be defined by the network of such correlations among physical systems. In fact, nature has evolved so as to manage these correlations in the most efficiency way, e.g., through genes, nerves, languages.

The description of information in terms of correlation between the states of physical systems is valid in both classical and quantum physics. In thermodynamics, the missing information about the microstate of a system, e.g., about the arrangement of the atoms of a tea cup, given the observation of its macrostate, e.g., its temperature, plays a key role in predicting the future behavior of the system. This missing information is referred to as entropy. In more detail, the entropy is the logarithm of the number of microstates that are compatible with a given macrostate. The entropy tends to increase in an isolated system, as information cannot materialize out of thin air and the amount of missing information can only grow larger in the absence of external interventions.

In quantum physics, as summarized by Wheeler’s “It from Bit” slogan, the entire theoretical framework can be largely built around two information-centric postulates: 1) In any system, the “relevant” information that can be extracted so as to make predictions about the future is finite; 2) Additional information can always be obtained from a system, possibly making irrelevant previously extracted information (to satisfy the first postulate).

The enthusiasm and confusion aroused by the concept of information among theoretical physicists pertain many fundamental open questions, such as: What happens to the missing information trapped in a black hole when the latter evaporates? Can time be described, as suggested by Rovelli, as “information we don’t have”? Related questions abound also in other scientific fields, such as biology and neuroscience: How is information encoded in genes? What is the neural code used by the brain to encode and process information?


The Library


Imagine a library — a real one, with actual books — with an unusual rule: no novels, essays or magazines are allowed, but only private journals, diaries, thoughts, rants, speculation, accusations, and any “true, authentic documents reflecting the real spirit of the people.” The library is open to the public, and each document can be read by any visitor, who can also request for a small fee to be informed about the identity and the address of the author.

This scenario, which is eerily prescient of today’s social media, was imagined by an Italian novelist in 1975, at the height of the Years of Lead. The novelist, Giorgio De Maria, writes in “The twenty days of Turin” that the appeal of the library derived by the prospect of being read by others, ideally creating a social web of connections and relationships.

But the social impact of the library turns out to be quite different from these lofty expectations, as the library ends up fostering a community of paranoid, resentful and isolated prosumers of information. As per Max Weber‘s prediction, in De Maria’s Turin, progress in communication technologies pushes the individual away from public life and into a “subjectivist culture” of “sterile excitation”.

The denouement of the novel sees old ideas and myths, in the form of monuments, come back to life, somehow resuscitated by the energy channeled by the community’s desperation. A bleak vision, ominously close to our present.

Spiking Neural Networks and Neuromorphic Computing

Brain_Chip_Wide.jpgDeep learning techniques have by now achieved unprecedented levels of accuracy in important tasks such as speech translation and image recognition, despite their known failures on properly selected adversarial examples. The operation of deep neural networks can be interpreted as the extraction, across successive layers, of approximate minimal sufficient statistics from the data, with the aim of preserving as much information as possible with respect to the desired output.

A deep neural network encodes a learned task in the synaptic weights between connected neurons. The weights define the transformation between the statistics produced by successive layers. Learning requires updating all the synaptic weights, which typically run in the millions; and inference on a new input, e.g., audio file or image, generally involves computations at all neurons. As a result, the energy required to run a deep neural network is currently incompatible with an implementation on mobile devices.

The economic incentive to offer mobile users applications such as Siri has hence motivated the development in recent years of computation offloading schemes, whereby computation is migrated from mobile devices to remote servers accessed via a wireless interface. Accordingly, user’s data is processed on servers located within the wireless operator’s network rather than on the devices. This reduces energy consumption at the mobiles, while, at the same time, entailing latency — a significant issue for applications such as Augmented Reality — and a potential loss of privacy.

The terminology used to describe deep learning methods — neurons, synapses — reveals the ambition to capture at least some of the brain functionalities via artificial means. But the contrast between the apparent efficiency of the human brain, which operates with five orders of magnitude (100,000 times) less power than current most powerful supercomputers, and the state of the art on neural networks remains jarring.

Current deep learning methods rely on second-generation neurons, which consist of simple static non-linear functions. In contrast, neurons in the human brain are known to communicate by means of sparse spiking processes. As a result, neurons are mostly inactive and energy is consumed sporadically and only in limited areas of the brain at any given time. Third-generation neural networks, or Spiking Neural Networks (SNNs), aim at harnessing the efficiencies of spike-domain processing by building on computing elements that operate on, and exchange, spikes. In an SNN, spiking neurons determine whether to output a spike to the connected neurons based on the incoming spikes.

Neuromorphic hardware is currently being developed that is able to natively implement SNNs. Unlike traditional CPUs or GPUs running deep learning algorithms, processing and communication is not “clocked” to take place across all computing elements at regular intervals. Rather, neuromorphic hardware consists of spiking neurons that are only active in an asynchronous manner whenever excited by input spikes, potentially increasing the energy efficiency by orders of magnitude.

If the promises of neuromorphic hardware and SNNs will be realized and neuromorphic chips will find their place within mobile devices, we could soon see the emergence of revolutionary new applications under enhanced privacy guarantees.

White Light/ White Heat

0508-bks-ferriscvr-master768-v2The latest novel by DeLillo may be about the fear of life and the fear of death and about the role that technology plays in activating and/or defusing both. In a previous novel, a character opined that

This is the whole point of technology. It creates an appetite for immortality on the one hand. It threatens universal extinction on the other.

The key mechanism behind the disruption and distress caused by technology in “Zero K” appears to be virtualization:

Haven’t you felt it? The loss of autonomy. The sense of being virtualized. The devices you use, the ones you carry everywhere, room to room, minute to minute, inescapably.

Virtualization refers to the realization of something — typically an operating system, a server or a network — on a different physical substrate, so that the virtual copy retains the main features (virtues) of the original and is indistinguishable from it. In (my interpretation of) DeLillo’s vision, the virtual copies of our selves stored on digital devices have become more real and relevant than the original.

In “Zero K”, escape, at least for the wealthy, is found in a cryogenically induced isolated state of pure thought after death. This state may be just another form of virtualization, but one that is out of time rather than ticking at the speed of Twitter updates. Waiting for the end of the world to bring better times.

My Generation


As the 3GPP standardization body continues its work towards the specification of the fifth generation (5G) of cellular systems, it is instructive to take a look at the current coverage map for the previous generations.  As of the end of 2016, according to the map above, a number of countries, including Ukraine, Mongolia, Afghanistan, Myanmar, Yemen, Syria and Libya, had only 3G coverage; while others, such as Central African Republic, Chad, Niger and Eritrea were only served by 2G (GSM) operators. Will 5G deepen the chasm between straggling economies and more technologically advanced nations, or will it instead provide shared benefits across the board?

The argument for the first scenario is clear: 5G is mostly envisaged as a platform to connect things, such as vehicles, robots and appliances, hence catering to vertical markets but neglecting the more basic needs of countries with limited broadband connectivity. The second, more optimistic, scenario is instead backed by the idea that “developing” countries could leapfrog previous “Gs” by leveraging novel architectures based on technologies such as wireless backhauling, small-cells and energy harvesting. This would allow them to benefit from the 5G-enabled connectivity among things for applications as diverse as smart transportation systems, e-health (e.g., remote surgery), remote learning and sensor networks for water management and agriculture.

It was suggested that, in addition to the three basic services currently defined for 5G, namely massive Machine Type Communication, enhanced Mobile Broadband, and Ultra Reliable Low Latency Communication,  an Ultra Low Cost Broadband service should be made part of the standard. Apart from isolated efforts by companies, such as Google X’s Project Loon and Facebook’s Aquila project, this laudable idea seems to have been mostly forgotten as of late, although the “frugal 5G” concept recently announced by the IEEE appears to be finally moving in this direction.

Choose Something Like a Star

darkforestIn front pages and news feeds cluttered with instantaneous tweet-sized proclamations and reactions, the announcement earlier this week that astronomers discovered seven Earth-sized planets that might be able to sustain life was difficult to categorize and easy to dismiss. But what if life was indeed found in one of these planets?  How would our public discourse change?

It is likely that a part of the world population would treat this news as another scientific “hoax” unworthy of further consideration, but it is hard to imagine that society as a whole, as well as national and international institutions, would be unaffected. Would there be popular movements advocating for escapist space explorations (perhaps facetiously) or even for an alien takeover with religious undertones? Would politicians be able to continue running on nationalist platforms centered on curbing immigration? Would the international community come together to face the threats, challenges and opportunities posed by unknown life forms (as in “The dark forest“)? Or would instead nations compete for scientific, or possibly colonial, dominance in a repeat of the European experience at the onset of the modern age?

(The title of this post is that of Robert Frost’s poem invoked by Dan Rather as a commentary to the news discussed here. The poem includes the following: “It asks of us a certain height,/ So when at times the mob is swayed/ To carry praise or blame too far,/ We may choose something like a star/ To stay our minds on and be staid.“)

Paying Our Dues


The powerful documentary “I am not your negro” by Raoul Peck recently introduced me to the important figure of James Baldwin. By juxtaposing recent events with clips of his interviews and excerpts from his final project, the film provides a striking demonstration of the relevance of Baldwin’s words on the state of racial relations in today’s United States. The reaction of part of the public to Peck’s work further attests to the timeliness of Baldwin’s ideas on this subject. The documentary also offers a glimpse on other, but not all, aspects of this thinking.

While, as Baldwin writes, “When one begins to live by habit and by quotation, one has begun to stop living“, a number of quotes from his writing on culture, progress and information, are worth reproducing here:

— “The paradox of education is precisely this – that as one begins to become conscious, one begins to examine the society in which he is being educated.

— “No people come into possession of a culture without having paid a heavy price for it.

—  “It was books that taught me that the things that tormented me most were the very things that connected me with all the people who were alive, or who had ever been alive.”

— “No one can possibly know what is about to happen: it is happening, each time, for the first time, for the only time.”

Halt and Catch Fire


(Halt and catch fire, Season 3, Episode 3)

We are in the late 80s, at a time when the commercial Internet had yet to be born out of the ARPANET and the NSFNET. The setting is a conference room at a small start-up in Silicon Valley that runs a bulletin board system, whereby users can connect via dial-up to exchange messages and trade goods  (all the while being represented as sprites seemingly inspired by Maniac Mansion).

The managers of the start-up are discussing how to improve user experience by finding the right compromise between processing at the users’ computers and at the company’s servers. The ensuing conversation should resonate with today’s engineers and researchers working on the optimal functional allocation between edge and cloud of 5G cellular systems:

— “Okay, up next, how are we doing on the speed of the background graphics? Well, we’ll never get under half a second at 2,400 baud. We’re gonna need to use Huffman, or even better, Lempel-Ziv compressions so we’re not sending all the bits through.

— “Okay, since when are we doing a graduate seminar on information theory? Our guys can’t handle that. No, no, no. That’s smart not sending all the bits through. We preload the most common backgrounds on the diskettes users already have and just send the catalog numbers. Okay, so just send the index to the scene.

— “Okay, that’s good. I see that. That’s good, right? No complex coding. Well, you’ll have a limited set of images and the user will get tired of waiting for the same-old same-old, but, yeah, it’s great if that’s what you guys want to do. Great.