In 1971 — writing in what was then the Soviet Union, was previously Poland and would later become Ukraine — Stanisław Lem imagined “The Futurological Congress” Held in Costa Rica, planet Earth, the “first item of business [of the congress] was to address first the world urban crisis, the second — the ecology crisis, the third — the air pollution crisis, the fourth — the energy crisis, the fifth — the food crisis. Then, adjournment. The technology, military and political crises were to be dealt with on the following day”. The conference room was overcrowded with representatives from many countries, so, to “help expedite the proceedings, […] the lecturer would speak only in numerals, calling attention in this fashion to the salient paragraphs of his work. […] Stan Hazelton of the U.S. delegation immediately threw the hall into a flurry by emphatically repeating: 4, 6, 11, and therefore 22; 5, 9, hence 22; 3, 7, 2, 11, from which it followed that 22 and only 22!! Someone jumped up, saying yes but 5, and what about 6, 18, or 4 for that matter; Hazelton countered this objection with the crushing retort that, either way, 22.”*
What type of futurological congress can we imagine today for the next century? The agenda would conceivably be a carbon copy of that from Lem’s assembly — today’s ecological, energy, political, and military crises differ in details but are no less daunting than in the 70’s — with some added sessions on cyber-security, bio-technology, and education. The scene could, however, look rather different. As the few delegates walk into the room — some virtually, some physically — their local AI platforms would interface with those of their colleagues from other countries. The low-level discussions would then take place at the speed of the AI algorithms ping-ponging numbers and only occasionally requesting some high-level directives from the human delegates. (Doing A, may cause war with a 10% probability, but may otherwise increase our GDP by 5%, choose A?) Countries without AI capabilities would be excluded from the proceedings — after all, those countries would be mostly depopulated and mined for energy resources to run the personal AI assistants of first-world citizens. The representatives would be creative types with non-technical backgrounds, trained to make decisions quickly based on instinct, while the AI algorithms would take care of all practical aspects.
Or so we could be led to imagine based on a number of popular books and editorials by today’s experts and gurus. The last 30 years of software development may instead conjure up scenes of representatives scrambling to get their software to boot and algorithms to connect, of sessions lost to updates and bugs while trying to contact the lonely person who still remembers what the algorithms do and how they were programmed.
More seriously, at issue here is the understanding of what an algorithm is. As Steven Poole writes on The Guardian, the “word sounds hi-tech, but in fact it’s very old: imported into English, via French and Latin, from the name of the ninth-century Arab mathematician al-Khwarizmi. Originally algorithm simply meant what is now called the “Arabic” system of numbers (including zero). […] To this day, algorithm is still just a fancy name for a set of rules. If this, then that; if that is true, then do this.” In other words, an algorithm does what it was programmed to do under the circumstances contemplated by the programmer. “If we thought of algorithms as mere humdrum flowcharts, drawn up by humans, we’d be better able to see where responsibility really lies if, or when, they go wrong.”
Using algorithms without understanding what they do and when is a form of proceduralism, “the rote application of sophisticated techniques, at the expense of qualitative reasoning and subjective and judgment,” which may well lead to illogical, unethical, or even deadly outcomes. The problem is compounded by the fact that the “algorithms” that are considered today as AI are not made of logical sequences of if-then-else statements, but are rather sophisticated pattern recognition mechanisms that operate on large troves of data.
Here is to hoping that, prior to plugging an AI into the next futurological congress network, a representative would have to follow Hannah Fry’s suggestion to address Tony Benn’s five simple questions:
“What power have you got?
Where did you get it from?
In whose interests do you use it?
To whom are you accountable?
How do we get rid of you?”
* 22 meant “the end of the world”.