Of note in relevance to this particular video are two points:

First, it is narrow to focus on only the “terrorist threats” that are opened up by the development of this sort of technology, and unrealistic to expect that military research in such areas will be used only for “defence” purposes. Kurzweil elides the fact that the very people he is working with to ostensibly combat the pitfalls of this technology (the US military) are the most likely to use it.

Secondly, Kurzweil is right to say that these advances pose an “existential” threat to us, but not for the reasons he believes. A threat is not existential insofar as it threatens our existence, but rather (if I can venture an off-the-cuff formulation) insofar as it threatens our very humanity. The fantasy of a set of artificial intelligences that are of equal or greater intelligence than humans,  which have perfect recall of facts and figures, and which lack many if not all of the physicality that at least in part constitutes human-ness should raise serious questions about what implications the production of such an intelligence might have for human decision-making, politics and freedom in general.

Jurgen Habermas and others have already indicated ways in which genetic engineering poses a threat to human autonomy, and robotics of the sort envisioned by Kurzweil should also provoke similar concerns.

While SALT has hosted a bevy of fascinating and influential guests, including Craig Venter, Jimmy Wales, Francis Fukuyama, and Ray Kurzweil, Daemon: Bot-Mediated Reality by author and software engineer Daniel Suarez was one of the most interesting and thought-provoking (mp3 here).

Suarez describes our world as being increasingly run, unconsciously, by an increasing number of bots, defined as “narrow AI”: robots that are very good at doing one, narrowly defined thing, like scraping auto insurance rates from websites, deciding whether you qualify for a mortgage, and so on. The scenario of increasing automation raises a host of troubling questions about the possibility of privacy (this on the heels of an announcement of the UK’s intention to scan every single email correspondence) in a world where proliferating electronic devices silently communicate your information and daily activity to each other, opening up more and more points of entry vulnerable to exploitation and digital eavesdropping.

But more worryingly, and more interestingly, is the spectre of a society in which decisions are made by closed-source algorithms (bots), the inner workings and logic of which is unknown to the majority of humanity. This, says Suarez, leaves societies open to domination by a small set of people who control these bots or understand how they work (here Suarez, I think somewhat illegitimately, conflates this prospect with the worrying but not intrinsically related phenomenon of the growth of malicious Internet-based botnets).

Suarez’s proposed solution–or, better, proposed avenue of exploration–is a sort of open-source operating system for democracy, in which decision making is fully decentralized, and open source bots are used simply to filter and sort information which is then decided upon and acted upon by three levels of people: average citizens, recognized experts, and then a third circle of even more respected experts.

For instance, people receive news from bots of an environmental spill on their mobile devices; they read it and classify it, as part of their civic duty; bots pass on the info to experts (as determined by social-network-like authority/ranking systems), who enact another layer of decision-making; bots then pass this information on to a final (set of) decision maker(s) who allocates necessary human and financial resources to resolve the issue, all of this taking place without any centralized government.

Whether this could still be called democracy is of course up for debate. Before this sort of technology, an advanced technocracy was perhaps not possible or viable without representative democracy, but Suarez paints a fairly convincing, if sketchy and in outline form only, picture of what such a system would look like.

I have to say that the proposed solution is not a whole lot less concerning than the alternative of a world run by bots and tyrants. In any case, the ideas are fascinating, and credit must be given to Suarez for formulating them and linking them together in such a creative and challenging way.

One of the lectures I’m particularly looking forward to downloading is the recent Daemon: Bot-Mediated Reality by Daniel Suarez, which I think has particular relevance given the recent and rather large f-up in which Google’s news crawler inadvertently “evaporated $1.14B USD”.

Unfortunately, I think that in the near future, as more and more processes are automated, we will see more such screw-ups of this scale. I can’t help but think that this might have been avoidable, though, if the indexing engine had been able to take advantage of semantic data rather than relying on scraping and evaluating natural language.

Nerdy and awesome

Though these friendly looking little guys were pulled from operation, there is no indication that the MQ-9 Reaper airborne wardroids (aka bringers of death from above) have been retired.

As usual in the US military, clear heads prevail.

*UPDATE* apparently this was a bit of an internet hoax, and the guns did not in fact accidentally aim at humans… according to the defense contractors who made the robots–whose credibility, incidentally, I do not doubt for one instant. Anyone who is wise enough to put guns on semi-autonomous robots is surely to be unquestioningly trusted.