Edward Snowden on the Dangers of Mass Surveillance and Artificial General Intelligence

Snowden doesn’t feature in the film, but it couldn’t exist without him: “iHuman” is an almost exhausting journey through all the issues that Snowden was trying to warn us about, starting with our civil liberties. Speaking after the film—which he “very much enjoyed”—Snowden admitted that the subject was still raw for him, and that the writing of his autobiography (this year’s “Permanent Record”), had not been easy. “It was actually quite a struggle,” he revealed. “I had tried to avoid writing that book for a very long time, but when I looked at what was happening in the world and [saw] the direction of developments since I came forward [in 2013], I was haunted by these developments—so much so that I began to consider: what were the costs of silence? Which is [something] I understand very well, given my history. When you see the rise of authoritarianism—even in Western, open societies—and you see how closely it dovetails with the development of technology that create stable states rather than free states, I think that should alarm us, and that drove me quite strongly in my work.”

Snowden used the example of the changing nature of surveillance. “Before 2013,” he noted, “there were specialists, there were insiders, there were intelligence officers, there were academics and researchers who understood all too well the possibility of mass surveillance. They understood how our technologies and our techniques could be applied to change the world of intelligence gathering from the traditional method—which was, you name a target and you monitor them specifically. You send officers into their homes. They plant a camera or a listening device. You have officers on the street who follow them to meetings, in cars and on foot. It was very expensive. And that created a natural constraint on how much surveillance was done. The rise of technology meant that, now, you could have individual officers who could now easily monitor teams of people and even populations of people—entire movements, across borders, across languages, across cultures—so cheaply that it would happen overnight.”

At the NSA, he continued, “I would come to my desk in the morning and all the information was already there. This was the burden of mass surveillance. Now, as I said, specialists knew this was possible, but the public was not aware, broadly [speaking], and those who claimed that it was happening, or even that it was likely to happen, were treated as conspiracy theorists. You were the crazy person [in] the tin foil hat. The unusual uncle at the dinner table. And what 2013 delivered, and what I see the continuation of today, is the transformation of what was once treated as speculation—even if it was informed speculation—to fact.”
Returning to the theme of whistleblowing, Snowden reaffirmed his belief that mostly it is a moral obligation. “It’s not about what you want,” he said flatly. “It’s about what we must do. The invention of artificial general intelligence is opening Pandora’s Box—and I believe that box will be opened. We can’t prevent it from being opened. But what we can do is, we can slow the process of unlocking that box. We can do it by days. We can do it by decades, until the world is prepared to handle the evils that we know will be released into the world from that box. And the way that we do that, the way that we slow that process of opening the box, is by removing the greed from the process, which I believe is the primary driver for the development of so much of this technology today.”

He continued: “We should not, and we must not, ban research into machine learning and artificial intelligence techniques that have human impact. But we can, and we should, ban the commercial trade in these technologies at this stage. And what that will do is it means that academic researchers—public interest organizations, the scientists and researchers who are driven by the public interest [and] the common good—will continue their work. But all of the companies that are doing this now hold it from these that are pursuing these capabilities to amplify their own power and profits, they will be deterred, because they will have less incentive to do these things now.”

Warming to his theme, Snowden reserved the full blast of his disdain for the likes of Google, Amazon, Facebook and companies such as Cambridge Analytica, that track our digital footprints and use algorithms to grab our attention. “What is happening is that we are being made prisoner to ghosts,” he said. “We are being imprisoned by models of [our] past behavior that have been determined by machines. We are being used against the future. Our past actions and activities are being used to limit the potential of human behavior, because decisions are being formed based on past observations and these models of past lives.

“[This kind of information] can’t be misused,” he stressed. “It must not be misused to decide who gets a job, who gets an education, who gets a loan, who gets [medical] treatment. But if we don’t change the direction that we see today, if we allow Facebook and Google and Amazon to pursue these models and to apply these models to every aspect of human decision-making—as they are very, very aggressively striving to [do] today. We will find [that] we have become prisoners of a past that no longer exists.”