Behaviour modification empires for rent

Of all the “what the hell is going on?” type books that I’ve read in the last few years, the one I enjoyed most was Jaron Lanier’s Ten Arguments For Deleting Your Social Media Accounts.

The title undersells this book’s importance, to my mind. After all, it’s neither self-help nor “clickbait” – it’s not like “10 arguments for quitting sugar”. I regard it more as an absolutely essential collection of insights (from a Silicon Valley insider) about why basic democratic and progressive norms seem to be undermined as a consequence of how social media works.

“But for the moment we face a terrifying, sudden crisis…
Something is drawing young people away from democracy.”
(Jaron Lanier, Ten Arguments…)

Algorithm Politics & mass manipulation of humans

“The short-term, dopamine-driven feedback loops we’ve created are destroying how society works.”
– Former Facebook vice president of user growth (quoted by Lanier)

Jaron’s book argues that while we should generally embrace the internet, we need to urgently reject what he calls “BUMMER” (his acronym for the destructive core of social media, short for “Behaviors of Users Modified and Made into an Empire for Rent”).

BUMMER is a sort of high-level business plan in which the end-users of social media are the product, not the customer (that’s why social media is free to use). The real customers are those who want to modify your behaviour in some way. The basic argument is that, statistically, social media algorithms boost certain negative aspects of human communication, since that’s what maximises engagement with the platform (thus maximising profit for the social media companies).

The algorithms don’t care how they maximise user engagement – it happens automatically (continually “optimised”), and it just so happens that tribalism and nasty adversarial conflicts tend to engage people more efficiently than, say, pleasantly reasonable discourse does. Nor do the algorithms care if the result is user addiction (with its related mental health problems).

“Social media is biased, not to the Left or the Right, but downward. The relative ease of using negative emotions for the purposes of addiction and manipulation makes it relatively easier to achieve undignified results. An unfortunate combination of biology and math favors degradation of the human world. Information warfare units sway elections, hate groups recruit, and nihilists get amazing bang for the buck when they try to bring society down.

“The unplanned nature of the transformation from advertising to direct behavior modification caused an explosive amplification of negativity in human affairs.” (Lanier, Ten Arguments…)

As the book frames it: “Social media is turning you into an asshole”. I’m reminded of the quote provided by Robert Anton Wilson at the beginning of his chapter on “The SNAFU principle” in Prometheus Rising:

“…the peculiar nature of the game…makes it impossible for [participants] to stop the game once it is under way. Such situations we label games without end.” (Watzlawick, Beavin, Jackson, Pragmatics of Human Communication – full quote here)

As for those who want to modify your behaviour, they range from advertisers to malign (and often secretive) parties seeking to amplify hatreds or swing elections. (Lanier doesn’t shy away from tackling emotive/controversial topics, such as Russian state exploitation of social media for disruptive purposes).

“Remember how it became cool in some liberal circles to cruelly ridicule Hillary, as if doing so were a religion? In the age of BUMMER you can’t tell what was organic and what was engineered.

“It’s random that BUMMER favored the Republicans over the Democrats in U.S. politics, but it isn’t random that BUMMER favored the most irritable, authoritarian, paranoid, and tribal Republicans. All those qualities are equally available on the left.” (Lanier, Ten Arguments…)

(Remember when Facebook promoted the “trending news” that “most doctors polled” had “serious concerns” about Hillary Clinton’s health, including the suggestion, in a poll question, that Hillary was a “flaming psychopath”? This “news” originally came from a rightwing group, AAPS, that promoted conspiracy theories, including that “vaccines cause autism“. It was also promoted by Trump and Wikileaks).

I found Lanier’s book to be an entertaining read, rich in insights (and in things you need to know about) – I recommend you read the whole thing for yourself. The bottom line is that the algorithms constantly monitor, via our online responses, preferences, framing, etc, the micro-level views/behaviours (you could call it the result of our “adaptive” unconsciouses) of hundreds of millions of people on an individual, targeted level (via their personalised social media feeds and searches), instantaneously in real time – modifying behaviour (so Lanier argues), in ways we’re not conscious of, and at the whim of parties who don’t have our best interests in mind.

Those algorithms? Lanier remarks that they’re among the best kept secrets on the planet – more carefully guarded than NSA or CIA state secrets. It’s worth quoting at length one example of how the book describes them as working:

“Black activists and sympathizers were carefully cataloged and studied. What wording got them excited? What annoyed them? What little things, stories, videos, anything, kept them glued to BUMMER? What would snowflake-ify them enough to isolate them, bit by bit, from the rest of society? What made them shift to be more targetable by behavior modification messages over time? The purpose was not to repress the movement but to earn money. The process was automatic, routine, sterile, and ruthless.

“Meanwhile, automatically, black activism was tested for its ability to preoccupy, annoy, even transfix other populations, who themselves were then automatically cataloged, prodded, and studied. A slice of latent white supremacists and racists who had previously not been well identified, connected, or empowered was blindly, mechanically discovered and cultivated, initially only for automatic, unknowing commercial gain – but that would have been impossible without first cultivating a slice of BUMMER black activism and algorithmically figuring out how to frame it as a provocation.

“BUMMER was gradually separating people into bins and promoting assholes by its nature, before Russians or any other client showed up to take advantage. When the Russians did show up, they benefited from a user interface designed to help ‘advertisers’ target populations with tested messages to gain attention. All the Russian agents had to do was pay BUMMER for what came to BUMMER naturally.” (Jaron Lanier, Ten Arguments…)

Update: I recommend watching The Great Hack (a new Netflix documentary), as it makes some of the same points that Lanier does about the urgency of the situation. It covers the threat to democracy posed by the new kind of “weapons grade” psychological propaganda that’s researched (and used) by entities such as Cambridge Analytica and SCL, using social media data mining, etc.

By the way, I’m aware that descriptions of this material (including my own, probably) sometimes sound a bit like paranoid sci-fi melodrama. Even the more sober reports often add to that effect. Read about the interventions of SCL Group (Cambridge Analytica’s parent company) in the 2010 elections in Trinidad and Tobago, for instance.