Can Humans be Hacked?: A Semi-Technical Investigation Into Whether Artificial Intelligence Can Control Our Minds (Yet)

[[This article appears in Issue One of The New Modality. Buy your copy or subscribe here.]]

Yuval Noah Harari, historian and author of bestselling books on humanity’s evolution such as Sapiens, recently warned of the “ability to hack humans” at an event hosted by the Stanford Institute for Human-Centered Artificial Intelligence and documented by Wired. Harari says that AI, especially when linked with biological knowledge, may allow us to produce “an algorithm that understands me better than I understand myself, and can therefore manipulate me, enhance me, or replace me.” 

Every new technology can amplify power and control, and every new medium has carried a threat of malevolent persuasion, from print to radio to television. Yet data-driven, individually-targeted online messaging seems like something new. 

For many, this first got real when the news story about Facebook and Cambridge Analytica erupted in 2017. Recap: Cambridge Analytica (CA) compiled a database of tens of millions of Facebook profiles. They collected this data at a time when connecting Facebook to another app revealed not just your information, but information on all your friends. They used this data to build models to predict personality from Facebook profiles — or at least, to guess scores on a standard psychology test. The personality results were then used to customize Republican political messages in the 2016 election. CA claimed they could tell each individual prospective voter whatever would be most persuasive for that person. 

Alexander Nix, CEO of CA, described how this allegedly works in a January 2017 interview with Vice. To understand the methodology, let’s consider two strategies for marketing the concept of gun rights: 

• One could use the threat of burglary, and a photo of a break-in, “for a highly neurotic and conscientious audience;” 
• On the other hand, one could use an image of a father and son hunting together at sunset “for a closed and agreeable audience... who care about tradition, and habits, and family.” 

The idea is to figure out which formulation of the message would inspire each person, and then deliver the message with enough frequency or force — enough, but not too much, or the target audience might get suspicious and defensive — to make the audience act differently in the real world. This plausible-sounding approach is hard to test, and Cambridge Analytica has never offered public evidence that it works.

Despite this lack of evidence, many scary articles were published about CA’s “psychological warfare mindfuck tool” (The Guardian, 2018). After the scandal got kicked around for a while, David Karpf of the George Washington University School of Media and Public Affairs wrote a thoughtful review of the situation for Civic Hall, in which he argues against the idea of a “secret plan through weaponized online propaganda.” Karpf notes that CA’s outrageous development costs were typical of vaporware (i.e., software that is announced with great fanfare but never actually materializes); that most favorable testimony about CA’s effectiveness comes from experts with careers staked on psychometrics; and that the company wouldn’t resort to bribes and honeypots, as reported by The Atlantic, to tempt their customers if their tech worked. 

Even Alex Kogan, a data scientist who helped CA with initial data collection, “didn’t believe that Cambridge Analytica, or anyone else, could produce an algorithm that effectively classified people’s personality,” according to the book Outnumbered by math professor David Sumpter. The most charitable interpretation of the CA situation is a gap between theory and practice, which is often enormous in artificial intelligence. 

Still, Facebook data does say something about our personality. At least, it says something about our Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism (a.k.a. the OCEAN or “Big 5” personality traits widely used in psychology research). In 2012, a team of Microsoft and Cambridge researchers tried to deduce personality traits from the Facebook data of 180,000 people. They found meaningful correlations between some OCEAN scores and user profile metrics, that is, a person’s number of Facebook friends, groups, likes, photos, statuses, and tags in others’ photos. 

The strongest correlation between Facebook stats and personality scores is intuitive: As a user’s relative number of friends increases, their relative Extraversion score does too. But the researchers learned that only Extraversion and Neuroticism could be predicted by an algorithm using Facebook data better than a human could guess. Plus, even when personality correlations are reliably detectable, they don’t necessarily give any persuasive power. What matters is not just what machines can learn about us, but what they can do with that knowledge. 

Most of what we know about online persuasion comes from advertising, such as the classic “banner ad” (the picture ads you see on most websites). In 2009, Jun Yan at Microsoft Research Asia led a study on maximizing effectiveness of banner ads with behavioral targeting and fine-grain user segmentation. Across 6 million search users and 18,000 real world ads, Yan’s team increased the probability of users clicking on the ads by 670%, and even up to 1,000% with more sophisticated techniques. These improvement rates may be less impressive than they seem, because the original click rates hover below 1% — i.e., at most, the ads got clicked 10 times out of every 1,000 times they were viewed — and few studies in the open literature report the absolute numbers.

Hence, tiny improvements can double or triple an advertiser’s sales, but it’s probably not the sort of revolution in persuasion worthy of a dystopian description like “psychological warfare mindfuck tool.”

Individualized targeting seems to be more powerful on social networks, perhaps because they know more about us, or perhaps because they are already more engaging spaces: Click rates on social networks are ten times higher than banner ads at baseline, even before research teams get involved. Personalization can increase click rates further. Pinterest — the third largest social network in the US, and one of the few that shares details about personalization strategies — reports a 48% jump in engagement from their latest effort to improve the relevance of recommended content. In other words, Pinterest reports a rise from 10 to 15 clicks out of 1,000 when the system guesses what would best suit that user. Interestingly, the effect is about the same for an improved version of the alleged Cambridge Analytica technique: A 2017 field study of 3.5 million users, published by Matz et al. in the Proceedings of the National Academy of Sciences and provocatively titled “Psychological targeting as an effective approach to digital mass persuasion,” simply matches Facebook ads to a user’s estimated Openness and Extraversion. This yields 40% more clicks (and 50% more purchases) compared to no personalization.

While advertising and shopping are the more lucrative applications of personalized persuasion, and headlines are full of lurid tales about geopolitical mind control, this type of technology can also be used to improve our health and habits. Customizing advice directly to individuals’ experience and preferences, the way a doctor or coach might, appears to be more persuasive than trying to sell them something. A 2013 “Meta-analysis of web-delivered tailored health behavior change interventions,” published by Lustria et al. in the Journal of Health Communication, found that 40 online programs (on physical activity, diet, quitting tobacco, etc.) showed “significantly greater” effects at post-testing and at follow-up when they used personalized interactions. When it comes to making positive behavior changes, willing human participation is likely to beat mere external “hacking” every time. 

So data-driven personalization is persuasive — to some extent, in some circumstances. Yet evidence on Cambridge Analytica’s core claim is sparse. There are few experiments testing the influence on voting outcomes. 

There is plenty of evidence that campaigns leverage data analytics to effectively target voters by demographics. According to a Guardian article exploring “Obama, Facebook, and the power of friendship: the 2012 data election,” Barack Obama’s Democratic presidential campaign customized a single donation request to 26 voter segments, while Republican Michele Bachmann showed slightly different online ads for Republican voters in each of 99 counties in Iowa, and Republican Rick Perry’s ads praising God only displayed to self-described “evangelical” Iowans on Facebook. 

On the foreign interference side, anthropologist Damian Ruck’s investigation of a Russian Twitter bot campaign in the 2016 election (published in the journal First Monday) finds a 1% poll increase for the Republican candidate for every 25,000 weekly re-tweets of their posts. So it appears that technology has optimized existing strategies to increase the volume, variety, and precision of advertising — but targeting an individual is still challenging, and of questionable value. A study by Lennart Krotzek called “Inside the Voter’s Mind” (International Journal of Communication, 2019) reports that, although personality-congruent ads significantly improve voters’ feelings towards a candidate, they don’t actually make someone more likely to change their vote. 

And yet I know from my own experience on the internet that there are marginal cases, especially where the stakes are lower. For example, I might see a banner ad for yoga pants similar to ones I’ve already shopped for, or be recommended those pants on Pinterest in the context of other beautiful and/or related photos, or see them on Facebook with the right text for my Openness or Extraversion level. Is this a special discovery, pants that deeply resonate with my unique aesthetic, or just a lucky calculation by the machine? If we assume they are using the best available persuasive strategies, then out of 1,000 random users in my pants-related situation, about 3 more would click on the banner ad and about 6 more would click on the Facebook ad (compared to a scenario without personalized advertising). 

Whether I’m persuaded by the technology to do something I wouldn’t have otherwise — to consider buying the pants — depends on whether or not I happen to be one of those extra 3 or 6 users. The effects of this “hack” are relatively small in an absolute sense, but there are nevertheless some people who never would have bought a product, formed a belief, or voted for a candidate were it not for personalized persuasion.