Dear Klout, This Is How You Measure Influence

“Up until now, we’ve thought a lot about influence in terms of how big someone’s microphone is,” NYU Professor of Business Sinan Aral, tells TechCrunch, “A more important question is, how much does someone’s influence, or their messages, or the things they have to say to us, change our behavior?” The solution, demonstrates Aral in a paper published today in the prestigious journal, Science, is to measure whether users are actually taking action: contributing to a campaign, buying a product, downloading an app, or reading an article. Furthermore, messages must be randomly sent to followers, otherwise we have no idea whether users would have taken an action if they had seen the update or not. Randomization and action-based measures are a high bar, but at least it will get us past knowing whether social media “influence” actually means anything.

Social influence scores, such as Klout, measure how likely the Facebook fans of Mitt Romney are to engage and share his updates with the rest of their social media friends. However, given that many followers of Romney are already die-hard Republicans, would they have already seen a video he shares on the Drudge report anyway? If Romney solicits his followers for money, would they have already contributed through another website? In other words, social friends are sheep-like: they read the same articles, download the same apps, and give money to the same causes and politicians. Aral says that caring about high follower count could be a “waste of money” if so-called “influencers” are not actually changing behavior. According to the paper, it turns out that follower counts and retweets may be much less important than demographics, such as age, personality type, and gender in determining who is influential and who’s susceptible to social media manipulation.

Aral’s experiment looked at how likely Facebook users were to adopt a a movie review app after one of their friends messaged them. “The experiment was conducted over a 44-day period during which 7,730 product adopters sent 41,686 automated notifications to randomly chosen targets amongst their 1.3 million friends,” reads the study. Most importantly, all messages were randomized, so he was able to test whether a message itself led to downloading the app, not because a user knew one of his friends was an avid movie buff and would like an app that spammed his friends with updates.

Aral found a few intriguing results: peers of the same age are most influential on each other, and women are less susceptible to influence than men. He also settled a long standing debate between Malcolm Gladwell and his critics over whether messages go viral due to an army of highly influential broadcasters or after being dropped into a population of share-happy users. According to Aral, a viral campaign needs both. Easily susceptible users tend not to be influential, and influentials tend to be quite stubborn. The trick is finding a population that has enough influencers to spread the message to enough susceptible users.

More important than Aral’s findings are his methods of randomization, which set the scientific foundation for measuring actual influence, rather than relying on follower counts or a Klout score. For instance, much fuss has been made about the “superiority” of Obama’s sizable social media presence. Obama has 14 times as many Facebook fans as Romney (27M vs. 1.9M), yet both presidential contenders get roughly an equal number of likes on their most popular posts. At the very least, this means that follower counts don’t make too much of a difference, and engagement (i.e “likes”) may be equally inconsequential.

Indeed, we’ve argued before that the influence of social media in something like elections is way over-hyped. If no one under the age of 35 had voted for Obama in ’08, he still would have every state except two (in other words, it wasn’t his large army of young social media followers that made a difference).

Social media is still in its infancy. Follower counts and retweets are a cheap, attractive way to identify Twitter and Facebook celebrities. While we still don’t know how exactly to measure influence, we have the scientific tools to start learning. In the meantime, be skeptical of anyone who tells you they know the answer.