Reach and impact: news metrics you can use

I’ve been thinking about this Tow study for a while now. It looks at how stats are used in the New York Times, Gawker and Chartbeat; in the case of the latter, it examines how the company builds its real-time product, and for the former, how that product feeds in (or fails to feed in) to a newsroom culture around analytics. There’s lots to mull over if part of your work, like mine, includes communication and cultural work about numbers. The most interesting parts are all about feelings.

Petrie says:

Metrics inspire a range of strong feelings in journalists, such as excitement, anxiety, self-doubt, triumph, competition, and demoralization. When devising internal policies for the use of metrics, newsroom managers should consider the potential effects of traffic data not only on editorial content, but also on editorial workers.

Which seems obvious, but probably isn’t. Any good manager has to think about the effects of making some performance data – the quantifiable stuff – public and easily accessible, on a minute-by-minute basis. The fears about numbers in newsrooms aren’t all about the data coming to affect decisions – the “race to the bottom”-style rhetoric that used to be very common as a knee-jerk reaction against audience data. Some of the fears are about how analytics will be used, and whether it will drive editorial decision-making in unhelpful ways, for sure, but the majority of newsroom fear these days seems to be far simpler and more personal. If I write an incredible, important story and only a few people read it, is it worth writing?

Even Buzzfeed, who seem to have their metrics and their purpose mostly aligned, seem to be having issues with this. Dao Nguyen has spoken publicly about their need to measure impact in terms of real-life effects, and how big news is by those standards. The idea of quantifying the usefulness of a piece of news, or its capacity to engender real change, is seductive but tricky: how do you build a scale between leaving one person better informed about the world to, say, changing the world’s approach to international surveillance? How do you measure a person losing their job, a change in the voting intention of a small proportion of readers, a decision to make an arrest? For all functional purposes it’s impossible.

But it matters. For the longest time, journalism has been measured by its impact as much as its ability to sell papers. Journalists have measured themselves by the waves they make within the communities upon which they report. So qualitative statements are important to take into account alongside quantitative measurements.

The numbers we report are expressions of value systems. Petrie’s report warns newsrooms against unquestioningly accepting the values of an analytics company when picking a vendor – the affordances of a dashboard like Chartbeat can have a huge impact on the emotional attitude towards the metrics used. Something as simple as how many users a tool can have affects how something is perceived. Something as complex as which numbers are reported to whom and how has a similarly complex effect on culture. Fearing the numbers isn’t the answer; understanding that journalists are humans and react in human ways to metrics and measurement can help a great deal. Making the numbers actionable – giving everyone ways to affect things, and helping them understand how they can use them – helps even more.

Part of the solution – there are only partial solutions – to the problem of reach vs impact is to consider the two together, but to look at the types of audiences each piece of work is reading. If only 500 people read a review of a small art show, but 400 of those either have visited the show or are using that review to make decisions about what to see, that piece of work is absolutely valuable to its audience. If a story about violent California surfing subcultures reaches 20,000 people, mostly young people on the west coast of the US, then it is reaching an audience who are more likely to have a personal stake in its content than, say, older men in Austria might.

Shortly after I arrived in New York I was on a panel which discussed the problems of reach as a metric. One person claimed cheerfully that reach was a vanity metric, to some agreement. A few minutes later we were discussing how important it was to reach snake people (sorry, millennials) – and to measure that reach.

Reach is only a vanity metric if you fail to segment it. Thinking about which audiences need your work and measuring whether it’s reaching them – that’s useful. And much less frightening for journalists, too.

Pocket Lint #27: dragons

If you’d like to get Pocket Lint as a regular-ish email on most Fridays you can sign up here. Delivery will be sporadic for a while.

Five stories about how, why and where we move.

That Dragon, Cancer – in its final hours of funding on Kickstarter.

“Mr. Jones, 64, has an intellectual disability and a swollen right hand that aches from 40 years of hanging live turkeys on shackles that swing them to their slaughter. His wallet contains no photos or identification, as if, officially, he does not exist. And yet he is more than just another anonymous grunt in a meat factory. Mr. Jones may be the last working member of the so-called Henry’s Boys — men recruited from Texas institutions decades ago to eviscerate turkeys, only to wind up living in virtual servitude, without many basic rights.”

What it’s like to be pregnant, bipolar and at significant risk of post-partum depression in the UK.

The truth about the teenager who disappeared in the Michigan State University steam tunnels, sparking a national panic about Dungeons and Dragons.

“Do your best. Every deity and the spirits of your dead comrades are watching you intently. It is essential that you do not shut your eyes for a moment so as not to miss the target. Many have crashed into the targets with wide-open eyes. They will tell you what fun they had.”

Parable of the polygons: a playable post on how a small amount of bias leads to segregation in society.

“Whites can live, love, study, work, play and die in segregation … and still profess that race has no meaning in their lives.”

The woman who saw dragons.

The Magic Circle is set inside a high-fantasy remake of a fictional 80s text adventure, also called The Magic Circle, that has been in production for many years. The original was made by Ish Gilder, “a mild-mannered game designer who won’t let us call him a genius” in the words of a fake website for the game. An audio diary early on (of course there are audio diaries) reveals that The Magic Circle has been in production for two decades. It’s vapourware. The environment resembles a whiteboard outline that has been erased and redrawn – a beautiful, sinewy vision of sketch lines and uncertain clouds, with only occasional flashes of colour showing live elements.”

The Rosetta landing, cartooned live, in gif form.

“The moon is much larger than it appears to be. This is worth remembering because next time you are looking at the moon you can say in a deep and mysterious voice, “The moon is much larger than it appears to be,” and people will know that you are a wise person who has thought about this a lot.

Poem of the week: Us Two, A. A. Milne

Game of the week: Detritus, which I made last time I moved 10,000 miles around the world.

Tumblr of the week is an Instagram instead: Follow Me.

Pocket Lint #26: implicit

If you’d like to get Pocket Lint as a regular-ish email on most Fridays you can sign up here.

“The 22 bus is the only route that runs 24 hours in Silicon Valley and it has become something of an unofficial shelter for the homeless. They call it Hotel 22.”

The problems with capitalism, as explained by a Minecraft hedge fund manager.

A medical actor who fakes illnesses so trainee doctors learn empathy. “Empathy means realizing no trauma has discrete edges. Trauma bleeds. Out of wounds and across boundaries. Sadness becomes a seizure. Empathy demands another kind of porousness in response.”

What does an orgasm look like?

Every email is a ghost story.

Harvard’s tests for implicit associations about race, gender and sexual orientation.

“With my form of multiple sclerosis, I quickly started to realise that while the disease would likely never kill me by itself, the sheer weight of time could still do some serious damage. You know. All the wonderful things I should have done before. All the terrible things that might now happen. Playing Spelunky’s genuinely refreshing in this regard: it’s a reminder that the only moment that really matters is the moment that’s currently unfolding. Strategy dries up and blows away in the present, and in its place you’re left with tactics, with what to do for the next thirty seconds. Forget the City of Gold – how do I handle this frog that’s blocking the exit? Forget my plans for the afternoon, what’s with this stutter?

On being a black male, six feet four inches tall, in America in 2014.

“Capturing Knight was the human equivalent of netting a giant squid. He was an uncontacted tribe of one.”

Revenge bento.

It’s a bad fence.

Poem of the week: Epistle: Leaving, Kerrin McCadden

Game of the week: Crossy Road

Tumblr of the week: To My Unborn Son

Pocket Lint #24: hills of beans

If you’d like to get Pocket Lint as a regular-ish email on Fridays you can sign up here.

“People do still donate at churches and other places. We literally have ten tonnes of beans. We have a ‘bean room’ at our central storehouse. People for some reason associate a foodbank with beans. But actually what we need is coffee, sugar, UHT milk, tinned fruit, tinned fish. A whole range of things. So we try and get the shopping list to people and ask them to buy from it. But whatever you do, you still get lots of beans.”

The rise and fall of Default Man: “When we talk of identity, we often think of groups such as black Muslim lesbians in wheelchairs. This is because identity only seems to become an issue when it is challenged or under threat. Our classic Default Man is rarely under existential threat; consequently, his identity remains unexamined. It ambles along blithely, never having to stand up for its rights or to defend its homeland.”

Of GamerGate and disco demolition.

Trouble at the Kool-Aid point.

handler/the-paranoid-style-in-gaming-misogyny-1d412f212bda">The paranoid style in gaming misogyny.

I Know This Sounds Like Spam, But I Really Did Double My Mass In TWO WEEKS And Now Women Can’t Get Enough Of Me And I’m SCARED

If you only come into contact with one thing about GamerGate, make it this video.

Robert Webb on growing up male: ‘Nobody ever told me: you don’t have to waste years trying to figure out how to be a “man” because the whole concept is horseshit. We are people, individuals comprising a variety of sexes, races, shifting sexualities and all the rest of it. Every convention that tries to reinforce this difference is a step back. Notions of gender pointlessly separate men from women, but also mothers from daughters and fathers from sons. The whole thing is – at best – just a stupefying waste of everyone’s time.’

Shadows of Mordor, Watch Dogs, and the politics of NPC agency.

‘The first funeral parlour he went to, in Hoxton, east London, told him they needed £2,500 upfront for the church and the vicar. “I said: ‘I can’t afford that.’ They said: ‘You won’t be able to bury him without that money’,” he says, at his flat. His father’s finch whistles in a cage by the window. Griffin is a former landscape gardener who gave up his job to care for his father when he became unwell. Another firm quoted him costs of around £4,000, asking for £1,000 upfront. “I told the funeral office I would just go to the unemployment office to see what they can give me. They said, ‘Oh no, we need the money upfront’. That’s when I started to get worried.”’

Tip sheet and resources for journalists – and others – dealing with graphic images and material.

What to expect when the internet tries to ruin your life.

Poem of the week: Proserpina, Going Deeper, by Jack Hollis Marr

Game of the week: Realistic Kissing Simulator

No, good content is not enough for Facebook success

At the recent ONA conference, Liz Heron, who oversees Facebook’s news partnerships, came in for some questioning about how news organisations can do well on the platform – something that’s a cause of some consternation for many, as it becomes increasingly clear how important it is as a mass distribution service. This is one of her responses:

This is a familiar line from Facebook – I’ve been on panels with other employes who’ve said exactly the same thing. But while I have the greatest respect for Heron and understand that she has to present Facebook’s best side in public – and that a tweet may be cutting context away from a larger argument – this statement is demonstrably false. Even skimming the rather fraught question of what exactly “good” means in this context, it’s questionable whether quizzes and lists such as those that have brought Playbuzz its current success are in any meaningful way replicable for most news organisations.

It’s not that Playbuzz is “gaming the algorithm” necessarily, though it may be. It’s that the algorithm is not designed to promote news content. Facebook’s recent efforts to change that are, quite literally, an admission of that fact. Facebook itself knows that good – as in newsworthy, important, relevant, breaking, impactful, timely – is not sufficient for success on its platform; it sees that as a problem, now, and is moving to fix it.

In the mean time, creating “good” content will certainly help, but it won’t be sufficient. You can bypass that process completely by getting your community to create mediocre content that directly taps into questions of identity, like Playbuzz’s personality quizzes, and giving every piece absolutely superbly optimised headlines and sharing tools. You can cheerfully bury excellent work by putting it under headlines that don’t explain what on earth the story’s about, or are too long to parse, or are simply on subjects that people will happily read for hours but don’t want to associate themselves with publicly.

Time and attention are under huge pressure online. Facebook are split testing everything you create against everything else someone might want to see, from family photos to random links posted by people they’ve not met since high school, and first impressions matter enormously. “Good” isn’t enough for the algorithm, or for people who come to your site via their Facebook news feed. It never has been. Facebook should stop pretending that it is.

Further reading: Mathew Ingram has context and a longer discussion.

Reddit thinks it’s a government, but doesn’t want to govern

In non-spoof news, today Reddit’s CEO posted a blog post about why it wasn’t going to take down a community specifically devoted to sharing naked photos of celebrities acquired by hackers and very much not endorsed by those pictured. Then, having drawn a line in the sand, it promptly banned the community. That caused, unsurprisingly, a lot of users to react with confusion and not a little anger, pointing out – among other things – that ban was more than a little hypocritical if Reddit was going to continue not to police other problematic communities (pro-anorexia and self harm communities, for instance), and suggesting that Reddit’s response was only because of the status, profile and power of the victims in this instance (the site doesn’t take down revenge porn, for example). There’s been another round of explanation, which boils down to: Reddit got overwhelmed and therefore had to take action. That actually bolsters some of the arguments made by users – that it’s only the high-profile nature of this incident that forced action – but if the first post is to be believed, Reddit doesn’t see that as a problem. It wants the community to choose to be “virtuous” rather than being compelled to do so – it wants its users to govern themselves. But it also thinks it’s a government. Yishan says:

… we consider ourselves not just a company running a website where one can post links and discuss them, but the government of a new type of community. The role and responsibility of a government differs from that of a private corporation, in that it exercises restraint in the usage of its powers.

Yishan simultaneously argues that Reddit users must arrive at their own self-policing anarchic nirvana in which no bad actors exist, and that Reddit is not a corporation but a governing force which has both the right to police and, strangely, the responsibility not to do so. Of course Reddit is a corporation, subject to US and international laws. Of course its community is not a state, and its users are not citizens. Yishan is dressing up a slavish devotion to freedom of speech regardless of consequence as a lofty ideal rather than the most convenient way to cope with a community rife with unpleasant, unethical and often unlawful behaviour. Doxxing, revenge porn, copyright infringement so rampant it’s a running joke, r/PicsOfDeadKids: none of these things are dealt with according to the social norms and laws of the societies of which Reddit is, in reality, a part. Only when admins become overwhelmed is action taken to police its community, and at the same time the CEO declares the site to be, effectively, the creator of its own laws. This would be nothing but self-serving nonsense if it weren’t for the way it’s being used to justify ignoring harmful community behaviours. Reddit’s users are right to point out that the company only acts on high-profile issues, that Reddit’s lack of moral standards for its users allows these situations to develop and makes it much harder for the company to police them when they do, and that the site’s users suffer as a result of its haphazard approach:

This is just what happens when your stance is that anything goes. If you allow subreddits devoted to sex with dogs, of course people will be outraged when you take down something else. If you allow subreddits like /r/niggers,of course they’re going to be assholes who gang up to brigade. The fine users of /r/jailbait are sharing kiddy porn? What a shocking revelation. The point is, you can’t let the inmates run the asylum and then get shocked when someone smears shit on the wall. Stand up for standards for a change. Actually make a stance for what you want reddit to be. You’ll piss off some people but who cares? They’re the shitty people you don’t want anyway. Instead you just alienate good users who are sick of all of the shit on the walls.

If Reddit thinks it’s a government, it should be considering how to govern well, not how to absolve itself of the responsibility to govern at all.

Pocket Lint #21: sound and fury

If you’d like to get Pocket Lint as a regular-ish email on Fridays you can sign up here.

The Times of London is pumping increasingly frenetic typewriter noises into its newsroom.

An excellent primer on this week’s implosion in videogame culture.

Today, videogames are for everyone. I mean this in an almost destructive way. Videogames, to read the other side of the same statement, are not for you. You do not get to own videogames. No one gets to own videogames when they are for everyone. They add up to more than any one group.”

“When there’s no immediate threat to our understanding of the world, we change our beliefs. It’s when that change contradicts something we’ve long held as important that problems occur.”

Stereotype lift persists online with virtual, gendered avatars.

The Man Without a Mask: How the drag queen Cassandro became a star of Mexican wrestling.

How social media silences debate.

“Banning comments—or moderating with an iron fist—is not squelching honest and open debate in the public sphere, anymore than refusing to publish every letter to the editor, unedited, in a print publication. Telling people to take their bullshit to Reddit is not a harbinger of Orwellian dystopia.

Classic first lines from novels in emojis.

How to listen to the radio properly: BBC guidance from 1940.

Tumblr of the week: Slug Solos.

Poem of the week: Mr. Grumpledump’s Song, Shel Silverstein.

Game of the week: Gridland, a match-3 game with a building/fighting day/night cycle.

Do real names really make people nicer online?

Mathew Ingram at GigaOm has an interesting look at some Livefyre research suggesting that if you force people to use their real names to comment on your site, the vast majority will just stop commenting.

Most of those surveyed said that they responded anonymously (or pseudonymously) because they didn’t want their opinions to impact their work or professional life by being attached to their real names, or when they wanted the point of their comment to be the focus rather than their identity or background. And close to 80 percent of those surveyed said that if a site forced them to login with their offline identity, they would choose not to comment at all.

The bottom line is that by requiring real names, sites may decrease the potential for bad behavior, but they also significantly decrease the likelihood that many of their readers will comment.

That led me to an interesting question this morning: do real names really reduce the potential for bad behaviour in comments? It’s a popularly held belief, but there doesn’t seem to be a great deal of evidence out there to support the idea that meatspace identities are any more useful than persistent pseudonyms when it comes to holding people accountable for their actions online. On Twitter, Martin Belam points out:

…but there’s a big gap between “named staff members early in comment threads” and “real names for everyone”.

I can find some evidence that persistent pseudonymity is a positive thing. Disqus did a very large study in 2012 on their comments database; though their methodology is opaque, their results showed pseudonymous commenters posted both the largest number and the highest quality comments across their network. Pseudonyms make people more collaborative and more talkative in learning environments; they positively influence information sharing; they encourage people to share more about themselves in the relative safety of an identity disconnected from meatspace.

There’s also evidence that anonymity – a complete lack of identifiers and nothing to chain your interactions together to form a persona, as distinct from pseudonymity, where you pick your own identity then stick with it – is a negative influence on the civility of debate, and that it engenders more adversarial conversations in which fewer people’s minds are changed.

There’s a Czech study into the differences between anonymity and pseudonymity and how to design for reduced aggression, apparently, but the actual link 404s and the abstract isn’t detailed on the difference. There’s interesting research into the social cost of cheap, easily replaceable pseudonyms, which allow effective anonymity through the evasion of reputation consequences; Reddit and Twitter are great examples of communities where you can see this behaviour in action.

Investigating this issue isn’t helped by the fact that researchers have in the past conflated anonymous and pseudonymous behaviours, but there’s increasing awareness now that the two engender big community differences; it’s also skewed by places like 4chan and Reddit being prominently discussed while less adversarial communities like Tumblr or fandom communities are less often scrutinised. (Male-dominated communities are covered more than female-dominated communities, and widely seen as more typical: so it goes.)

I’ve found one study that suggests anonymity makes for less civil comments, but I can’t access the full study to find out what it says about pseudonymity. There are suggestions, like Martin’s, that moderation helps keep things civil; there’s evidence that genuine consequences for poor behaviour helps too. And there’s this:

“While evidence from South Korea, China, and Facebook is insufficient to draw conclusions about the long-term impacts of real name registration, the cases do provide insight into the formidable difficulties of implementing a real name system.”

But where’s the proof Facebook is right when it claims its real name policy is vital for civility on the site? Where’s the evidence that Facebook comments are more civil than a news site’s because of the identity attached, rather than because the news site goes largely unmoderated and Facebook’s comment plugin broadcasts your words to everyone you know on there? Or because most people just don’t comment any more, so arguments die faster? I am struggling to find one study that demonstrates a causal link.

So this is an open request: if you know of more studies that indicate that denying pseudonymity improves comment quality, let me know on Twitter @newsmary or share them in the comments here. If I’m wrong, and it makes a big difference, it’d be great to be better informed. And if I’m right, and it’s community norms, moderation and reputational consequences that matter, it’d be great to put the idea that real names are a magic bullet for community issues to bed once and for all.

The chilling effect

I’m finding myself withdrawing from Twitter a little, at the moment. Some of that is an ongoing process that started when I moved to Australia and left much of my busy timeline behind; friends are living at different times now, and Twitter is different out here. But some is a response to the corrosive atmosphere around games right now, and the way it’s come to a head in the form of the attacks on Depression Quest developer Zoe Quinn over the last week.

The whole situation has become far too complicated for most folks to follow, but, broadly speaking, it started with a disgruntled ex making unpleasant allegations about private affairs very publicly, went through a point where indie game devs were having accounts hacked just for saying publicly that they supported Zoe, and is now at a stage where people seem to think it’s a good idea to dress up as fantasy racing birds and protest the state of games journalism ethics to get the attention of the “real media”, as though reporting on unsubstantiated allegations by an interested party would be good journalism rather than abysmally, impossibly awful. Liz Ryerson has a very strong round-up of the state of affairs here.

So.

There are consequences for speaking out. There are always consequences. I’m logging on to Twitter, almost any time of the day or night, and I’m seeing friends frustrated by dealing with people who want to tear them down for supporting a friend, a colleague, someone whose work they admire. The chilling effect here is huge, and not just applicable to those who have already spoken. I am finding myself withdrawing because I can’t face watching this happen again, after watching friends and colleagues and people whose work I admire driven completely out of the industry in the past.

And I’m frustrated with myself, because I have a platform that intersects with the games industry. I have a committed hobbyist relationship with videogames; I play a great deal, write about some, and occasionally create strange little pieces. If I was ever going to have a professional career in videogames, that was scotched long before the women I’m watching being pushed out now, when my all-girls’ school refused me permission to cross over to the boys’ school to study IT and electronics when I was 14. (Institutional sexism: it’s a thing.) So I have a platform as someone with an interest but no financial stake, and a successful career as a non-games journalist, and the ability to stand up and say, as a person and a gamer and a journo: this is not OK.

And yet. The chilling effect is such that I am frightened to do so. I use Twitter for work; turning off my mentions and retreating until an attack dissipates is an option that hurts me professionally. I have a mental illness, and I do not know how that might interact with a coordinated attack. Visibility is power, when it comes to speaking out against this bullshit. Visibility is also a great weakness.

This is how I’m feeling, watching a woman being attacked for daring to be female and make games and remain human. Relatively speaking, I’m both protected and powerful. Now imagine how it must feel to watch without that protection or that power. Imagine how it might feel as a teenager who wants to make games, watching someone who looks like you be punished for doing so. Imagine how it affects your choices, not just about whether or not to withdraw from Twitter but whether or not to take certain classes, or whether or not to release side projects online. Imagine trying to decide whether your future creative happiness is worth risking this level of psychological violence. Imagine doing it anyway, and being attacked for it. Imagine deciding that opposing it is too dangerous, and joining the chorus out of self-preservation, in the hope that maybe, just maybe, they won’t attack you next time.

The people involved in these attacks, the hacks themselves but also the vicious teardowns of Quinn’s works and reputation and the harassment of her supporters, just want women to shut up. It’s not about games and it sure as hell isn’t about journalistic ethics; it’s just about keeping girls out of the clubhouse by any means necessary. They don’t like it when we speak, and they really don’t like it when we shout back. But I can’t be pushed out of an industry I’m not in; all I can do is discuss things on the sidelines. If I get attacked for doing so, all it’ll do is prove my point.

Pocket Lint #20: Snowblind

If you’d like to get Pocket Lint as a regular-ish email on Fridays you can sign up here.

10 Beautiful Interracial Arrests.

We study white people. We are taught this as a tool of survival. We know when there is unrest in the souls of white folks. We know that unrest, if not assuaged quickly, will lead to black death. Our suspicions, unlike those of white people, are proven right time and time again.”

Can we imagine, for a minute, what it would look like if officers were trained in mediation? What if you called the police when you witnessed a violent fight; officers arrived ready to separate the parties, come to a non-violent resolution, and make sure each person got home safely. As long as they are connected to the system of incarceration, we cannot expect the police to take this role.”

Fark adds misogyny to its moderation guidelines.

Hellhole: on the US prison system’s use of solitary confinement as torture.

Ted talks are lying to you.

Adderall’s technology problem: “Tech should to be a viable career path, not an investment market for a wealthy select few who aren’t on the ground floor. And it should be an inclusive industry that doesn’t favor the young, able, and self-destructive. But if we maintain this idolization of high-producing individuals, the rat race will persist. As long as there is an economic incentive to harm oneself in hopes of performing the superhuman, those who will not — or, for those of us with ADHD, cannot — will remain subhuman.”

The P.T. Twitch stream that first revealed the game’s secret: a strange, extremely scary free demo released on the PS4 turns out to be a trailer for an absolutely huge videogaming collaboration. And will probably be better than the final game.

My partner’s Kickstarting his game Goblin Quest, and a Twine game coauthored with me is one of the stretch goals. It’s going well so far.

The etymology of “cladly dressed”.

“Many of these women come from hours away, one from a little town on the Kentucky border that’s a seven-hour drive. They don’t know much about Dr. Parker… What they do know is this: He is the doctor who is going to stop them from being pregnant.”

Tumblr of the week: If They Gunned Me Down.

Poem of the week: Common, A Letter To The Law.

Game of the week: Kindness Coins.