Tag: data journalism

Rewiring the state

Last year I reported on Young Rewired State, when a group of committed and amazingly talented youngsters descended on Neontribe‘s offices for three days of making stuff. (The report’s not online because at the time the EDP’s CMS automatically took everything offline after two months; no, I still don’t know why they thought that was a good idea.) I remember being stunned by the results and by the people involved – the energy, the excitement, and the apps they made.

On Saturday, I went up to Neontribe’s new offices in Norwich along with 14 others to be part of the first regional Rewired State hack day – but as a developer, not a hack. (For the record: I’m no dev, I just mash things together and swear at them till they more-or-less work in a cargo-cult sort of way.) I got to work with three brilliant young people – Callum, Isabell (@issyIO) and Ben. They were all way ahead of me – I can’t wait to see what they make in the future.

The aim of the day was to use local government data to make apps. We started out with a computer each, a list of data sources, a whiteboard full of ideas and a lot of very tasty food (seriously, the catering was amazing). Oh, and 8 hours to go before we presented our work to a room full of local dignitaries. So no pressure.

The team I was working with decided to create Kebab Hunter, an app that mashed together reviews of local takeaway joints with hygiene data from Norwich City Council, then plotted them on an augmented reality Layar that could be viewed on your smartphone. The result was an app you could use to quickly find a nearby takeout that not only serves tasty food but also won’t make you ill. The bits I did were mostly to do with finding, cleaning and mashing data together (I’m indebted to @psychemedia for this timely post). Most of the time (well, when not eating the delicious cake, anyway) I had my head down, so it wasn’t really until the very end that I got to see what other people had been working on.

For me, with my very limited hodge-podge set of skills, the day was exhilarating. The time limit gave it a focus and a sense of urgency, and working alongside such a talented group of people was a pleasure and a privilege – and a very fast and efficient way of learning. I felt like my brain had gone through a wringer at the end of it, but in a very good way. And we had some awesome things to show for the effort.

Here’s what we made (I’ll add links to this list if I can find them, and if there are any mistakes let me know – I didn’t catch everyone’s name):

  • Where does Norfolk’s money go? A map of Norfolk council spending – Sym Roe
  • Bin Posse. Reminders of what goes out when by SMS – Rupert Redington
  • AV findings. Where voted “Yes”, and what were they like? (Apparently Yes to AV is strongly correlated with museum visits) – Chris Heath and Katja Mordaunt
  • Bridge Headroom. How much space is there under Potter Heigham bridge? – Michael Holness
  • Words about Norfolk. What words does Wikipedia link to Norfolk? – Rob Young
  • Festival timeline. The Norfolk and Norwich Festival lineup, displayed to investigate – Harry Harrold
  • Hey Chief! A humourous look at the value for money of Norfolk Fire Service. (Norwich has a lot of cat-related incidents, we learned) – Peter Chamberlin, Heydon Pickering, Michael Holness
  • Kebab Hunter. Augments a phone camera’s eye view of Norwich, with takeaway food safety info and reviews – Callum Weaver, Mary Hamilton, Ben Holloway and Isabell Long.

At the end of the day we presented our creations to local politicians, council staff and each other in the Octagon Chapel, a beautiful and oddly stately venue for such a high-tech day. I hope the folks who saw what we made see what’s possible when you get interested, creative people with l33t skills in a room with their data.

Afterwards, people were talking about the power of open data – its scariness, the fact that transparency can’t be done half-heartedly, the fact it can’t be controlled, but also the freedom to experiment and the excitement of possibility. And the need for devs, designers, interpreters and even journalists to bridge the gap between spreadsheets and stories, between data and people. Those were good conversations, and I hope a lot more comes out of this event.

Very, very good day.

Edited to add: @harryharrold has collected the whole day as it happened. Includes geese.

Data journalism at the Frontline

Last week I went to a Frontline Club event on data journalism. There’s a live stream of the talks and questions here along with a good brief rundown of who said what. It’s well worth watching all the way through for four good talks exploring four very different aspects of data journalism.

First up, Simon Rogers of the Guardian Datablog, talking about crowdsourcing, wading through the MP’s expenses releases and the Wikileaks Afghanistan files, and working to visualise the data they receive. My highlights:

  • A huge amount of the hard work done on the MP’s expenses project was done by one person. Some individuals give a great deal.
  • For the Wikileaks files, they set up tasks and asked people to complete them. Simon didn’t say whether this had much of an effect on bounce rates or completion – but it’s likely that they did. And if so, perhaps the next step is gamifying the process.
  • But the Guardian hasn’t yet had any useful data come from crowdsourcing. Maybe they’re asking the wrong questions or requesting that people do the wrong tasks; maybe the wisdom of crowds is a different beast from the action of crowds.

Second, Julian Burgess, currently developing at the Times and soon to be off to New York, talking about how to tackle data. This was perhaps the most directly practical talk, with pointers to some great tools and tips on techniques – his slides are online here. My highlights:

  • When presented with data, don’t panic. Take your time, work through a sensible series of steps to analyse and work out the best approach to what’s in front of you.
  • There are buckets and buckets of tools out there, most if not all of them free. They’re big, they’re powerful, they’re incredibly useful. I took away the impression that a key skill for data/CAR journalists is knowing which tools are going to be good for handling which datasets – when to use Wordle vs ManyEyes, when to use Freebase Gridworks rather than Google Spreadsheets.
  • Metadata. This is where my little linked-data brain lit up as a puzzle piece fell into place. Hidden metadata is still part of a dataset – and it can tell stories about how the data came to be in this place, in this way, shape and format. That’s useful.

David McCandless, whose gorgeous data visualisations can be found at Information Is Beautiful spoke about visualising data and the stories it can reveal. My highlights:

  • Visualising something really well takes a lot of time and a lot of hard graft. It takes – this is going to sound obvious – a vision.
  • Making something that’s both beautiful and conveys information is hugely difficult and walks a very fine line between appearance and utility, but it’s more than worth the balancing act when it works. Successful data journalism needs to be interesting, easy, beautiful and true.
  • You don’t just tell stories with visualisation, you find them too. Weird spikes, unusual patterns, data points that look like anomalies – they all prompt further questions. By asking why something is the way it is, you get stories.

Finally Michael Blastland, freelance journalist and creator of BBC Radio 4’s More Or Less, discussed the problems with numbers. My highlights:

  • Numbers are slippery. Where do they come from and why? Just like quotes from sources, every stat is compiled by someone with an agenda and a purpose and most of them are biased in ways we can’t begin to guess till we start digging. Don’t use data just because it’s convenient.
  • Sometimes the story behind the number is more interesting – and more in the public interest – than any story based on the number.
  • I need to learn more maths. Specifically, statistics.

Most journalists haven’t been taught the skills they need to do what these guys do (and they were all guys; that’s not a bad thing per se, but worthy of a note that more women on the stage would be good to see next time). Every speaker told the audience that there’s no special education required to work in their field. We don’t need to be programmers, or designers, or statisticians. But we need to be interested and open-minded and both willing and ready to learn.

But doing this well takes a team, and it takes time. Most journalists will never get the chance to learn or teach themselves. And even if they do – you can be a jack of all trades, you can take a project through from finding the numbers to analysing the data to making it look amazing and simple and easy to use, but it takes a harsh amount of time and is punishingly frustrating to do alone.

Data journalists need support. Time, resources, connections, and people. I’ve not yet met anyone who can do all of this – or even most – alone; certainly not in their spare time, working in the gaps, at the ends of long days. All the people who spoke at the Frontline Club were at the top of the market, doing brilliant work that reaches people, making useful journalism. We need more like them – but we also need the support systems that allow people like them to grow and thrive. Next time, I’d like to see conversations about company culture, about how to evangelise to your newsdesk, about time management and learning and how, exactly, we free up time and space for data work in newsrooms all over the country, from the ground up.

On a related note, the next meetup of Hacks and Hackers London is on October 20. If all goes well, I’ll be there. Come join in.

JEEcamp thoughts on data journalism

Information VisualizationI spent some time talking to Martin Belam (@currybet) about data journalism and the importance (or otherwise) of journalists learning to code.

He said, as he’s said before, that it’s more important for journalists to know whether something is or isn’t possible than for us to necessarily be able to do it ourselves.

And for working journalists whose day to day job doesn’t carry a coding requirement already – and particularly those of us who are lucky enough to be in a workplace where there are developers or programmers who can take our ideas and make them flesh (ie. not me), he’s almost certainly right.

Those skills are becoming more and more important. With the birth of data.gov.uk and the increasingly open approach to information that the new coalition government is likely to take, sifting and analysing data to find the stories is going to be a vital skill for a lot of journalists.

We need to know our way around a spreadsheet. We need to be able to spot patterns in data and understand not only what they mean but also how we can use them to reveal stories that are not only relevant but useful.

We need to know where our skills can get us. We need to know our capabilities and our limits – and, crucially, we must be aware of what we don’t know. That’s not just knowing that there are holes in our knowledge, but knowing the shape of those holes so that we can try to get our problems a little closer to a solution.

Journalism is about asking the right questions. We research stories before we interview subjects so that we can ask pertinent questions whose answers will illuminate the subject. We need to be able to do the same thing with our data – we need to know what questions to ask and how, so that even if we can’t make the tools ourselves we can hand over the task to someone else without asking the impossible or wasting their time.

But most of the time, certainly for journalists on regional papers and I would wager for many in other areas, those people who know how to make the tools just don’t exist. I have friends who code, but I can’t ask them for a favour every time I want to create a news app, or diff two versions of a stack of documents, or visualise a complex dataset, or tell the story of 100 people’s losses from an investment fund going bust in a way that conveys both the scale and the humanity of the problem.

Regional journalists work on hundreds of stories that could be made vastly easier or more beautiful or more accessible through a touch of computer work (spreadsheets, maps, things that aren’t quite coding but sort of almost are and look like it to the untrained eye). A few of us can create those additions; the rest just write the story, and our papers and websites are poorer for it.

We work on a few stories – and the number is increasing – that are perfect for news apps, web coding, multimedia packages or other more complex solutions that very, very few of us can create. But no one else will do it for us.

On top of that many of us struggle with inflexible content management systems that penalise or make it literally impossible to display data-driven work online. Faced with that problem, some budding computer-assisted-reporters give up before they’ve even started.

So I’m not going to stop learning Python. It’s not a complete solution to the problem – for that we need real, systemic change so that the businesses we work for all value data work, understand its increasing relevance, reflect on current practice and support training journalists to do an evolving job.

But for me, it means that in the future I might be able to create better stories, automate processes within series or campaigns or multiple follow-up stories, make my job easier and make a better experience for the reader all at the same time.

At least, until we all have newsroom developers.