The Data Journalism Handbook – a free, open-source book that aims to help journalists to use data to improve the news – has been released today at the International Journalism Festival in Perugia.

The book is an international, collaborative effort involving dozens of data journalism’s leading advocates and best practitioners – including from Australian Broadcasting Corporation, the BBC, the Chicago Tribune, Deutsche Welle, the Guardian, the Financial Times, Helsingin Sanomat, La Nacion, the New York Times, Pro Publica, the Washington Post, the Texas Tribune, Verdens Gang, Wales Online, Zeit Online and many others.

It is freely available online at datajournalismhandbook.org and a print version is forthcoming from O’Reilly.

Here’s an excerpt from the preface:

This book is intended to be a useful resource for anyone who thinks that they might be interested in becoming a data journalist, or dabbling in data journalism.

Lots of people have contributed to writing it, and through our editorial we have tried to let their different voices and views shine through. We hope that it reads like a rich and informative conversation about what data journalism is, why it is important, and how to do it.

Lamentably the act of reading this book will not supply you with a comprehensive repertoire of all of the knowledge and skills you need to become a data journalist. This would require a vast library manned by hundreds of experts able to help answer questions on hundreds of topics. Luckily this library exists and it is called the internet. Instead, we hope this book will give you a sense of how to get started and where to look if you want to go further. Examples and tutorials serve to be illustrative rather than exhaustive.

We count ourselves very lucky to have had so much time, energy, and patience from all of our contributors and have tried our best to use this wisely. We hope that – in addition to being a useful reference source – the book does something to document the passion and enthusiasm, the vision and energy of a nascent movement. The book attempts to give a sense of what happens behind the scenes, the stories behind the stories.

The Data Journalism Handbook is a work in progress. If you think there is anything which needs to be amended or is conspicuously absent, then please flag it for inclusion in the next version. It is also freely available under a Creative Commons Attribution-ShareAlike license, and we strongly encourage you to share it with anyone that you think might be interested in reading it.

There are hundreds of public domain works scattered all over the internet – from well known projects like the Internet Archive, Project Gutenberg and the Wikimedia Foundation’s Wikisource and Wikimedia Commons projects, to national and international portals like Europeana and the nascent Digital Public Library of America.

And of course there are numerous small islands and islets that are disconnected from these bigger federating enterprises, clustered around different authors, topics, genres, and periods, run for and by special interest communities.

As a researcher and as a reader, I want to be able to browse across these different sources (I don’t mind where texts come from, I just want them to be accurate, to have a stable URL and not to have to trawl around too many different places to look for them). As a potential contributor to the commons, I would like to know which works are available and which are not yet available in digital form, so that I can try to scan and upload them myself or encourage libraries, archives or other institutions to do so.

Currently you can look for specific works on a case by case basis using search engines, or you can browse or search within specific collections, but it isn’t straightforward to get a comprehensive overview of freely available works by a given author. Wikisource is reasonably good for this, but bibliographies and links to digital copies are far from complete for many authors.

University students, researchers and teaching staff are very well placed to help to curate the commons of digital content, author by author, topic by topic, discipline by discipline. To do so, they need (i) an incentive to spend time on this, and (ii) a mechanism to contribute. I’d like to see if we could address some of these points with the TEXTUS project, an open source platform for working with collections of texts.

Regarding (i) one incentive would be to work on a project that recognised and trusted by peers and endorsed by well known institutions and scholars in your field. Hence rather than just contributing to a ‘public domain content’ initiative, you could contribute to a project relating to specific authors, works, or topics that you are doing your research on (a bit like Nietzsche Source, Darwin Online or Copyright History). Users could set up their own project at their own URL, curated by and for a specific scholarly community, each with its own editors, advisors and contributors. This is to recognise the value and importance of trust, reputation and peer review in scholarship. Ideally each TEXTUS instance should be a project that could be cited in a academic paper, and which contributors could list on their CV. This would be following in the footsteps of projects like Open Journal Systems, an open source platform for running open access journals that currently powers over 11,000 journals around the world in a wide variety of fields.

Regarding (ii) ultimately TEXTUS should have an intuitive interface that makes it very easy for scholars to upload scans, transcribe these into plain text and correct transcriptions, and create and edit scholarly bibliographies on different authors and topics. Ideally copies of scans should be uploaded to places like the Internet Archive, plain text transcriptions to Project Gutenberg and Wikisource, and additional and amended metadata records should find their way back into a shared pools of open metadata – just as the original CDDB system or the more recent MusicBrainz project let users share metadata about CDs.

Something like this would have the virtue of helping students and researchers to get more out of freely available digital content, and at the same time harnessing their expertise to curate and enrich the commons for all to enjoy.

Here’s a picture that shows how this might work:

If you’d like to follow our progress or help out, you can join one of the mailing lists or follow @textusproject on Twitter.

‘Community‘ symbol from The Noun Project.

Here is a preview of some illustrations for the Data Journalism Handbook, a free, open source reference book which shows how journalists can use data to improve the news. They were created by the talented Kate Hudson, based on the original designs she did for the book at MozFest 2011.

If you want to be notified when the book is released, you can sign up on the website.

CHAPTER 1: INTRODUCTION

What data journalism is and what it might mean for news organisations. Leading data journalists tell us why they think it is important and what their favourite examples are. Finally data journalism is examined in its broader historical context.

CHAPTER 2: IN THE NEWSROOM

How data journalism sits within newsrooms at the Australian Broadcasting Corporation, the Chicago Tribune, the Guardian, the New York Times, the Zeit Online, and elsewhere. We learn about how to hire developers, how to engage people around a topic through hackathons and other events, cross-border collaboration, and business models for data journalism.

CHAPTER 3: CASE STUDIES

Data journalists tell us about projects that they have worked on – from election monitoring to looking into how public funds are spent, from covering corruption and riots to in depth investigations into education and healthcare.

CHAPTER 4: GETTING DATA

Where to find data on the web, how to request it using freedom of information laws, how to screen scrape and crowdsource it, and how you can republish it and give others permission to reuse it.

CHAPTER 5: UNDERSTANDING DATA

How to make sense of your data – including tips on working with numbers and statistics, how to get stories from data, data journalists’ tools of choice, and how to use data visualisation to find insights in data.

CHAPTER 6: DELIVERING DATA

How to deliver your data to the public – from news apps, to data visualisations, to engaging audiences around your project.

If you have cunning ideas for where we should distribute them, or if you’d like to help distribute some in your area do get in touch!

She uses a process diagram to represent different aspects of the data journalist’s workflow – from getting data to delivering it to their readers in stories, visualisations and apps. The poster will be presented at the Information Design Conference 2012 in London. The design is based on Kate Hudson’s illustrations for the book.

The first edition of the book is nearly completed, ready for its launch towards the end of April at the International Journalism Festival in Perugia. If you want to be notified when the book is released, you can sign up on the website.

A few years ago I used to work at several college and departmental libraries at the University of Cambridge. One of the tasks which library staff regularly had to undertake was to cross reference the latest copies of all relevant reading lists with their collections, to ensure that they had copies of all the books that their students and staff needed.

When I was there, the libraries would print out copies of the PDF reading lists published by the departments and then go through each list with a pen or pencil, searching for each item in the catalogue and then ticking it off if they had it. Given that there are over 100 libraries at the university and over 100 faculties and departments which produce reading lists, that adds up to a lot of library staff time.

While I was there I volunteered to try to devise a way to make it easy for departments to create machine readable reading lists, which would mean that the process of cross-referencing them with library information systems could be automated. Then librarians could spend more time doing things that computers couldn’t do – like getting to know their users and their collections.

This received a luke-warm reception from other librarians I spoke to. I had an extended correspondence about this with a systems librarian at the Cambridge University Library, who was keen but busy. I also wanted to link to digital copies of texts which had entered the public domain – either from within the library catalogue or via an ancillary service – but curating and promoting access to freely available online resources (as opposed to subscription based resources) was not considered to be part of the librarian’s role.

Now, years later, I’m very keen to make it easy for people to create and work with machine readable reading lists using TEXTUS, an open source platform for working with collections of texts which is currently being funded by JISC. In particular I’d like to pilot this with OpenPhilosophy.org, to try to create more machine readable versions of reading lists from philosophy departments in the UK.

In this first instance, this would enable students and staff to easily find freely available public domain works which they could read on their computer or device, annotate and print out. In the future, with the help of some clever scripts, it would enable them to find copies of key texts in their local library, or in other libraries. In both cases lecturers could provide students with a single URL for their reading list, which would help them to find copies of the works they need that are scattered in a variety of digital and physical locations.

To make this happen I’d propose working with students and staff in philosophy departments to create machine readable versions of their reading lists. Once we have these, we can start to match them to digital copies and experiment with scripts to run against university library information systems. If you’d like to participate in this as a student, lecturer, or librarian, or if you’re just generally interested in making this happen – please do drop me a note.

Europe’s Energy, a project I helped to create to put EU energy targets into context, has just won a Silver Award at Malofiej 20. The Malofiej Awards recognise innovative infographics from around the world:

The Malofiej Awards have since 1993 given recognition to the best infographics published in print and on-line across the globe. Every March an international jury meets for nearly four days at the School of Communication at the University of Navarra in Pamplona (Spain). They select those works worthy of gold, silver or bronze medals from among dozens of entries submitted to the competition.

The awards are named after Alexander Malofiej, “an Argentinian cartographer considered to be a pioneer in infographics, and who died in 1987″. This year there were 1,356 entries from 151 media organisations in 29 countries.

The Europe’s Energy project was born out of a series of hackathons I organised in December 2010, which sought to encourage developers, designers and data journalists in several European cities to do useful and interesting things using information from Eurostat. Having a personal interest in climate change, energy and carbon emissions, I was very keen to look into datasets related to these areas. This became the focus of our event in London – and we looked into how energy consumption and production changed in different EU member states over time. The Guardian covered this and I did a writeup with a wishlist of other things we wanted to explore.

In early 2011, I spoke to a press officer at the European Commission who was looking for visual material that would help to put EU energy policies into context for a big meeting at the European Council. I spoke to Friedrich Lindenberg and Gregor Aisch and we decided to try and put something together to enable more people to understand what the EU energy targets meant, how ambitious they were, and how they looked in the context of energy production and consumption in different countries. Soon we were on regular calls with Dirk Heine, Guo Xu, Nathaniel Scheer, and doing late nights to try and get hold of the data we needed, to figure out what it meant, and how best to present it to the public. In just under two weeks the Europe’s Energy was born. In the last 24 hours we crowdsourced translations of the project into 16 languages and I wrote an article for the Guardian Datablog.

The project would not have been possible without the willingness of our team to drop everything and wade knee deep into EU energy policy, data wrangling and data visualisation design. It was supported by the LOD2 project, part of which aims to encourage more people to use open data to provide value to society.

There is a lot more work to be done to continue to gather, combine, interpret and present data related to climate change, energy and carbon emissions. Hopefully Europe’s Energy and projects like it will make some small contribution towards increasing public understanding of complex, but very important, issues. One hopes that Gregor’s beautiful graphics just might help to encourage a few more people to find out more about energy policy, and perhaps even to act to encourage decision makers to increase the share of green energy in the mix.

I’ve started planning this, with the working title “Romanticism Without Borders”. Below is a tentative sketch about the idea. If you’re interested in finding out more or in being kept in the loop, please add your details here.

Romanticism Without Borders is an international, interdisciplinary series of workshops about the influence and legacy of different forms of romanticism around the world, across national borders, across genres. The first events will be held in Berlin, London, and New York City.

Submissions should be in English, accessible for the non-specialist, and no more than 3,000 words in length. Very concise, clearly written, highly focused pieces are strongly encouraged. Topics might include:

• History of the use of terms such as ‘romanticism’ or ‘romantisch’, attempts to define romanticism, the development of the romantic canon.
• Reception and influence of romantic authors and works in different countries – e.g. reception of Coleridge in Germany or Novalis in Britain.
• History of translations and editions of romantic works, as well as reviews, anthologies, and critical or historical texts in countries around the world.
• The role of scholars, critics, writers and others who have helped to translate, introduce of promote romantic authors or works in countries and languages around the world.
• Influence of romantic authors and works on figures not traditionally associated with romanticism – e.g. Ralph Waldo Emerson, Friedrich Nietzsche, Walter Benjamin.
• How romantic works have affected or shaped different genres and media, e.g. photography, film, theatre.
• How romantic works and ideas have shaped theory or practise in other fields and disciplines – e.g. ecology, law, psychiatry.

Work is now underway on OpenPhilosophy.org, a website that will enable users to transcribe, translate, annotate and create bibliographies of public domain philosophy texts.

Today we did some basic mockups for what different pages on the site might look like. Here’s a quick look.

Front page

• Top bar: Small logo in top left. About page, login and register on top right.
• Top section: Search and browse by author, language and period on left. Beautiful image on right.
• Middle panels: For featured content: e.g. featured texts, featured authors, and active transcription projects.

Text page

• Top bar: Small logo in top left. About page, login and register on top right.
• Page body: Formatted text.
• Left hand navigation: Section markers with options to cite section or add section to a list.
• Right hand navigation: Options to add to list, toggle annotations, export (in a variety of formats, including nice PDF edition for print), cite (to get a stable URI for the text or a clipboard ready citation in a variety of common formats), toggle page numbers.
• Text highlight: Options to annotate, cite, and favourite.

List page

• Top bar: Small logo in top left. About page, login and register on top right.
• Page body: Title of list. Brief description of list. Optional headings and subheadings. List item, with options to go to full text, add to list, or cite. Optional comment on list item (for annotated bibliographies).
• Right hand navigation: Export in a variety of formats. Sharing options. Ordering options (alphabetically, chronologically, …).

Author page

• Top bar: Small logo in top left. About page, login and register on top right.
• Page body: Author name. Author dates. Drop-down menu to sort chronologically or alphabetically. List of items. Options to view full text, add to list, cite, or delete (if an administrator) for each item.
• Right hand navigation: Featured lists with works by this author, e.g. annotated bibliographies, recommended reading lists, etc.

User page

• Top bar: Small logo in top left. About page, login and register on top right.
• Page body: User name. User picture. Affiliation, website and contact details. Groups of which user is an administrator. Activity feed, including transcriptions, annotations and uploads.

If you’re interested in finding out more, you can request an invite to test the project when it is ready, join the public open-philosophy discussion list and/or follow @OpenPhilosophy on Twitter.