Imagine a thousand-volume work entitled The Intellectual History of Europe. Imagine also a great convocation of resurrected thinkers, at which every person mentioned in the pages of this work is given a copy and invited to begin by reading the passages concerning himself or herself, and then to read alternately backwards and forwards until he has mastered the full thousand volumes. An ideal work of this title would fulfil the following conditions:

1. The person whose activities and writings are being described finds the description intelligible, except for the parenthetical remarks which say things like ‘This was later to be known as …’ and ‘Since the distinction between X and Y was yet to be drawn, A’s use of “Z” cannot be interpreted as …’, and he comes to understand even these remarks as he reads on.

2. On finishing the book, everyone described endorses the description of himself as, though of course insufficiently detailed, at least reasonably accurate and sympathetic.

3. The entire assemblage of the resurrected, at the point at which they have all read through the book, are in as good a position to exchange views, to argue, to engage in collaborative inquiry on subjects of common interest, as secondary sources for their colleagues’ works can make them.

This seems a plausible ideal for intellectual history because we hope that such history will give us a sense of Europe as (in the phrase which Gadamer has adapted from Hölderin) ‘the conversation which we are’.

The Data Journalism Handbook – a free, open-source book that aims to help journalists to use data to improve the news – has been released today at the International Journalism Festival in Perugia.

The book is an international, collaborative effort involving dozens of data journalism’s leading advocates and best practitioners – including from Australian Broadcasting Corporation, the BBC, the Chicago Tribune, Deutsche Welle, the Guardian, the Financial Times, Helsingin Sanomat, La Nacion, the New York Times, Pro Publica, the Washington Post, the Texas Tribune, Verdens Gang, Wales Online, Zeit Online and many others.

It is freely available online at datajournalismhandbook.org and a print version is forthcoming from O’Reilly.

Here’s an excerpt from the preface:

This book is intended to be a useful resource for anyone who thinks that they might be interested in becoming a data journalist, or dabbling in data journalism.

Lots of people have contributed to writing it, and through our editorial we have tried to let their different voices and views shine through. We hope that it reads like a rich and informative conversation about what data journalism is, why it is important, and how to do it.

Lamentably the act of reading this book will not supply you with a comprehensive repertoire of all of the knowledge and skills you need to become a data journalist. This would require a vast library manned by hundreds of experts able to help answer questions on hundreds of topics. Luckily this library exists and it is called the internet. Instead, we hope this book will give you a sense of how to get started and where to look if you want to go further. Examples and tutorials serve to be illustrative rather than exhaustive.

We count ourselves very lucky to have had so much time, energy, and patience from all of our contributors and have tried our best to use this wisely. We hope that – in addition to being a useful reference source – the book does something to document the passion and enthusiasm, the vision and energy of a nascent movement. The book attempts to give a sense of what happens behind the scenes, the stories behind the stories.

The Data Journalism Handbook is a work in progress. If you think there is anything which needs to be amended or is conspicuously absent, then please flag it for inclusion in the next version. It is also freely available under a Creative Commons Attribution-ShareAlike license, and we strongly encourage you to share it with anyone that you think might be interested in reading it.

Today the Public Interest Research Centre (PIRC), an independent charity “integrating research on climate change, energy and economics”, released a set of Climate Factsheets to help to communicate climate science research to a broader public:

Climate change research encompasses tens of thousands of peer-reviewed studies, decades of observations and the work of thousands of scientists. But too often this valuable knowledge doesn’t reach the people who need it most: climate change communicators & campaigners. This document takes the latest scientific research and translates it into practical factsheets on a wide range of climate change topics, ensuring that those responsible for communicating it to a wider public have easy access to the best available evidence.

They continue with a nice quote from Michael Le Page, writing in the New Scientist:

You can’t fake spring coming earlier, or trees grow- ing higher up on mountains, or glaciers retreating for kilometres up valleys, or shrinking ice cover in the Arctic, or birds changing their migration times, or permafrost melting in Alaska, or the tropics ex- panding, or ice shelves on the Antarctic peninsula breaking up, or peak river flow occurring earlier in summer because of earlier snowmelt, or sea level rising faster and faster, or any of the thousands of similar examples. … put all the data from around the world together, and you have overwhelming evidence of a long-term warming trend.

The factsheets cover a range of key topics, including temperature, arctic sea ice, drought, El Niño, flooding, heatwaves, snow & cold, species extinction, the seasons and wildfires:

Each factsheet contains a summary of key findings, context, background science, a section on ‘what the sceptics say’, a section on ‘what the scientists say’, a section on ‘what the data says’, further links and a list of contact details for scientists who work on the topic.

Nearly every claim contains a footnote with a reference to an academic paper, or to further information – with a total of 10 pages of references at the end of the document.

Finally, PIRC have released the set of factsheets under an open license and encourage others to reuse and redistribute them:

The Climate Factsheets are licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License. We actively encourage reproduction in all forms and by all means.

The factsheets look like they could be an invaluable reference for non-experts who want to know more about the evidence behind climate change. By trawling through dozens of papers and talking to dozens of experts, the authors give the reader lots of pointers for further reading and – perhaps more importantly – furnish them with the background knowledge they need to communicate some of the most important scientific research on climate change to others.

There are hundreds of public domain works scattered all over the internet – from well known projects like the Internet Archive, Project Gutenberg and the Wikimedia Foundation’s Wikisource and Wikimedia Commons projects, to national and international portals like Europeana and the nascent Digital Public Library of America.

And of course there are numerous small islands and islets that are disconnected from these bigger federating enterprises, clustered around different authors, topics, genres, and periods, run for and by special interest communities.

As a researcher and as a reader, I want to be able to browse across these different sources (I don’t mind where texts come from, I just want them to be accurate, to have a stable URL and not to have to trawl around too many different places to look for them). As a potential contributor to the commons, I would like to know which works are available and which are not yet available in digital form, so that I can try to scan and upload them myself or encourage libraries, archives or other institutions to do so.

Currently you can look for specific works on a case by case basis using search engines, or you can browse or search within specific collections, but it isn’t straightforward to get a comprehensive overview of freely available works by a given author. Wikisource is reasonably good for this, but bibliographies and links to digital copies are far from complete for many authors.

University students, researchers and teaching staff are very well placed to help to curate the commons of digital content, author by author, topic by topic, discipline by discipline. To do so, they need (i) an incentive to spend time on this, and (ii) a mechanism to contribute. I’d like to see if we could address some of these points with the TEXTUS project, an open source platform for working with collections of texts.

Regarding (i) one incentive would be to work on a project that recognised and trusted by peers and endorsed by well known institutions and scholars in your field. Hence rather than just contributing to a ‘public domain content’ initiative, you could contribute to a project relating to specific authors, works, or topics that you are doing your research on (a bit like Nietzsche Source, Darwin Online or Copyright History). Users could set up their own project at their own URL, curated by and for a specific scholarly community, each with its own editors, advisors and contributors. This is to recognise the value and importance of trust, reputation and peer review in scholarship. Ideally each TEXTUS instance should be a project that could be cited in a academic paper, and which contributors could list on their CV. This would be following in the footsteps of projects like Open Journal Systems, an open source platform for running open access journals that currently powers over 11,000 journals around the world in a wide variety of fields.

Regarding (ii) ultimately TEXTUS should have an intuitive interface that makes it very easy for scholars to upload scans, transcribe these into plain text and correct transcriptions, and create and edit scholarly bibliographies on different authors and topics. Ideally copies of scans should be uploaded to places like the Internet Archive, plain text transcriptions to Project Gutenberg and Wikisource, and additional and amended metadata records should find their way back into a shared pools of open metadata – just as the original CDDB system or the more recent MusicBrainz project let users share metadata about CDs.

Something like this would have the virtue of helping students and researchers to get more out of freely available digital content, and at the same time harnessing their expertise to curate and enrich the commons for all to enjoy.

Here’s a picture that shows how this might work:

If you’d like to follow our progress or help out, you can join one of the mailing lists or follow @textusproject on Twitter.

‘Community‘ symbol from The Noun Project.

Here is a preview of some illustrations for the Data Journalism Handbook, a free, open source reference book which shows how journalists can use data to improve the news. They were created by the talented Kate Hudson, based on the original designs she did for the book at MozFest 2011.

If you want to be notified when the book is released, you can sign up on the website.

CHAPTER 1: INTRODUCTION

What data journalism is and what it might mean for news organisations. Leading data journalists tell us why they think it is important and what their favourite examples are. Finally data journalism is examined in its broader historical context.

CHAPTER 2: IN THE NEWSROOM

How data journalism sits within newsrooms at the Australian Broadcasting Corporation, the Chicago Tribune, the Guardian, the New York Times, the Zeit Online, and elsewhere. We learn about how to hire developers, how to engage people around a topic through hackathons and other events, cross-border collaboration, and business models for data journalism.

CHAPTER 3: CASE STUDIES

Data journalists tell us about projects that they have worked on – from election monitoring to looking into how public funds are spent, from covering corruption and riots to in depth investigations into education and healthcare.

CHAPTER 4: GETTING DATA

Where to find data on the web, how to request it using freedom of information laws, how to screen scrape and crowdsource it, and how you can republish it and give others permission to reuse it.

CHAPTER 5: UNDERSTANDING DATA

How to make sense of your data – including tips on working with numbers and statistics, how to get stories from data, data journalists’ tools of choice, and how to use data visualisation to find insights in data.

CHAPTER 6: DELIVERING DATA

How to deliver your data to the public – from news apps, to data visualisations, to engaging audiences around your project.

If you have cunning ideas for where we should distribute them, or if you’d like to help distribute some in your area do get in touch!

She uses a process diagram to represent different aspects of the data journalist’s workflow – from getting data to delivering it to their readers in stories, visualisations and apps. The poster will be presented at the Information Design Conference 2012 in London. The design is based on Kate Hudson’s illustrations for the book.

The first edition of the book is nearly completed, ready for its launch towards the end of April at the International Journalism Festival in Perugia. If you want to be notified when the book is released, you can sign up on the website.

A few years ago I used to work at several college and departmental libraries at the University of Cambridge. One of the tasks which library staff regularly had to undertake was to cross reference the latest copies of all relevant reading lists with their collections, to ensure that they had copies of all the books that their students and staff needed.

When I was there, the libraries would print out copies of the PDF reading lists published by the departments and then go through each list with a pen or pencil, searching for each item in the catalogue and then ticking it off if they had it. Given that there are over 100 libraries at the university and over 100 faculties and departments which produce reading lists, that adds up to a lot of library staff time.

While I was there I volunteered to try to devise a way to make it easy for departments to create machine readable reading lists, which would mean that the process of cross-referencing them with library information systems could be automated. Then librarians could spend more time doing things that computers couldn’t do – like getting to know their users and their collections.

This received a luke-warm reception from other librarians I spoke to. I had an extended correspondence about this with a systems librarian at the Cambridge University Library, who was keen but busy. I also wanted to link to digital copies of texts which had entered the public domain – either from within the library catalogue or via an ancillary service – but curating and promoting access to freely available online resources (as opposed to subscription based resources) was not considered to be part of the librarian’s role.

Now, years later, I’m very keen to make it easy for people to create and work with machine readable reading lists using TEXTUS, an open source platform for working with collections of texts which is currently being funded by JISC. In particular I’d like to pilot this with OpenPhilosophy.org, to try to create more machine readable versions of reading lists from philosophy departments in the UK.

In this first instance, this would enable students and staff to easily find freely available public domain works which they could read on their computer or device, annotate and print out. In the future, with the help of some clever scripts, it would enable them to find copies of key texts in their local library, or in other libraries. In both cases lecturers could provide students with a single URL for their reading list, which would help them to find copies of the works they need that are scattered in a variety of digital and physical locations.

To make this happen I’d propose working with students and staff in philosophy departments to create machine readable versions of their reading lists. Once we have these, we can start to match them to digital copies and experiment with scripts to run against university library information systems. If you’d like to participate in this as a student, lecturer, or librarian, or if you’re just generally interested in making this happen – please do drop me a note.

Europe’s Energy, a project I helped to create to put EU energy targets into context, has just won a Silver Award at Malofiej 20. The Malofiej Awards recognise innovative infographics from around the world:

The Malofiej Awards have since 1993 given recognition to the best infographics published in print and on-line across the globe. Every March an international jury meets for nearly four days at the School of Communication at the University of Navarra in Pamplona (Spain). They select those works worthy of gold, silver or bronze medals from among dozens of entries submitted to the competition.

The awards are named after Alexander Malofiej, “an Argentinian cartographer considered to be a pioneer in infographics, and who died in 1987″. This year there were 1,356 entries from 151 media organisations in 29 countries.

The Europe’s Energy project was born out of a series of hackathons I organised in December 2010, which sought to encourage developers, designers and data journalists in several European cities to do useful and interesting things using information from Eurostat. Having a personal interest in climate change, energy and carbon emissions, I was very keen to look into datasets related to these areas. This became the focus of our event in London – and we looked into how energy consumption and production changed in different EU member states over time. The Guardian covered this and I did a writeup with a wishlist of other things we wanted to explore.

In early 2011, I spoke to a press officer at the European Commission who was looking for visual material that would help to put EU energy policies into context for a big meeting at the European Council. I spoke to Friedrich Lindenberg and Gregor Aisch and we decided to try and put something together to enable more people to understand what the EU energy targets meant, how ambitious they were, and how they looked in the context of energy production and consumption in different countries. Soon we were on regular calls with Dirk Heine, Guo Xu, Nathaniel Scheer, and doing late nights to try and get hold of the data we needed, to figure out what it meant, and how best to present it to the public. In just under two weeks the Europe’s Energy was born. In the last 24 hours we crowdsourced translations of the project into 16 languages and I wrote an article for the Guardian Datablog.

The project would not have been possible without the willingness of our team to drop everything and wade knee deep into EU energy policy, data wrangling and data visualisation design. It was supported by the LOD2 project, part of which aims to encourage more people to use open data to provide value to society.

There is a lot more work to be done to continue to gather, combine, interpret and present data related to climate change, energy and carbon emissions. Hopefully Europe’s Energy and projects like it will make some small contribution towards increasing public understanding of complex, but very important, issues. One hopes that Gregor’s beautiful graphics just might help to encourage a few more people to find out more about energy policy, and perhaps even to act to encourage decision makers to increase the share of green energy in the mix.

Earlier this week the Guardian, Forbes and others covered the discovery of 500 fairy tales collected by 19th century folklorist Franz Xaver von Schönwerth. I sent a note about this to Professor Jack Zipes, who promptly replied urging caution about the discovery and pointing to many other (in his view more interesting) 19th century collections from France and Germany. An expanded version of his note is now up on the website for the Sussex Centre for Folklore, Fairy Tales and Fantasy.

Says Professor Zipes:

I have only read Schönwerth’s tales from the earlier three volumes, and they range from boring to good examples of Bavarian customs. Nothing to get excited about, just as there is nothing to get excited about in the more recent example provided in The Guardian.

He continues:

I can point to some brilliant German collections by Theodor Vernaleken, Johann Wilhelm Wolf, Ignaz and Joseph Zingerele, Heinrich Pröhle, Josef Haltrich, Christian Schneller, Karl Haupt, Hermann Knust, Carl and Theodor Colshorn, etc. etc. and even more brilliant French collections by François-Marie Luzel, Paul Sébillot, Emmanuel Cosquin, Jean-François Bladé, Henry Carnoy, etc. etc. that contain tales fastidiously recorded by these folklorists, who translated them from dialect versions. They also include raw dialect versions with their translations.

And then:

There is also the question of artistic value. Many of the European folklorists like the Grimms, had a great artistic sensibility. The artistic power of the Grimms’ tales and other collections can be experienced when they are read aloud. I believe that the best folklorists always had to “translate” and “adapt” the tales they collected, and they did this while trying to remain true to the spoken word. So, you can praise Schönwerth’s “raw” tales, but those that I have read thus far lack an important element of artistic re-creation.

He says that “we have not yet translated the best European folk-tale collections into English and given them their due recognition” and that “the general public is not aware that Schönwerth’s work was just a drop in the bucket of folk-tale collecting in Europe during the nineteenth century”.

Salman Rushdie takes up the idea of a “Sea of Stories” from the Kathasaritsagara (literally: “Sea of the Rivers of Story”) an 11th century collection of Indian fairy tales and folktales. Liquid metaphors are an attractive way of alluding to the richness and reciprocal influence of various fairy tale and storytelling traditions. The many tales in One Thousand and One Nights wonderfully exemplify how narrative themes and patterns are echoed, refracted, and parodied. The epic, rhapsodic, near geometrical complexity that emerges from the relations between the tales leaves the reader with a taste of the infinite, a sense of awe that could easily be described as oceanic. Rushdie writes:

So Iff the Water Genie told Haroun about the Ocean of the Streams of Story, and even though he was full of a sense of hopelessness and failure the magic of the Ocean began to have an effect on Haroun. He looked into the water and saw that it was made up of a thousand thousand thousand and one different currents, each one a different colour, weaving in and out of one another like a liquid tapestry of breathtaking complexity; and Iff explained that these were the Streams of Story, that each coloured strand represented and contained a single tale. Different parts of the Ocean contained different sorts of stories, and as all the stories that had ever been told and as many that were still in the process of being invented could be found here, the Ocean of the Streams of Story was in fact the biggest library in the universe. And because the stories were held here in fluid form, they retained the ability to change, to become new versions of themselves, to join up with other stories and so become yet other stories; so that unlike a library of books, the Ocean of the Streams of Story was much more than a storeroom of yarns. It was not dead but alive.

While the Sea of Stories will remain a Platonic fantasy, one can imagine its worldly counterpart in the form of a comprehensive scholarly index of fairy tales and folk tales from around the world. One could explore the index by language, country of origin, date, and author. Perhaps one could explore linkages between early sources and contemporary retellings. Or explore tales by theme or trope, hopefully without falling prey to a Proppian hubris of comprehensive classification and analysis.

This is something that I’ve been wanting to pursue for a while as a project tentatively dubbed the ‘Synoptic Folktale Index’ with the Sussex Centre, which was founded by my dad. Several people have expressed support for the idea. Professor D. L. Ashliman has very kindly offered to donate his collection of Folklore and Mythology Electronic Texts to the project, as well as a large private database of bibliographic references. Professor Jack Zipes has given us several big bibliographies of works that he’s used for his own research and publications. The index could be powered by the open source TEXTUS platform, which would enable users to update bibliographies and upload, transcribe and translate texts. The index would enable users to see which tales have and have not been translated, and a translation fund could help to incentivise new translations.

In Rushdie’s world, the Sea of Stories is a fertile source for storytellers, whom, if they are brave and “very, very careful, or very, very highly skilled”, can “dip a cup into the Ocean” and “fill it with water from a single, pure Stream of Story” (ibid, p. 72). The protagonist of the book learns that “nothing comes from nothing”, “no story comes from nowhere”, and that “new stories are born from old” (p. 86). This is explained with reference to the digestive systems of “artistic Plentimaw fishes”, who help to generate new tales by combining (parts of) old tales.

Many German folklore collectors in the 18th and 19th century believed that folk culture was a fertile soil out of which new works could grow – by retelling, reworking, synthesising and incorporating traditional tales. Johann Gottfried Herder helped to popularise the idea that literary genius could grow out of folk culture – an idea which was fostered by his former teacher Johann Georg Hamann, who was in turn influenced by the vegetative metaphors of English poet Edward Young, who was reacting against what he considered the staid, formalist aesthetic theories of his contemporaries. The paradigmatic case to which German literary theorists after Herder turned was Shakespeare, whose works they believed had grown out of an abundant amalgam of tales, legends, and myths.

One can envisage that of the many thousands of folk tales that have been collected, some will be of historical interest to those specialist oceanographers who are interested in the subtle inflections and shades of variation in the great waves that roll across the Sea of Stories. But perhaps for the rest of us, the value of a given tale will for the most part be proportional to the talents of the teller.