Observing the web

Abstract

It is ten years since the concept of Web Science was conceived against a backdrop of a dramatically evolving Web. At the time social networks were in their infancy and linked data was only talked about in the research labs. Today as we celebrate the 25th anniversary of the birth of the Web and we become increasingly aware of the pressures on it going forwards, I see Web Science as more significant to the evolution of the Web than ever. It is so important we can present real evidence to the major stakeholders in the development of the Web to demonstrate the likely consequences of decisions they might take. It is my hypothesis that we cannot achieve this without establishing an international project to observe what is happening on the Web and to set this in the wider social context and what has happened before. This means developing mechanisms to share data and data analytics across different projects and cultures over time -- including the creation of "safe harbours" in which private and public data can be integrated with out compromising privacy, confidentiality, or data security issues. This thinking has led to the establishment of the Web Observatory project, the latest incarnation of which will be presented in this talk. There are many such observatories in existence. The grand challenge is to create a distributed framework to facilitate the virtual integration of the data that resides in the various repositories and the sharing of data analysis tools as well as the results of the research that such international collaboration will engender.