Research Data as Web Archives – Web Archives as Research Data*

By digitalethnographyresearch

Svenja Kunze** and Leslie Gauditz***

Good research data management is an essential to digital ethnography but also a challenging process. A wealth of information must be filtered along the lines of the research interests, but also collected, sorted, and stored in a technically and ethically appropriate way. Thereby, this process has many similarities with the work of archives and records management – disciplines which are currently in the process of adapting to the digital age. Alongside legal and ethical issues, such as data protection, copyright, and consent, it must be decided which data from the abundance of the web should be archived and in which formats, so that web-based information can still be easily displayed and used by researchers in the future; reason enough to enter into a collaborative dialog between social sciences and archiving to learn from each other.

The collaboration of digital ethnography and archiving under discussion in this blog post happened in the context of the research project “Emergent Norms in Corona Protests?” (funded by the Volkswagen Foundation’s ”Corona Crisis and Beyond” research grant). This was a joint research project of the Department of Sociology at the Helmut Schmidt University (HSU) in Hamburg and the research group Macro-Violence at the Hamburg Institute for Social Research (HIS), in cooperation with the archives at HIS. It ran from July 2021 to February 2023, thus investigating “live” during the second half of the Corona crisis and in its immediate aftermath. The project aimed to combine a sociological research interest – in protest during the COVID-19 pandemic – with an exploration of how digital expressions and organisation of resistance via websites, blogs, and social media can be captured and documented in the sense of an archival collection. This involved the authors of this blog post, an archivist and a sociologist, in discussion of research data at the intersection of data collection (digital ethnography) and long-term data preservation (archiving).

Where does Digital Research Data Archiving stand?

Their discussion took place against the background that the increasing digitisation of information confronts archivists and researchers with the task of adapting proven processes, methods and skills to the new technologies. The field of web archiving is at the nexus of research data archiving of digital data, i.e., making digital and web-based data from past research projects available for future validation or for new questions of the scientific community. Despite the relevance of these ongoing processes in view of the increase of data volume – not least due to the everyday spread of social media – it is only slowly progressing in the relevant disciplines (archival studies, sociology, anthropology, museum studies etc.) or even finding its way into university teaching. As Imeri, Klausner and Rizzolli (2023) discuss, drafting a research data management plan has increasingly become a requirement from grant givers and ethics committees but guidelines for how to write such plan are only being developed in recent years by few institutions and do not focus on digital data. Services for the preservation of qualitative research data, such as the Bremen Qualiservice, are still scarce, or depend in quality and accessibility on your local universities’ or libraries’ structures. And while archivists have been preserving large parts of the internet for decades (see e.g. Gomes et al. 2021) they usually only meet researchers when the latter use already archived material. Some projects in which researchers and archivists collaborated consciously were reflected in the network for web archiving (WARCnet, e.g. Weber 2020), but overall collaborative projects are still scattered and need to be brought into dialogue in order to jointly develop good processes. This is the context in which this blog post is written.

The ethnographic research and the data: COVID-19 protests

The ethnographical research on the COVID-19 protests, which was the basis of this blog post’s discussion, was interested in norms and shared understandings of protest participants. To understand the shared knowledge and issues of the protesters, we followed protest specific social media and messenger services on different platforms as well as livestreams of demonstrations. The data corpus consisted of screenshots (e.g. of chats), downloaded media (e.g. YouTube videos discussing health information, influencers discussing the measurements) and export files of social media. The research participation in social media groups was covert and thus without consent. Media which was being used for publication or presentation was anonymised if dealing with private people but not if it was content by influencers who posted publicly in such channels. Equipped with the ethnographic knowledge gained through following the social media, an interview study was conducted afterwards. At the time of this writing we have not archived the collected data as we had neither legal nor IT support and could not resolve all technical, legal and ethical issues during the funding period (see below).

The view from the archives

To briefly introduce The Hamburg Institute for Social research (HIS) and its archives: HIS was founded in 1984 as a private research institute. Apart from research projects in sociology and the humanities, it has as tradition of engaging with the professional and general public through its in-house publications (the internet portal “Soziopolis.de” and publishing house “Hamburger Edition”), or via its publicly accessible library and archives. The archives at HIS started this work in 1988, initially to support research projects on protest and resistance in the Federal Republic of Germany. Its collections still focus on social movements after the 1950s and have grown to hold more than 2,000 shelf metres of material (plus a digital archive), such as brochures and pamphlets, posters, newspapers and magazines, and more than 350,000 photographs and audio-visual materials. Although the archives are the largest on social movements in Germany, it is still a very small institution compared to university or state archives with its 5 members of staff. Some of its activities are described in its blog.

“Classical” archive and collection material. Images by Svenja Kunze/HIS Archives.

Most archived materials were produced by different activists, for example for campaigning, or refer to internal communication of protest groups. There are also finance or legal papers, or personal papers of (former) activists. Such “classical” media form more than 90% of the collections. Most collecting happens retrospectively: political initiatives hand over material they created and collected throughout their activities and sometimes activists donate their private collections (e.g. political pamphlets rediscovered when tidying up the attic during Covid lockdown). There are also acquisitions from antiquarian sales and auctions.

Classic archiving process. Images by Svenja Kunze/HIS Archives.

The archivists at HIS are good at archival processing “as they know it”, which means organising, describing and preserving information “on paper”, and this works well for documenting resistance and social movements up to the 1990s. Clearly, all of this classical material can be digitised, usually by being scanned, and this is done to improve access for users. But there is an ongoing question of how to document contemporary protests, as was the case with the COVID-19 protests, whose organising and communicating are volatile and ephemeral in the archival sense as they consist of so-called “digital born data”.

Archiving meets research data management

HIS archives mainly collect on protest and social movements, but are also responsible for
research data management within the research institute, including the long-term preservation of research data. Traditionally, data from research projects at HIS consisted of interview transcripts, (photo)copies of documents from other archives as obtained during historical research projects, (e.g. relating to war crimes), press cuttings or other published materials. These materials have been transferred to the archives retrospectively at the end of a research project.

For the project we now wanted to discuss how to manage social media data collected during the digital ethnography on COVID-19 protests immediately after collecting it. We learned from each other, what to look out for to manage the data in a way which not only benefits the current research needs, but also keeps the archival perspective of long-term data storage and access in mind. A big issue was the technological nature as data formats and structures should be suitable for (digital) long-term preservation.

Archivists do not have a specific research question in mind when they select material. Instead they preserve for a future use with a broad and unpredictable range of research questions in mind. For this, archivists need to know where material comes from, how it was collected and organised (the so-called metadata) and whether it is well suited for long-term storage. Researchers, on the other hand, select and organise their (digital) data throughout the research, along the lines of its topic and research question(s).

Circular data collection process vs. long-term preservation

As a digital ethnographer you are already busy with making decisions around data selection: Which channels should I follow? Which internet communities are relevant to the research question? Which posts should I save and analyse? Digital ethnography is a circular research process. Therefore, you don’t really know where you will end up when you begin. In the case of the COVID-19 protests the relevant media platforms even changed throughout the collection period, as deplatforming forced activists and influencers off of facebook or YouTube and into less “mainstream” platforms such as Telegram, Odysee, or Discord.

Screenshot Youtube by Leslie Gauditz.

All in all, the communication of the movement was scattered over a multitude of platforms. Channels and data formats differed accordingly. The collection contained screenshots and images (memes, photos, illustration) in the formats of *jpg, *gif, *png, *tgs, *pdf. There was also audio and video data (voice messages, memos, interviews) in the formats *ogg, *opus, *m4a, *mp4. Exporting the chat history in messengers of the comments of posts and videos happened via the data formats of *json, *xlxs, and *html. Lastly, the researcher herself produced digital data, such as notes and memos saved in audio or text files (*docx, *m4a).

The data analysis was done via coding. For this, any web content needed to be saved in a way that ensured a good visual overview, but also so that the resulting files should be compatible with qualitative analysis software such as Atlas.ti or MAXQDA (we worked with the latter). If not, they needed to be converted (e.g. opus to mp3)

It happened that for web archiving, i.e. the open-ended storage of web-based information, not all of these formats were suitable. A format specifically developed for archiving whole websites is *warc. Warc-files usually cannot reproduce the design in the same way as the original what creates an information loss for researchers.

Other formats which archivists can work with well are, at the time of this writing: *json, *wav, *mp4, *pdf, *jpg or *tiff in high resolutions. These formats are stable over time, facilitating storage and access, and commonly used so they can be shared and exchanged between different institutions.

Screenshot MaxQDA by Leslie Gauditz.

Lessons

There are three lessons to be learned for data management at the intersection of social research and archiving.

First, technological issues need to be harmonised. Data formats need to work for data analysis as well as for long-term preservation. It is advisable to consult or employ someone for technical support and involving such expertise should be part of project plans and grant applications. In bigger archives that work with digital data, technical support has become the norm since neither archivists nor qualitative researchers are usually sufficiently trained or specialised in the latest technological advancements.

Secondly, methodological affordances concerning the data selection and collection process need to be discussed. This involves technical infrastructures, but also the clarification and harmonisation of workflows.

Thirdly, there are ethical concerns that intertwine with legal requirements, which are dealt with differently in qualitative research and in archiving, and need to be brought into agreement. Social media data collected during research is more current than material that usually reaches an archive, thus the consent concerns a different time range. Our impression is that archived information is not as sensitive to people 20 years later. In ethnographic research trust plays a crucial role: When researchers work closely with specific people and (digital) communities, they often build a relationship of trust to gain access. This trust might be broken when archiving data. A person might consent with their information to be used in a specific research context, but do not wish that this data will be kept beyond the project, or to be used by other researchers. If consent forms are used, they need to cater to both needs, or rules of anonymisation need to be negotiated.

Concluding lesson: Embedding data management from the start

The project “Emerging Norms in Corona Protests?” served as a test case in order to start a dialogue between archival practices and social science research. Learning what colleagues from the other discipline wanted or needed involved a couple of meetings and some discussions. Only then we became familiar with each other’s work in terms of concepts and methods, standards, best practice and even the technical terms of the fields.

We understood that archives can support researchers at project planning stage and throughout the projects e.g. regarding legal requirements for collecting, processing, storing, and sharing research data (copyright, data protection), or meeting research data management requirements of project funders (open data, FAIR data). All of this involves raising awareness for the possible “afterlife” of research data.

In a nutshell: Interdisciplinary dialogue is necessary to improve archival work in the 21st century. For us, so far, the journey was the reward.

References

*This text is a shortened version of the presentation under the same title given at Web Archive studies network (WARCnet) Closing Conference at the University of Aarhus in October 2022.

**Svenja Kunze is Head of Archives at the Hamburg Institute for Social Research (HIS). She holds an M.A. in Modern History from the University of Münster as well as a PGDipl in Archives and Records Management from the University of Liverpool. Before joining HIS, she worked as a Project Archivist at the Bodleian Libraries, University of Oxford, and for company archives in Germany.

***Leslie Gauditz is a sociologist specialising in research on political conflict, activism, and qualitative methods. She holds a PhD from Bremen University. In 2021-2022, as a researcher at Helmut Schmidt-University Hamburg in the project “Emergent Norms in Corona Protests?”, she conducted a digital ethnography of social media that was used by people who protested restrictive measures and the vaccine during the COVID-19 pandemic.

Leave a Reply

Your email address will not be published. Required fields are marked *