On Tue 14th I returned to my former employer to attend a conference entitled Working with Paradata, Marginalia and Fieldnotes: The Centrality of By-Products of Social Research. I’d spotted this last week online, and managed to squeak in a place at the last minute thanks to a drop out. I wasn’t 100% sure the day would be that relevant to me, but since I’m doing social fieldwork as part of my PhD I thought it would only be to my benefit to attend, so at the very least to expand my knowledge a bit more. As it was it turned out to be an excellent, stimulating and enjoyable day; and the following is just the highlights. The day was introduced by Henrietta O’Connor and VC Sir Bob Burgess (UoLeicester)
- Paradata then and now (pt1): Heather Elliott et al. (Institute of Education)
- Looked at paradata on the field notes from the famous study from the 1967-68 by Peter Townsend on poverty in the UK. Neatly identified a typology of 6 types of paradata (amplification, justification, explanation, evaluative, debriefing and standpoint). These typifications come to life when we do a narrative analysis, depending on the sort of stories that the field researchers are revealing in their marginalia. Position within the field team hierarchy can also be an important issue.
- There was also an exploration of a narrative analysis of the sources based on this approach, demonstrating how the story of one informant in the study could emerge from the paradata – noting that the field worker themselves in the original study could have positioned the narrative to a degree. This reflexive approach also applied to the modern researcher, where their cultural position in considering the data may shape their interpretation. It was noted that where the field worker sat in the hierarchy of the research team and their own background (student, market researcher etc) could also colour their approach and their willingness to add useful paradata.
- The question of for whose benefit paradata notes were made and the audience to whom they were intended to speak was raised (reflecting on the work of Heather Jackson). The advantage for the field worker seems to be offering
opportunities for clearer, better coding, and provides an opportunity itself to provide further insight into their interactions with subjects. It was noted that
different styles of marginalia could be seen from different field workers. For example some who made a lot of paradata notes in margins ignored the space at the end which called for such additional comments. One commonly noted addition was field workers adding paradata/marginalia when they were skeptical of the responses from the interviewees; seeking a way to try and reveal the truth – even if the explicit research responses didn’t give this.
- Achieving high response rates and balanced samples in household surveys: The role of paradata. Mark Hanly (UoBristol)
- In this study there were three types of paradata – call records, interviewer observations and interviewer-respondent interactions. They were used to (1) inform the non-response results (2) Improve the field work in a cyclical process (3) Correct where it was possible to ID the non-respondents (statistically). There was also work reported on that related to doorstop recordings and how closely fieldworkers adhered to the interview script, noting that a low use of filler words (er, ahm, ah etc) was as counter productive in achieving a good interaction with potential subject as too many of them.
- Mark also talked about about digital/web paradata that can be captured using online surveys today. For example key stroke information can be automatically collected and is a cheap, effective and hidden way to capture additional information on how people approach surveys.
- It was also possible to capture device type information (browser, device model etc) as a user agent string and questionnaire navigation (mouse movement, clicks). This information can be used to reshape, develop and revise surveys so that areas where people struggle are enhanced. He also demonstrated a pop-up where people spent too little time thinking before answering a tricky question, and explained how this had helped to refocus participants attention and subsequently they would approach the survey answers with more consideration.
- Fieldnotes, marginalia and paradata in youth employment restudies, 1960-1985. Henrietta O’Connor & John Goodwin (UoLeicester)
- Very interesting talk which stemmed from research on the paradata left behind by an earlier failed social experiment and involved following up with some of the subjects and workers. Included the key point of setting up a file for fieldworker notes, so as to capture everything (ephemera and substantial materials) for future use, inspiration or analysis. Some useful discussions around the ethics of doing secondary research on materials and following up with subjects; given they had agreed to participate in the original research did this still apply 40 years later for follow up? Given that there was often candid data about their lives, habits and circumstances, it was an area to tread cautiously but positively for the benefits and insight that further research could bring out. Some interesting comments on the paradata from fieldworkers on their subjects demonstrating more positive insights into the circumstances of the better off, and more critical notes on those in the working classes.
- Issues over interpretation of fieldnotes paradata were raised, such as understanding shorthand and abbreviation; sometimes standardised across multiple fieldworkers but not always. In some cases going back to the original fieldworkers was the only way to demystify some of this. The same was true of some of the actual data where numerical codes were note associated with any key, and needed additional input to translate. However, the conclusions were that massive information on subjects can be just hidden in the paradata – e.g. medical information, that otherwise would have lost or unrecorded. Trends to archive research data digitally may mean that this sort of paradata would be lost.
- Paradata then and now (pt2): Dave Gordon and Eldin Fahmy. (UoBristol)
- I confess of all the talks, this was the one where my concentration or interest probably lapsed the most. Not sure if this was a result of the subject matter, post lunch fugue or due to the heavy qualitative element involved. Did highlight that paradata in C20th is easier to capture and work with, whereas C21st (and late C20th) is harder, due to being born digital. Talk also covered aspects of behaviour coding and cognitive coding.
- Marginalia and the History of Reading: The example of the UK reading experience database. Shafquat Towheed (OU)
- A slightly different tack coming from an English rather than social scholar, but did raise some very valid points about the loss of paradata through the work of archivists to clean up books that have been annotated by readers. Chimed with earlier comments about digitsation of field notes which also lose this additional data as it’s not considered key data by some doing the digitisation process. Had some interesting points also on capturing stealthy surveillance data, and some lessons from the Mass Observation experiments of the 1930s and 40s. Finally covered an examination of the prolific reader and writer Vernon Lee, and the value of capturing her annotations on the hundreds of books she had read and written about. A key lesson to come from this was that paradata is produced by researchers in the process of doing their research all the time, and what is peripheral or accidental in one field may well be core for other fields. As paradata are are temporal, contingent and vulnerable to loss their retention should be included in research and institutional data policies and practices.
- Lunch Discussions
- Over lunch I was lucky enough to sit down with 3 other PhDs and one soon to be PhD students, where we had a range of discussions largely about the changing HE environment and the scholastic nature (or lack thereof) of undergraduate students as the result of the perceived dumbing down of the school curriculum. Given my recent thesis chapter on marketisation of HE I had a few things to throw into the ring. Our discussions carried on so long, that we had to suddenly scamper back up the stairs for the afternoon session!
Three key things I learned during the day:
- Is it MaringAYlia or MarginAHlia. Turns out the pronunciation doesn’t matter, although one sounds much posher than the other.
- What Marginalia and Paradata actually are. Annotations in the margins and notes scribbled by researchers around their field notes which can amplify, enhance and reveal more than just the raw data.
- Set up a file. A simple tip for fieldworkers, and one that I’ll endeavour to follow – but having a box file to just throw everything into (including random thoughts, and scribbles) is as valuable a resource for the original researchers as it is for future secondary researchers.
For the back channel discussion (and there was a lot of it) – have a look at the twitter feed generated from the day. For once you’ll see it wasn’t mainly me (partly as I was trying to type notes, and partly because there were times when I couldn’t think of anything intelligent to say). Overall though the day was well worth my £4.70 rail ticket, and the refreshments and facilities were excellent. My thanks to all who participated and organised the day.