Amplifying Hidden Voices in Primary Source Databases

By Lindsay Gulliver, Publishing Manager at AM and Erin Pearson-Willbery, Senior Editor at AM.

“Marginalisation speaks of lack of access, denial of equal economic opportunity, political access or representation, civic rights, and often even basic human rights; it implies institutionalised exclusion, discrimination, and the domination by one group over others; it results in the historical invisibility and silencing of those at the margin.” - Kathleen Phillips Lewis, “How to Research Marginal Groups” in Research Methods Primary Sources (AM: Marlborough, 2021)

Events that take place in the past are fixed, yet interpretations of those events are not and, in fact, can differ radically from person to person and group to group. No single source can reveal every facet of an event or a person’s experience of that event and that is why it is important to cross-reference a range of sources and, crucially, consider a range of perspectives to build up a fuller picture of history.

At AM, our mission is to create accessible, sustainable and inclusive websites which reimagine primary sources, empowering current and future generations to challenge, analyse and debate critically. Digitising content created by or for underrepresented communities is central to achieving this goal, but we recognise and acknowledge that historic collection practice has shaped the types of sources that have survived to the modern day, at times excluding the voices and narratives of non-hegemonic peoples from the preserved record. Further to this, we recognise that many communities record and share histories in ways that have not traditionally been collected or housed in archives, such as through folklore and oral tradition. We understand, too, that by digitising primary sources, we contribute to the continued survival and discovery of such materials, reinforcing histories which may exclude the voices of certain people and communities. Technology can play an important role in redressing the imbalance of representation and we’re excited by these possibilities.

Whether a researcher is using one of our resources to find a particular source or discover new evidence to support their argument, we know that good metadata is vital for navigating primary source databases. That is why our editorial teams think carefully about how best to describe and display the documents we digitise and why we are constantly striving to improve our processes.

Often, data from library catalogues informs the metadata at the heart of AM resources. Over the past 30 years, we have built strong and resilient relationships with archives and libraries around the world, and we are grateful for the catalogue data we receive. However, we also recognise that through accessioning and historic cataloguing, sources created by marginalised peoples and communities - from ancient times up until the modern day - will have been subjected to decisions by individuals from outside the creator community, and that we may inadvertently reinforce biases and inaccuracies applied to those sources. It is vital that we acknowledge where problematic or outdated metadata may exist and how best to contextualise or supplement that data. Recently, we have begun to publish language and terminology statements which detail the origins and potential challenges of metadata found in our resources and we will continue to implement these pages within our sites so that researchers are forewarned of problematic or outdated terms they may find.

It is not possible (or preferable) to erase biases held within primary sources themselves, but the ability to use exciting new technologies like Handwritten Text Recognition (HTR) software does afford researchers methods to bypass biases that may exist in the catalogue data.

HTR enables researchers to search thousands of manuscript pages at once, using software powered by artificial intelligence. Our in-house technical team regularly implements the most up-to-date version and, subsequently, we have seen its power and capability steadily increase over time. Back in 2017, we first applied HTR to Colonial America, our resource dedicated to the renowned Colonial Office (CO5) files held by the National Archives at Kew which document the history of the Thirteen Colonies from early interactions with Indigenous peoples through to the dawn of the American Revolutionary War. Since then, we have seen how well this technology can be used to navigate biases, revealing the voices of marginalised people and groups, and are committed to applying this software to all manuscript sources within our products, helping researchers to search historic texts freely. With one-click, researchers can exclude metadata results from their searches and return just the results within sources, thus amplifying the voices of figures who may not have been recorded in catalogues and improving discoverability.

Whereas primary source databases have traditionally enabled scholars to search the metadata applied to each record, this software promises greater fidelity of search than traditional cataloguing can offer. If researchers discover a named individual within a single source, they can now perform a search and return all results across every document in a resource regardless of whether that figure is a colonial governor - whose name would certainly have been captured in the catalogue from inception - or a colonised, enslaved or otherwise marginalised person - whose name may never have been recorded in the catalogue or known to history.

In 2021, AM published the first module of Colonial Caribbean. This resource includes nearly 600,000 images detailing the history of British rule in the Caribbean throughout the early modern period and a search for “slave” would return nearly 1,500 different documents. Professor Kristy Warren, in a video interview recorded for Colonial Caribbean, explains that interactions between the British state and enslaved people can often be uncovered when something “was felt to have gone wrong”, for example, when there is a legal case or petition submitted to the colonial governor. By using metadata filters for document type, searches for the voices of enslaved peoples in the Caribbean can be narrowed down using, say, “Petitions,” “Legal Documents” or “Newspapers.”

In addition to building search tools and filters into our platform, AM resources include commissioned essays and features - like Professor Warren’s interview - that are intended to support the critical analysis of the primary sources we digitise. This is particularly important when we publish sources created by or for unrepresented communities. We believe it is important to acknowledge where gaps may exist in the historical record and amplify information about groups or individuals for whom primary sources may not exist. Our essays, interviews and exhibitions encourage researchers to examine sources created by dominant powers and contextualise the impact that they may have had on non-hegemonic peoples; they also highlight more diverse sources and place those sources within their historic context while illuminating key research themes.

The award-winning second module of Sex and Sexuality focuses on self-expression, community and identity and includes primary source records created throughout the nineteenth and twentieth centuries. Amongst the academic essays and interviews commissioned to contextualise these sources, users will find a fascinating piece on the categorisation and self-description of trans people:

“In the early part of the twentieth century, trans people drew on the wealth of new terminology they had available in medical literature, and used new theories of sex to convey how they saw themselves. They called themselves inverts, homosexuals, transvestites, eonists, hermaphrodites, and feminine men … When researching and writing about trans history, then, it’s important to hold two things in your mind simultaneously. One is that transness as it exists now is rooted in a long history - trans people are not some newfangled invention, as some anti-trans rhetoric would have it. The other is that what sex and gender even are change over time, and so too do the categories that people use to understand their own experiences.” - Beans Velocci, “Changes Over Time: Making Sense of Trans Categories in the Archive” in Sex & Sexuality (AM: Marlborough, 2021)

As per the examples cited by Beans Velocci, above, it will often be vital to consider how people or events may be described in a historic document even if such terms are unacceptable today; in the Colonial Caribbean resource, searching for “runaway slaves” or “runaways,” for example - while archaic - will provide the most accurate results for enslaved people who made attempts to escape their enslavers. In addition to commissioning features which help users to critically analyse primary sources, AM also aims to publish searching guides to share practical information on how best to optimise search capability, with general tips and - where appropriate - specific examples.

Events that take place in the past are fixed. Historic collection practice has shaped the archival record that survives and, furthermore, the histories that have traditionally been written about both dominant powers and underrepresented communities. However, by acknowledging bias and gaps, commissioning contextual guidance, and integrating innovative searching tools into our platform, we at AM aim to provide the tools that will enable researchers to read against the grain and amplify the voices hidden within history.

About the authors 

Lindsay Gulliver joined AM in 2014 and serves as Publishing Manager within the Editorial Department. In addition to helming the company’s new learning tool, Research Methods Primary Sources, she has worked on the production of more than a dozen traditional collections.

Erin Pearson-Willbery is a Senior Editor at AM. Since joining the company in 2014 she has had the opportunity to lead several editorial projects and work on a broad range of primary source collections including, Sex & Sexuality, Ethnomusicology: Global Field Recordings and Colonial Caribbean to name a few.

This article was first published in Against the Grain, September 2022.