Challenges for the development of fair language-based assessments of health, education, behavior, and beyond

Linguistic behavior serves as a reliable, inexpensive, and increasingly automated resource to assess different aspects of individuals and societies, ranging from education to health and behavior. Speech samples might help detect incipient health issues; newspaper corpora are used to identify stereotypes and implicit societal biases; and wordlists are the basis to determine verbal development. However, these and other relevant developments (which we label language-based assessments or LanBAs) have been concocted, tested, and deployed primarily on just a handful of large and commercially central languages, with English dominating the scene. Since the 6,500 extant languages can and do vary substantially (and in every dimension), transferring LanBAs from English to them is often fraught with important technical and linguistic challenges, which might not always be appreciated by users and researchers. The consequences of this bias, which we are only starting to understand, is that speakers and signers of minority languages have at their disposal more expensive, less efficient, and potentially biased LanBAs. A novel source of worldwide inequity looms large across multiple social arenas. We propose to address this complex and multifaceted issue by gathering a diverse set of experts (covering cognitive neuroscience, digital humanities, comparative linguistics, developmental science, and cultural psychology) with three main tracks of activity. First, we will critically synthesize the scientific evidence revealing the Anglophone bias in LanBAs, aiming at finding differences and commonalities across our fields of practice. Second, we will engage policy-makers, experts on language technologies, and other non-academic agents with the purpose of building a clear course of action to transfer our findings into practical recommendations that can improve the current state of the art in the development of LanBAs. Third, we will engage with the general audience through diverse media strategies, including filming a mini-documentary on the topic of the group over the visits to the different IAS.