Dr. Judith Bishop, Senior Director of AI Specialists at Appen – Interview Series

Dr. Judith Bishop, is a Senior Director of AI Specialists for the APAC/US region at Appen. She is leading and growing a top-notch team of highly qualified and experienced linguists, computational linguists, and experts in all modes of human communication (speech, writing and gesture), to deliver AI training data with an unrivaled combination of quality and speed.

What initially attracted you to linguistics?

I first heard about linguistics from a favorite English teacher in high school. I was one of those kids who are equally drawn to foreign languages and humanities, and math and science subjects. Linguistics is the science of how language works, so it brought those interests together for me. Like so many people, once I learned about it, I was completely hooked. What could be more fascinating than how we communicate our thoughts and feelings to each other? Linguistics explores the language structures that, for all the differences in sounds and writing systems, are often similar under the surface, since all are a product, ultimately, of our common human existence.

Could you share the genesis story of how you found yourself working in AI?

I’ve worked at Appen since 2004 supporting the development of language technology products and services. Over this time, AI has emerged as a comprehensive framework, mission and vision for technology to mimic and extend human capabilities of communication, reasoning and perception. In 2019 my team rebranded itself as AI Specialists, recognizing that our linguistic and language knowledge is critical to the AI enterprise. Our annotated data provides essential support for the success of human interactions with AI products and services.

You’ve been working in AI for over 16 years, what are some of the biggest changes that you have seen?

The major shift has been a diversification of focus from core technology development to the long tail of use cases and applications. For most of my career, the focus of language-based AI has been to develop and refine a core set of models that mimic human speech perception and production, namely, speech recognition, speech synthesis, and natural language processing. Datasets typically conformed to common labelling and data sampling standards and conventions, such as those developed by the Speecon consortium (Speech-Driven Interfaces for Consumer Devices.) These standards have allowed core technology developers to benchmark their performance on common data structures and supported the rapid evolution of AI.

The pervasive expansion of AI use cases in more recent years, however, has brought with it the recognition that the core, generic AI models built with this data do not work adequately on more specialized data types without further tuning. Moreover, having been developed on data that was deliberately clean and ‘standard’, these models must now be trained or updated to understand and respond to all the diversity of human inputs: all dialects, all accents, all ethnicities, all genders, and all other dimensions of human difference.

Could you discuss the importance of unbiased data in machine learning?

Machine learning models, whether supervised, unsupervised or reinforcement learning models, will reflect biases that are present in the data they are trained on. Alyssa Simpson Rochwerger and Wilson Pang provide several excellent examples of this issue in their recent book, Real World AI. If there is insufficient training data for a segment of the population, the AI model will be less accurate for that segment.

In another common case, the representation of the population may suffice, but if the training data contains correlations between data points that reflect actual, but undesirable, conditions in the world (such as a lower rate of full employment for women, or a higher rate of incarceration for African Americans,) resulting AI applications can reinforce and perpetuate those conditions.

Associations present in language at large can create biases in NLP…