How to Make AI Equitable in the Global South

(Illustration by iStock/ArtemisDiana)

In 2021, I and my colleague Ishita Rustagi asked how social change leaders and machine learning developers could advance “gender equitable AI.” By that, we mean AI that purposefully and proactively promotes greater inclusion and gender equity, not only preventing discrimination but prioritizing inclusion at the core and addressing historical inequities in both the development and management of AI tools. AI technologies are largely developed in the Global North–without necessarily considering differences across and within developing countries—and so, the international development community has an important role to play to support and advance approaches for more equitable AI.

That article helped inform how the Innovation and Technology divisions at USAID were planning their first grant mechanism related to AI in international development, the Equitable AI Challenge, which invested in innovative approaches towards identifying and rectifying gender biases within AI systems, specifically in a global development context.

Three years, five completed projects, and one Equitable AI Community of Practice later, what have we learned about gender equitable AI focusing on the Global South? And what is the role of international development practitioners?

1. There are persistent data gaps from marginalized communities in the Global South. Collecting more inclusive and equitable data requires clear intentionality. Digital gender data gaps stem from differences in Internet and smartphone access and use, which are larger in many Global South countries, and these data gaps impact what and from whom machine learning tools learn. When the University of Lagos and Nivi partnered to enhance the gender awareness of a health chatbot deployed in Nigeria, they needed to collect additional health data in Nigeria to better understand gendered differences in health queries and challenges. At first these efforts overrepresented single men, married women, and people in urban settings: the team had to be very conscious about what aspects of identity to prioritize, while working towards iteratively and intentionally building representation amongst different demographics.

2. Fully understanding how gender inequities manifest in data can be challenging. When Itad worked with the Mexican State of Guanajuato's Ministry of Education to identify and mitigate gender bias within Guanajuato’s new AI-based early alert system, they found a variety of potentially gendered variables from which the algorithm could learn. This examination was prompted after the team identified a four percent gender bias in the tool and sought to determine how gender might be unknowingly reflected in certain variables: for example, when considering attendance as a feature for school dropout risk, girls may inadvertently be penalized for missing school due to their monthly period.

3. The danger of being gender “blind.” Researchers from the University of California-Berkeley and University of Houston partnered with RappiCard Mexico to develop a gender-differentiated credit scoring model (i.e. a female-specific credit scoring morel and male-specific credit scoring model). The team found that the female-specific credit scoring model improved loan approvals for women compared to a pooled model as it removed some model bias present in the pooled gender “blind” model without giving up predictive power. This example illustrates how gender “blind” algorithms can ignore or hide existing inequities, inadvertently embedding them, while also making it more challenging to assess if discrimination is occurring. The same can occur with other demographics, such as race.

4. When data doesn’t capture gender identity, it complicates efforts for gender equitable AI. The College of William & Mary’s AidData, in partnership with the Ghana Center for Democratic Development (CDD-Ghana), evaluated gender bias in AI tools that estimate household wealth and are used to inform welfare distribution. The team needed to identify gender head of household to assess potential gender bias in the household wealth estimations. However, many of these AI tools learn from and use existing Demographic and Health Survey data, which are anonymized and clustered to protect privacy of individual households and don’t include gender information. To conduct the gender evaluation, the team had to use other information and imperfect gender-based assumptions to assign “household gender”. As another example, Nivi identified gender of users through their Facebook accounts while also asking for gender directly. However, the self-identified gender for an individual on Facebook was different than that on the tool, complicating approaches to support gendered health information (perhaps because of transgender or nonbinary individuals putting different answers in different places, or shared accounts and devices among family members).

5. Lack of transparency leads to trust gaps. When AI tools provide recommendations or predictions, people decide whether and how to act on them. However, with minimal transparency on the inner workings of AI systems, people can either not trust the technology—and therefore may not use it—or use the outputs without question. For example, the Itad consortium saw implementation problems with teachers who were provided the results of the AI-generated predictions for school dropout risk, but did not understand how to question results produced by the AI model, or how to use them appropriately.

Where to go from here?

To start: Equity is not the status quo. Hence, equitable tools need to be planned for in advance and throughout the process, and starting with this expectation can provide a north star for a variety of considerations: who is on the team, who is missing, who is represented or under-represented in the data, as well as more technical considerations around data inputs, data documentation, algorithm development, and ongoing management.

For international development practitioners, we recommend five key practices:

1. Work responsibly to build representative datasets. While digital inclusion efforts supporting smartphone and Internet access can reduce digital divides and provide troves of data that can help address gender data gaps, it is critical to consider privacy and safety implications. Consider partnering with local organizations that can help with data collection and management as well as identify appropriate safety considerations. International development practitioners should explore data cooperatives and other mechanisms that consider power in dataset development and ownership. Ensure that people can self-select their gender identity and other demographic information.
2. Meaningfully (and immediately) engage the people who will be using the tools. Many foundation models and AI tools are developed in the Global North, often lacking data representing diverse communities globally and a deeper understanding of contexts where they may be implemented. Employ team members from the geographies where the tool is being implemented and work with target users in design, validation, and testing as co-creators. Furthermore, social scientists or gender experts in teams can be key to help identify gender-related gaps in the development of AI systems and help track impacts.
3. Track and control for gender in algorithms. It may not always be clear how gender norms are reflected in data and algorithms, so it is important to audit tools to understand performance across demographics.
4. Prioritize transparency and training to enhance trust. Use resources that outline what is in datasets or models (such as dataset nutrition labels or model cards) and build in transparency measures for non-technical stakeholders. This can include conducting training for stakeholders to understand how to responsibly use and manage the AI tool(s), ethical concerns, gender equity implications, and strategies for responsible deployment and management. More efforts and funding are needed to explore what it means to be transparent to various global communities that have lower digital literacy.
5. Conduct risk assessments and assess impacts over time. Risk assessments can be implemented before or after developing a product to explore potential harms, biases, privacy considerations, and transparency issues. Assessments can also include how the tool is being used over time and by whom, its impacts, as well as any issues for equitable access by different communities or stakeholders. Part of the assessments can include ongoing audits, including assessing performance across different gender. Technical tools exist to support such audits, but they are insufficient to understand the range of potential harms, since they have largely been developed in and for Western contexts, and focus more on technical forms of bias or fairness versus broader ways that AI tools can advance discrimination.

For funders, more research and projects exploring equitable AI models and datasets are needed, as well as participatory processes that prioritize equity, inclusion, and agency. Funders can help address data gaps by supporting data collection efforts that prioritize community agency and control. (An exciting new fund mechanism, the Data Empowerment Fund, aims to do just that.) When funding development and implementation of AI models and applications, prioritize applicants that center equity, integrate risk assessments, and provide funding support for tracking impacts as well as lessons learned. But funders must also invest in equitable processes—as opposed to investing in a particular tool output—that prioritize the agency of marginalized communities globally and truly put their needs, values and perspectives at the center. Funders can also support community building amongst practitioners, researchers, and policymakers – like USAID’s Equitable AI Community of Practice.

Looking Ahead

As the use of AI continues to grow, there are incredible opportunities to ensure these tools are developed and implemented to enhance gender equity. We cannot passively assume that AI “for good” will necessarily result in the equitable outcomes we desire. Doing so would mean repeating past failures of the development sector, with higher stakes. Given the opacity and scale of AI tools, particularly in this new age of generative AI, we are headed for more disastrous results than unused, rusting technologies that litter the history of the development industry. As international development practitioners and funders, we must hold ourselves to higher standards, not only pursuing tools that are “less biased” than the status quo, but tools that are emancipatory and center equity.

We must not fall into the seductive traps of “techno-solutionism,” and imagine that social problems can be solved by technology alone. It can be tempting to invest in the development or scaling of digital technologies including a particular AI technology. In some cases, that is an important move alongside implementing these recommendations and rigorous evaluation. However, by investing in equitable co-creation processes that prioritize the agency of different people and communities and being open to both technological and non-technological solutions, we can more responsibly reach the goals we desire.

Read more stories by Genevieve Smith.