OpenSource.net

For decades, the topic of inclusion and marginalization of underrepresented groups in Open Source software communities has been a prevalent topic of discussion.

There have been recurring studies and reports on the state of open source participation that spotlight persistent issues that plague these communities; Marginalization of certain individuals based on their race, gender, ability, skills, etc. For instance, in the women constitute about a third of tech workers while in Open Source, women, along with other gender minorities, make up less than 10% of the population. Another common disparity that’s often overlooked is the low employment rates of neurodiverse groups – some 15-20% of the global population – despite evidence that their work in the tech sector is more consistent and productive.

The question of how an Open Source community can break down these barriers to participation and foster more inclusion and belonging for all individuals, regardless of abilities, skills, experiences and differences, is key to Open Source community health. 

DEI metrics and initiatives have been introduced to play a crucial role in addressing this question. In October 2022, the Linux Foundation’s Community Health Analytics in Open Source Software or CHAOSS Diversity Equity Inclusion (DEI) Working Group launched an interview campaign to explore how CHAOSS DEI metrics are perceived and to identify ways to enhance the efficacy of these metrics in addressing these barriers. In this article, we present findings from these interviews that shed light on the role of DEI metrics in overcoming social barriers in contributing to open source. The interviews highlight the common challenges of DEI initiatives and their impact on Open Source projects. We’ll explore how these DEI metrics can contribute to overcoming social barriers and create more welcoming environments for contributors from all backgrounds.

The current state of DEI in Open Source

For a better understanding of how CHAOSS DEI metrics can impact Open Source community health, let’s dissect recent studies and findings that showcase the state of DEI in Open Source.

Barriers to diversity and inclusion

• Exclusive language and behavior: The use of exclusive language, threats, disrespectful behavior and online harassment can alienate potential contributors, making them less likely to participate. During our interviews, some participants admitted that they often experience microaggression in day-to-day interactions in their respective communities. Additionally, reports from the Linux Foundation 2021 research also indicate that women, non-binary, LGBQ+ and people with disabilities were twice as likely to have experienced threats of violence in the context of an Open Source project. Transgender respondents were three times as likely to face these threats. 

• Lack of representation: Some Open Source communities still struggle with the challenges of homogeneous leaders and members. A research paper published by IEEE found that the lack of gender diversity remains an ongoing issue among Open Source projects. However, many of the projects also suffer from a lack of female inclusion in leadership positions. When underrepresented groups do not see individuals who look like them in leadership roles or as active contributors, it can be discouraging, making them less likely to get involved. The Linux Foundations report also indicated that ​​although 82% of participants felt welcomed in Open Source, demographic segmentations show varied sentiments. The 18% of those who do not feel welcome were from disproportionately underrepresented groups: people with disabilities, transgender people and racial and ethnic minorities. During our interviews, some participants pointed out that in some communities, while minority groups are encouraged to participate and contribute, they are often overlooked for leadership roles.

• Socioeconomic barriers: The 2023 “Open Source Maintainer” study indicated that Open Source contributors and maintainers are mostly students looking to build on their skills and experiences. That requires access to resources including a stable internet connection, a powerful computer and free time to contribute. All of these factors pose a challenge and create disparities in participation. When the question: “Can you describe how your unique attributes, traits, characteristics, skills, experience and background have affected your participation in open source?” was asked, one interviewee even  stated that: “In some communities with lots of different people from various backgrounds,  the presence of various backgrounds doesn’t always translate to equity — in the sense that people who already have access to resources and advantages end up getting more, making the rich even richer.”

• Education on DEI: Recognizing and addressing common disparities that exist in our communities is essential for progress. It’s nearly impossible to effectively address these matters without first acknowledging their existence and comprehending their impact. However, during these interviews, we discovered that over 40% of the interviewees were not knowledgeable about CHAOSS DEI metrics with most of them just hearing about them for the first time. The Open Source maintainer research study further revealed that fewer than a third of interviewees recognized the presence of formal DEI programs in their projects. This highlights the urgent need to improve awareness and drive initiatives that foster diversity, equity and inclusion in Open Source. 

• Accessibility: A critical aspect of Open Source software that often remains overlooked is accessibility. Unfortunately, a significant number of Open Source projects do not prioritize accessibility in their software, communication platforms, or technical documentation. This oversight has a profound impact on contributors with disabilities, who often experience a sense of exclusion when attempting to engage and collaborate with these communities. In our interviews, participants also mentioned that from their experiences,  accessibility is frequently an afterthought, trailing product releases.

• Implicit bias: Unconscious biases are also common factors that affect the way contributors from underrepresented groups are welcomed and how their ideas are received. These biases significantly influence the decision-making in a community, in turn affecting inclusive participation. Although actions stemming from biases are unintended, they end up hindering the progress of diversity and inclusion efforts in an Open Source community.

What are DEI metrics?

CHAOSS metrics are indicators used to assess Open Source community health. The DEI metrics aim to evaluate and reveal the diversity, equity and inclusion efforts within open-source projects and communities. These metrics are used to:

• Quantify diversity: By collecting and evaluating data on the demographic makeup of contributors, DEI metrics initiatives provide a clearer picture of the community’s diversity. This data can help identify gaps and track progress over time. This data can help our leaders make informed decisions that create safe, welcoming, and equitable developer environments.
• Reveal disparities: DEI metrics could reveal/identify, to some extent, incidents of both explicit and implicit bias that might exist in a community. For instance, in communities where members with disabilities, women, or people of color are not considered for leadership roles, this could signal a bias in promotion or recognition. 
• Monitor inclusive practices: Metrics observe incidents of harassment, the use of inclusive language, contribution participation, and other behaviors that affect the community’s inclusivity. They ensure access to resources (e.g., code of conduct) and understanding whether barriers to entry, like socioeconomic factors, limit participation. For instance, in a project with many non-native English contributors, a lack of translations in key languages for code, documentation and guides poses a challenge. Another known barrier to Open Source participation is that much of it relies on voluntary labor, while the majority of unpaid care work falls on those in marginalized groups. Metrics can help us understand how to support diverse users and accommodate their needs.

Open Source projects can leverage CHAOSS DEI metrics to proactively address biases, expand the talent pool and ensure belonging, ultimately breaking down social barriers and promoting diversity in Open Source communities at large.

Why an interview campaign?

Since underrepresented groups have the most potential to benefit from DEI metrics in Open Source, we launched an interview campaign to help us evaluate actual DEI activities in OSS and how they relate to the metrics we presently measure at CHAOSS through interviews.

This study aimed to help us comprehend the impact of current DEI metrics in real-world circumstances, especially for underrepresented groups in open source. Identify new metrics and methods for measuring DEI and also identify gaps to be addressed in current DEI metrics. 

Do metrics address all of these challenges?

While conducting interviews, we discovered that DEI metrics, though effective, may only provide a partial solution to some of the DEI challenges in our communities. Some participants shared valuable insights about the use of diversity metrics.

For example, some interviewees expressed concerns about relying solely on metrics to gauge progress. They argued that merely tracking the increase in the number of diverse individuals, such as going from 40% women in 2022 to 50% women in 2023, might not be a fully positive way to measure change or growth. Instead, the focus should be on whether the product or Open Source community has remained strong or improved since that time. If the answer is yes, then this improvement may be attributed to an increase in diversity.

Another participant emphasized that people from diverse backgrounds have unique perspectives and mindsets. They suggested that general metrics might not be inclusive of these diverse perspectives because they are often created by a group of individuals from one community without perspectives from the world. Since diversity can mean different things to different people, it’s hard to say whether these metrics address all of these challenges from individual perspectives. Hence, using the same pool of individuals for the creation of DEI metrics could influence their credibility. 

Some participants raised concerns that relying on metrics might lead to unfair judgments of people’s abilities and skills. Others shared their experiences of being assigned more unpaid work because they were the only diverse representatives for an organization. Additionally, some feedback questioned the effectiveness of DEI metrics in communities or groups with a higher proportion of individuals who are neurodivergent, stating that it can lead to challenges in understanding and adhering to the customary social norms or expected behaviors within that community.

In terms of successfully implementing these metrics, we understand that DEI metrics alone may be unable to address all DEI challenges if the barriers of these DEI are not recognized and tackled. Overall, feedback from interviewees emphasized that it’s not just about creating metrics but empowering people to understand and use them. So, even if you have metrics, they will only be effective if people are aware of them and use them to make changes for the better.

The impact of CHAOSS DEI metrics on Open Source communities

CHAOSS DEI metrics have found application in various scenarios, spanning software development, governance and policies, events, communities and beyond.

The interview campaign comprised an anonymous survey with over 180 participants and one-on-one interviews with 19 people. 

During the interviews, we asked participants to share their DEI experiences in Open Source. The goal was to help us understand how their differences may have influenced their participation and how communities can leverage metrics to address some of these barriers.

Based on these interviews, we gained insights from participants who shared real-time examples of how these metrics have been instrumental. They shed light on how CHAOSS DEI metrics have positively impacted individuals and communities.

• Some participants mentioned they have greatly benefited from the CHAOSS DEI metrics at events they attended in the past and these metrics were enforced to create a more inclusive event culture. Notably, they appreciated Open Source events that considered attendees’ safety, diverse speaker line-ups and live captioning for some talks, which was especially valuable to attendees who may be hard of hearing. Additionally, they indicated that the availability of diversity access tickets eliminated concerns about participation cost.

• For event organizers, the CHAOSS metrics on event diversity offered valuable insights into better connecting with attendees and speakers, handling conflict cases and improving the overall event experience. They emphasized that DEI metrics were pivotal in encouraging active participation and enforcing a code of conduct—for example, the State of Open Con23 and Open Source Summit Europe 2023.

• Participants indicated that these DEI metrics were utilized in their project to ensure smooth collaboration and healthy communication among team members. They also added that these metrics helped assess their progress, particularly in areas like leadership and mentoring and in supporting contributors with diverse backgrounds and skills, ensuring inclusive contributions. Example the Apache Traffic Control project.

• A participant said the DEI metrics were being utilized in the  DEI research that was conducted by the Apache Diversity and Inclusion project. They said these metrics were being incorporated to help determine actionable steps to take based on their research outcomes.

• Some participants also said the CHAOSS metrics have been instrumental in the development of a code of conduct and in implementing inclusive practices in their communities and projects. The Good Docs Project was cited as an example of this.

Harnessing the power of DEI metrics in FOSS

After our interview campaign, it became clear that CHAOSS DEI metrics have the potential to address various challenges and help address barriers to contribution within respective communities.

Diversify recruitment

• Metrics on community activity can be used to diversify the recruitment process and offer equal opportunities to contributors regardless of their differences.
• Metrics on labor investment can also be utilized by Open Source leaders to understand how much value a potential candidate provides to a project as they contribute. 
• Metrics on contributor onboarding, mentorship and contribution attribution can help community leaders identify a person suitable for a leadership role or full-time hire and subsequently craft a position that aligns with their strengths.

Diversify leadership

• Metrics can be used to diversify the leadership board and serve as a key performance indicator for community health.
• Metrics on board and council diversity can help in promoting inclusive representation within communities.
• By improving the power structure within communities, most Open Source communities can avoid dictatorships among leaders.
• Organizations and community leaders can use DEI metrics to brainstorm strategies for diversifying their communities and to assess whether they are effectively fostering inclusive participation.

Inclusive event participation

• Metrics are helpful for their event, promoting inclusive events and cultures. 
• Metrics on public health and safety can be used to ensure participants feel safe to attend in-person events regardless of their differences.
• Metrics on the event code of conduct are helpful to organizers in figuring out better ways to relate with attendees.
• Metrics on diversity access tickets also help participants from underrepresented groups attend events.

Psychological safety awareness

• Metrics on psychological safety and burnout are essential for creating a safer environment in Open Source, where individuals can freely communicate their concerns
• Metrics also encourage the need to provide support to all contributors and maintainers.

Inclusive contribution and retention

• Metrics on mentorship can be employed to help mentors and mentees collaborate better. 
• Metrics on leadership and mentorship can help encourage more folks to participate, also giving equal opportunities to contributors regardless of the differences.
• Metrics relating to contributor onboarding and conversion rates can be used to discern the progression of new community members as they evolve into enduring and engaged contributors.
• Metrics on mentorship address the common barrier of entry for contributors, especially those from underrepresented groups.

Accountability 

• Through the CHAOSS project badging initiative, metrics can be used to hold Open Source projects accountable for diversity and inclusion efforts. By quantifying the DEI status quo, communities are more likely to take action to address disparities.
• These can be used to tackle the segregation, language barrier, similarity, and confirmation bias that exist in most communities. 
• CHAOSS DEI metrics can cover access to knowledge and education so more people are aware of these differences and how to respect and abide by the code of conduct.

Data-driven decision-making

• Metrics can be used to gather data on demographics, diversity in contribution roles and psychological safety.
• Metrics can help communities in creating and enforcing codes of conduct.
• Metrics enable communities to make informed decisions to address specific DEI challenges.

Conclusion

CHAOSS DEI metrics are becoming invaluable in addressing DEI challenges in Open Source communities. While they hold the potential to transform these communities into more inclusive and equitable spaces, it’s essential to acknowledge and address the associated challenges, foster understanding and champion the adoption of DEI metrics across the Open Source landscape. Only through collective effort and ongoing commitment can Open Source communities continue to evolve and thrive in the spirit of DEI.

Photo by Shane Aldendorff on Unsplash