A portion of the featured image was generated using AI.
Earlier this year, Humane Intelligence announced our partnership with Radiant Earth to move our bias bounty program onto Zindi, made possible by the generous support from the Heising-Simons Foundation. Zindi hosts a community of more than 100,000 data scientists and AI practitioners across 180+ countries with a user base centered in Africa and growing reach across global emerging markets. We have been working closely with our partners to co-design the first challenge and to build something new: a bias scoring API that automatically evaluates not just how accurate a submission is, but accurate for whom. This means equity analysis is no longer an afterthought. It’s built into how submissions on relevant challenges are judged. We’re excited to share the following updates ahead of the challenge launch around September 2026.
Our first bias bounty on Zindi will ask the question, “Are the communities most vulnerable to climate risk properly represented in open map data?”
Open map data can underpin emergency dispatch, evacuation routing, and disaster relief planning. While open map data is treated as a public good, coverage is often uneven. Our hypothesis for this challenge is that rural, tribal, or lands where socially vulnerable communities live don’t have the same mapping coverage as other areas. Where those gaps overlap with high climate exposure (wildfire corridors, heat-deadly metropolitan areas, drought-prone tribal statistical areas) the consequences can be life threatening.
Participants will measure coverage gaps in Overture Maps across U.S. Census tracts by comparing the open data to authoritative reference sources, including Census TIGER/Line Roads, Microsoft Building Footprints, and HIFLD critical facility datasets. They will then evaluate those gaps against the CDC Social Vulnerability Index, the U.S. Climate Vulnerability Index, tribal land boundaries, and hazard data on heat, wildfire, and drought (defined in this challenge as a prolonged period of abnormally low precipitation that results in water shortages). We focused on these three hazards in part because each is supported by reliable, standardized federal monitoring datasets with consistent national coverage, allowing for meaningful comparison across very different regions of the country.
Focus regions for the challenge include:
Each region was selected for its intersection of climate exposure, infrastructure, and community vulnerability.
A core piece of this challenge, and one that will also benefit future data science challenges on Zindi, is a new bias scoring API.
Standard data science challenges score submissions on accuracy alone, but Humane Intelligence’s Bias Bounties have always asked a different question: accurate for whom? Until now, answering this question has required custom-built scoring rubrics for every challenge. The Humane Intelligence Bias Scoring API turns this aspect of evaluation into platform infrastructure.
Every submission to the coverage gap task will be automatically run through the API, which compares participant results against preloaded demographic and vulnerability data and computes disparity metrics across predefined strata, including:
Participants receive a bias scorecard alongside their accuracy score, showing not only how well their model performs, but where and for whom it performs better or worse. The bias scorecard does not affect leaderboard placement in this challenge, but participants are encouraged to engage with the results as part of their submission narratives.
After this challenge is complete, the API will be available for future Zindi challenges. To date, most existing fairness tooling sits inside a research notebook and requires a data scientist to apply it deliberately. The new bias API will run automatically alongside traditional measures of accuracy, shifting disparity measurement from an optional research step into a key feature of the challenge. This is an important step toward making bias measurement a core aspect of data science challenges and ML evaluations.
The experience of being undermapped, underserved, and over-exposed to climate risk is a global one, and the practitioners best equipped to address it are not always the ones geographically closest to the communities being studied. Far too often, it is data scientists in the high-income countries who draw conclusions about low- and middle-income (LMIC) data, communities, and conditions.
Running this challenge on Zindi inverts that pattern, and we believe the inversion is valuable in both directions: U.S. communities benefit from the analytical perspective of practitioners who have lived with the conditions now arriving here, and scientists from LMICs gain a platform to apply their expertise to a high-profile question outside their own region.
The challenge design is in late-stage scoping with our partners. We will share more details, including timeline, prizes, and how to participate, in the coming weeks. Sign up for Zindi, follow Humane Intelligence on Linkedin, and sign up for the Humane Intelligence newsletter to be the first to hear when registration opens.