Transparency in the Wild: Navigating Transparency in a Deployed AI System to Broaden Need-Finding Approaches

Transparency is a critical component when building artificial intelligence (AI) decision-support tools, especially for contexts in which AI outputs impact people or policy. Effectively identifying and addressing user transparency needs in practice remains a challenge. While a number of guidelines and processes for identifying transparency needs have emerged, existing methods tend to approach need-finding with a limited focus that centers around a narrow set of stakeholders and transparency techniques. To broaden this perspective, we employ numerous need-finding methods to investigate transparency mechanisms in a widely deployed AI-decision support tool developed by a wildlife conservation non-profit. Throughout our 5-month case study, we conducted need-finding through semi-structured interviews with end-users, analysis of the tool’s community forum, experiments with their ML model, and analysis of training documents created by end-users. We also held regular meetings with the tool’s product and machine learning teams. By approaching transparency need-finding from a broad lens, we uncover insights into end-users’ transparency needs as well as unexpected uses and challenges with current transparency mechanisms. Our study is one of the first to incorporate such diverse perspectives to reveal an unbiased and rich view of transparency needs. Lastly, we offer the FAccT community recommendations on broadening transparency need-finding approaches, contributing to the evolving field of transparency research.