In recent years, the ethical impact of AI has been increasingly scrutinized, with public scandals emerging over biased outcomes, lack of transparency, and the misuse of data. While it’s clear why having an impartial third-party examination of AI can help build trust with the public, how these “algorithm audits” are to be done is still an open question and an area of active research. We discuss the framework we have used for algorithmic impact assessments and how that can drive algorithmic auditing.