Trustworthy AI

Image created with DALL·E by OpenAI

Trustworthiness is a quality requirement for increasingly powerful and prominent AI systems. Trustworthiness depends on a variety of properties such as transparency, fairness, reliability, and robustness, however, many of which remain poorly understood and difficult to implement. For example, it remains unclear to what extent transparency can actually be achieved in high dimensional nonlinear systems, what fairness actually consists in, and how we can assess the reliability of AI-generated content. To a large extent, these are philosophical questions that require close analysis of relevant concepts, as well as a realistic assessment of current technical capabilities against normative and societal constraints.

Transparency: ECPAI members Carlos Zednik, Philippe Verreault-Julien, Céline Budding, and Yeji Streppel are engaged in several projects to specify norms and standards of explainability and transparency. To this end, they actively collaborate with AI researchers and industry representatives to develop, implement, and evaluate methods in explainable AI in domains such as medical decision-making, automated driving, and language-production. They also participate in national and international efforts to promote explainability and transparency through regulatory means such as standardization.

Reliability: Elizabeth O’Neill aims to uncover and characterize the criteria that can be used to judge whether or not an AI system’s recommendations should be considered reliable. In particular, her NWO-funded research projects consider the reliability of AI systems capable of moral reasoning.

Fairness & Trust: ECPAI members Philip Nickel and Patrik Hummel investigate the conditions for trustworthiness in data-driven decision-making, with a particular focus on medical contexts. To what extent can and should AI systems be trusted to make high-stakes medical decisions? What role does fairness play in the medical decision-making and access to state-of-the-art AI?

Funded Projects


Associated Researchers

  • Céline Budding
  • Elizabeth O’Neill
  • Philip J. Nickel
  • Yeji Streppel
  • Philippe Verreault-Julien
  • Carlos Zednik