This article discusses the concept of a black box and black box problem in relation to clinical AI. It explains how a black box clinical AI may produce a black box problem, and how explainability is important to realise design publicity or fulfil an essential duty of disclosure of material information in health professional-patient relationships. The article also addresses potential criticisms of the black box problem, such as when conditions like the patient’s capacity to choose from among clinical AI predictions are not available, or when black-boxed clinical AI are proprietary. It argues that regulations should be in place to foster relations between the industry and key stakeholders in the healthcare system.
