Doctors, lawyers, investigators, notaries, and tax collectors: they all increasingly rely on AI to help them in their decisions. Take a doctor who seeks help with diagnosis and a treatment plan based on his ‘decision support system’. Ideally, the computer should act as a ‘second opinion’, but before you know it, it takes the lead. Haselager: “Ultimately, it is the doctor who makes the decisions, but due to time constraints, the system becomes the guide. This can reinforce tunnel vision, and lead to erroneous decisions. We tend to lean on AI because we can be cognitively lazy creatures, and also because it is not so easy to go against AI.”
Is that a bad thing? Are we drowning in poor decisions as a result of our current use of decision support systems? Yes, it can be a very bad thing, says Haselager on the basis of previous research. In one study, the researchers deliberately added distorted information to an AI tool, thus introducing irrelevant divisions between population groups in the decisions. Group A was always the dupe, but not a single user noticed it. “We say that these systems are self-learning, and we have great confidence in them, but nobody actually knows exactly how this kind of system works. As a doctor or investigator, you don't actually know exactly how the system does what it does, and you don't see clearly the reasons why the system makes decisions. As a result, in some cases, you might not see that it leads to systematic distortions in your decisions.”
Reflection machine
Those calling on users of these systems to remain alert can do two things. The first is build in more feedback in the working environment, more peer consultation, and more space for the thorn in the side at the team meeting. “That's great, and it does happen, but it takes a lot of manpower,” says Haselager, who therefore advocates for plan B: the technological fix. “I am often told that my idea only contributes technologically, and yes, it is true that my work lies in AI, but I believe that a complementary AI, one that does not replace humans but complements and invites human thought, may well help.”
This complementary form of AI is the ‘reflection machine’ devised by Haselager, and which he describes as a Socratic conversation with your computer. What is crucial in this feedback loop is the precise formulation of the questions the AI asks the user, how the AI formulates these questions, and their timing. “There's a lot of psychology involved in that. That is why it is such a good thing that our AI is embedded in the social sciences faculty; at other universities, AI tends to be embedded within the natural sciences.”
A good ‘reflection machine’ will be perceived by users as a ‘counter-thinking buddy’, ‘a friend who provokes you to think in a good conversation’. Researchers are currently investigating whether this system works, whether it really leads to better decisions, and to what extent it is accepted by users. Haseleger describes the success of ‘his’ machine as ‘very exciting’. The idea is gaining further significance because it can contribute to the now prevailing debate on responsibilities. After all, it’s really hard to shift the blame for a wrong decision ‘taken’ by a computer to the computer, or to the creators of the software. “Ultimately, it is us humans who bear the responsibility, and the reflection machine is helping us become more aware of this fact."
Image source: Drew Dizzy Graham via Unsplash