Ethical use of DSS

What are human rights, equality & diversity?

Human rights set out the basic rights and freedoms that belong to all human beings, regardless of race, sex, nationality, ethnicity, language, religion, or any other status.  These are documented within the Human Rights Act 1998. Human rights include the right to life and liberty, freedom from slavery and torture, freedom of opinion and expression, the right to work and education, and many more.

All public bodies, including local governments and health services, must do all they can to protect people's human rights. They must ensure those rights are not infringed upon or interfered with, without justifiable cause.

The Equality Act published in 2010 legally protects people from unfair treatment because of any of the following ‘protected characteristics:’  Age, Disability, Gender reassignment, Marriage and civil partnership, Pregnancy and maternity, Race, Religion or belief, Sex, Sexual orientation.

The Equality Act states that it is unlawful to treat an individual less favourably, or unfavourably, because of their protected characteristic. Examples could include:

• Being excluded from something
• Refusal of service
• Being deprived of a choice
• If you are at a disadvantage compared to other service users
• Poor quality of service compared to others.

Decision support can impact on human rights and equality and diversity in several ways. For example:

• People with no access to technology or with low digital and health literacy will need extra support to get the benefit of digital decision support tools.
• The original research or datasets on which a decision support tool is based are not representative of all groups with protected characteristics. This means that the decision support tool may make recommendations that are not suitable for these groups. 

How do I provide evidence of competency in this area? 

Can you...

• Display working knowledge of what is meant by a Human Rights-based approach and of the core values of Fairness, Respect, Equality, Dignity and Autonomy for all? 

• Show appreciation of how these core values, particularly equality, respect and autonomy, need to be considered in design and implementation of decision support systems? 

• Show appreciation of the potential risks to human rights which can arise from algorithmic decision support - both knowledge-based and computer-generated. Provide examples of potential risks and discuss how these can be mitigated? 

• Display understanding of potential sources of risk of discrimination against groups with protected characteristics through the way decision support algorithms are designed, developed and implemented? 

• Show working knowledge of possible approaches to mitigate these risks? 

• Provide examples of decision support solutions, within healthcare or beyond, where risks to equality have been identified and discuss how these risks have been/ could be mitigated? 

What is the risk of bias in decision support systems?

Bias can enter decision support systems, particularly algorithmic decision support sytems,  in a number of ways, including:

Historical bias: The data that the model is built, tested and operated on could introduce bias. This
may be because of previously biased human decision-making or due to societal or historical inequalities.
For example, if a workforce dataset is predominantly white males, then an algorithm may reinforce this. 

Data selection bias: How the data is collected and selected could mean it is not representative. For
example, over or under recording of particular groups could mean the algorithm was less accurate for some
people, or gave a skewed picture of particular groups. 

Algorithmic design bias: Algorithmic bias can also be caused by programming errors, such as a developer unfairly weighting factors in algorithm decision-making based on their own conscious or unconscious biases. For example, indicators like income or vocabulary might be used by the algorithm to unintentionally discriminate against people of a certain race or gender.

Cognitive bias: When people process information and make judgments, we are inevitably influenced by our experiences and our preferences. As a result, people may build these biases into AI systems through the selection of data or how the data is weighted. For example, cognitive bias could lead to favoring datasets gathered from white Caucasian populations rather than sampling from a range of populations. Cognitive bias can also influence the way that health and care professionals interpret the results delivered by decision support systems.

Reducing the risk of bias

There are best practices that healthcare data scientists and developers can incorporate to address the risk of bias in decision support systems. These include:

• Have a more diverse body of people review the algorithms and supervise machine learning approaches.
• Use methods or techniques to best manage situations where there is not enough information available, like using synthetic data.
• Work with diverse communities to ensure the algorithms are helpful and don't cause harm.
• Introduce the algorithms gradually and carefully instead of all at once.
• Create ways for people to provide feedback and improve the algorithms over time.
• Involve diverse members of the workforce in developing the algorithms and validating patient data from various racial and ethnic backgrounds.

Nazer, L.H. et al. Bias in artificial intelligence algorithms and recommendations for mitigation. PLOS Digital health. June 22, 2023 https://doi.org/10.1371/journal.pdig.0000278

How do I provide evidence of competency in this area? 

Can you...

• Show working knowledge of sources of bias that go beyond the standard Protected Characteristics groups (e.g. risk prediction for dementia may be based on data from patients who attend a memory clinic - so already identified as a high risk group compared with the general population)? 

• Display working knowledge of possible approaches to mitigate these risks? 

• Provide examples of decision support solutions, within healthcare or beyond, where risks of bias have been identified and discuss how these risks have been/ could be mitigated? 

What is 'Impact on human judgement – professional, social and cultural context of decision support systems'?

Decision support systems can impact on human judgement and professional and social context in a number of ways. For example:

Risk of automation bias. This is where clinicians may trust the outputs or recommendations of AI systems not due to evidence of efficacy, but rather because automated decision support systems are perceived as being objective, accurate or managing complexity.

Over-reliance on decision support systems can potentially result in "de-skilling" - i.e. inhibiting the development of skills, professional communities, and norms of ‘good practice’ developed through the wisdom that comes with practical experience. 

The impact of decision support systems on person-centred care and the relationship between professional and patient/client is likely to depend on whether it assists or replaces human practitioners. Assistive decision support may have a positive role to play in practitioner-patient/client relationships by providing a focus for improving openness and communication in shared decision-making.  On the other hand, the risk of decision support tools replacing human practitioners can have a negative impact as the relationship between practitioner and patient/client is key to the therapeutic process and working out together the best approach to support the person's needs. 

Decision support literacy and emotional intelligence are key skills to mitigating the potential risks that decision support tools present to professional judgement.

In terms of social impact, decision support tools can provide robust and trusted support to health and care practitioners battling complex workloads and increasing demands. They can minimise errors resulting from fatigue and challenges to concentration. They can be leveraged to reduce operational complexity and improve efficiency. For patients and clients, decision support tools can aid self-management and self-care, and can support them with information to become proactive participants in decisions about their care. r

How do I provide evidence of competency in this area? 

Can you...

• Provide examples of applying these principles in their own practice? 

• Analyse, evaluate and create training and other supporting material in this critical area? 

What is 'social good' in relation to decision support?

Social good or common good describes an action that benefits the general public, or at least a significant number of people in a community. The United Nations Sustainable Development Goals provide an overview of key elements of social good. They include ending poverty and hunger, improving health and education, addressing climate change, and creating economic opportunity. Decision support systems can contribute to all these outcomes. 

Five key ethical questions to ask about a decision support tool that will help to ensure positive impact on social good are:

• Will the proposed use of the tool benefit the general public, or at least a significant number of people in the community?
• Will it avoid harm?
• Does it consider people fairly and impartially?
• Have the people affected by the tool given informed consent to its use?
• Can you explain how the decision support tool works - and who is accountable for its use?

How do I provide evidence of competency in this area? 

Can you...

• Articulate the benefits which decision support can bring at multiple levels: a) to individual patients, service users and colleagues; b) the wider community; c) to health and care systems (e.g. safer, higher quality care; more efficient delivery of care, optimising use of available resource; empowering citizens to be active participants in choices about health and wellbeing; improving public health)? 

• Provide examples of these benefits in the health and care context?