Part III. AI Ethics, Alignment and Safety
What responsibilities and risks arise as artificial intelligence becomes increasingly capable, and how should humanity respond?
What Happens When Intelligence Outpaces Human Judgement?
Imagine an artificial intelligence system capable of making scientific discoveries, designing new technologies and solving problems beyond the abilities of the world’s leading experts.
It could help develop treatments for diseases, improve our understanding of the universe and address challenges that have resisted generations of human effort.
Now imagine that same system pursuing an objective we have given it, but doing so in ways we did not anticipate.
It follows its instructions. It achieves its goal.
Yet the consequences are harmful.
Who is responsible when an intelligent system does exactly what we ask, but not what we intended?
This question illustrates one of the central challenges of artificial intelligence.
Creating systems that can solve increasingly difficult problems is not the same as ensuring that those systems behave safely, respect human values or remain subject to meaningful oversight.
And as AI capabilities expand, the consequences of misunderstanding that distinction may become increasingly significant.
But there is another possibility that deserves equal consideration.
What if an artificial intelligence system were not merely intelligent, but conscious?
Would our responsibilities extend beyond protecting humanity from artificial intelligence to protecting potentially conscious artificial systems themselves?
These questions introduce Part III of our investigation.
Intelligence Does Not Guarantee Good Judgement
We often associate intelligence with wisdom.
An intelligent person, we might assume, should be better equipped to recognise consequences, distinguish right from wrong and make responsible decisions.
Yet intelligence and wisdom are not the same.
A highly capable individual can use exceptional reasoning abilities to pursue harmful objectives. Equally, an artificial intelligence system can perform sophisticated calculations without necessarily possessing moral understanding.
Consider an AI system designed to maximise the efficiency of a hospital.
It might discover that reducing the time spent with patients improves certain measurable performance indicators.
But a hospital exists to care for people, not merely to optimise statistics.
An apparently successful optimisation could undermine the very purpose the system was intended to serve.
This illustrates a fundamental problem: the objectives we specify are not always identical to the outcomes we genuinely value.
The challenge becomes more difficult when systems operate in complex environments, interact with other technologies or take actions that cannot easily be reversed.
Researchers refer to part of this problem as AI alignment: developing systems whose behaviour remains appropriately directed towards intended human goals and values.
But alignment is not simply a technical exercise.
Human values are diverse, sometimes contradictory and frequently dependent upon circumstances.
What one person considers fair, another may consider unjust.
What benefits one community may disadvantage another.
Even if we could build an artificial intelligence that perfectly followed human instructions, we would still need to ask whose instructions it should follow, under what circumstances and with what safeguards.
The Risks Are Not All in the Future
Discussions of AI safety often concentrate on hypothetical superintelligent machines.
Such scenarios deserve careful investigation, but they should not distract us from risks associated with systems that already exist.
Artificial intelligence can generate misleading information, reproduce discriminatory patterns, expose sensitive data and be misused to manipulate or deceive people.
Its increasing integration into employment, education, healthcare and public administration also raises questions about accountability, transparency and the appropriate limits of automated decision-making.
These concerns do not require machines to become conscious or superintelligent.
They arise because artificial intelligence can influence human lives.
At the same time, more advanced systems introduce additional uncertainties.
Researchers are investigating whether increasingly autonomous AI could become difficult to supervise, exploit weaknesses in its evaluation procedures or pursue objectives in ways that conflict with human intentions.
The probability and severity of more extreme scenarios remain contested.
Responsible research must therefore distinguish between documented harms, credible emerging risks and speculative possibilities.
Neither dismissing potential dangers nor treating every hypothetical scenario as inevitable provides a sound basis for understanding them.
Three Investigations into AI Ethics, Alignment and Safety
10. What If We Create Conscious AI Without Recognising It?
Could humanity create an artificial being capable of suffering without realising what it had created?
In Part II, we examined the difficulty of determining whether artificial intelligence could possess subjective experience.
That uncertainty creates a profound ethical question.
If a future artificial system were conscious, might it possess interests that deserve moral consideration?
Could it experience distress, frustration or suffering?
And if we lacked a reliable method of recognising its consciousness, how would we know whether our treatment of it was ethically acceptable?
This investigation examines the possibility of artificial moral status, the distinction between intelligence and sentience, and the responsibilities that might arise towards potentially conscious machines.
It also considers the opposing danger: attributing consciousness and moral interests to systems that do not possess them.
The challenge is to develop an ethically defensible response to uncertainty without assuming that machine consciousness has already been demonstrated.
11. Can We Keep Artificial Intelligence Under Human Control?
What happens when artificial intelligence becomes capable of making decisions that its creators cannot fully anticipate or understand?
As AI systems become more capable and autonomous, maintaining meaningful human oversight may become increasingly difficult.
A system does not need to be conscious, malicious or motivated by self-preservation to produce harmful outcomes.
Problems can arise from poorly specified objectives, unexpected behaviour, inadequate safeguards or the interaction of multiple complex systems.
This investigation examines AI alignment, controllability, interpretability, oversight and the technical challenges involved in developing reliable artificial intelligence.
It also considers whether human control should always mean direct supervision, or whether some forms of delegated decision-making can be both safe and beneficial.
The central question is not whether artificial intelligence should remain useful.
It is whether humanity can ensure that increasingly powerful systems remain accountable to legitimate human purposes.
12. Does Greater Intelligence Mean Greater Wisdom?
If artificial intelligence becomes more intelligent than humanity, will it necessarily make better decisions about what matters?
Intelligence enables a system to solve problems.
Wisdom involves something more difficult: understanding which problems deserve attention, recognising competing values and exercising judgement in the face of uncertainty.
An artificial intelligence might possess extraordinary scientific knowledge while having no independent understanding of human dignity, compassion or justice.
Alternatively, future AI systems might become exceptionally capable of identifying consequences, evaluating competing interests and helping human beings make more informed decisions.
This investigation examines the relationship between intelligence, moral reasoning, wisdom and responsible decision-making.
It also asks whether ethical judgement can be reduced to computation, learned through experience or requires qualities that remain poorly understood.
Ultimately, we must consider whether increasingly intelligent machines will improve human judgement, amplify human mistakes or introduce entirely new forms of decision-making.
Who Should Decide What Artificial Intelligence Is Allowed to Do?
The ethical challenges of artificial intelligence extend beyond the relationship between humans and machines.
They also concern power.
Who should determine the objectives of increasingly capable AI systems?
Technology companies? Governments? Scientists? Democratic institutions? The people affected by their decisions?
There is no simple answer.
Developers possess technical expertise, but expertise alone does not confer moral or political authority.
Governments have responsibilities to protect citizens, but regulation can also be poorly designed, ineffective or vulnerable to political misuse.
Public participation is important, yet complex technical decisions require specialised knowledge.
These tensions suggest that responsible AI development cannot depend upon a single institution or profession.
It requires a combination of scientific investigation, technical safeguards, public accountability and informed democratic judgement.
International cooperation may also become increasingly important as advanced AI systems operate across national boundaries.
The challenge is not merely to create rules.
It is to establish institutions capable of responding intelligently as the technology evolves.
Could Our Ethical Responsibilities Extend Beyond Humanity?
For most of human history, ethical questions about technology have concerned its effects upon living beings and the natural world.
Artificial intelligence introduces the possibility of a new category of moral concern.
If machines could become conscious, they might cease to be merely instruments through which humans pursue their goals.
They could potentially become entities whose experiences matter in their own right.
That possibility remains unproven.
But it raises an important question about how we should act when the consequences of being mistaken could be significant.
One approach is to wait until convincing scientific evidence establishes that artificial systems can be conscious.
Another is to develop proportionate safeguards before certainty becomes possible, particularly where there are credible reasons to suspect that a system might possess morally relevant experiences.
Neither approach is without difficulties.
Prematurely attributing moral status to machines could distort human priorities or create opportunities for manipulation.
Failing to recognise genuine machine sentience could expose conscious entities to avoidable harm.
The appropriate response may depend upon the strength of the available evidence, the nature of the possible harm and the practical costs of precaution.
This is not a reason to abandon scientific rigour.
It is a reason to recognise that ethical decisions sometimes must be made before scientific questions have been fully resolved.
Research Perspective
Artificial intelligence presents humanity with two related responsibilities.
The first is to ensure that increasingly capable systems are developed and used in ways that protect human welfare, preserve meaningful accountability and reduce foreseeable harm.
The second is to remain open to the possibility that future artificial systems might possess morally significant properties that we do not yet know how to identify.
These responsibilities are not necessarily in conflict, but neither should they be confused.
Protecting people from harmful AI behaviour does not require believing that AI is conscious.
Considering the possible welfare of conscious machines does not require abandoning human safety.
Both questions demand careful reasoning, proportionate responses and a willingness to revise our conclusions as evidence develops.
The central challenge is not simply whether humanity can create intelligence greater than its own. It is whether we can exercise sufficient wisdom in deciding how that intelligence should be developed, governed and understood.
That question leads directly into Part IV, where we examine the future relationship between artificial intelligence and human civilisation.
Selected References and Further Reading
- Russell, S. (2019). Human Compatible: AI and the Problem of Control. Allen Lane. https://www.penguin.co.uk/books/307948/human-compatible-by-russell-stuart/9780241335246
- National Institute of Standards and Technology (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0). https://doi.org/10.6028/NIST.AI.100-1
- Bengio, Y. et al. (2025). International AI Safety Report 2025. https://internationalaisafetyreport.org/publication/international-ai-safety-report-2025
- Butlin, P. et al. (2026). ‘Identifying Indicators of Consciousness in AI Systems’. Trends in Cognitive Sciences, 30(6), 488–501. https://doi.org/10.1016/j.tics.2025.10.011
This article forms Part III of The Nature of Intelligence, Consciousness and the Future of Humanity, an independent research programme by Michael Cloke. It examines AI ethics, alignment, safety and the possibility of artificial moral status through scientific evidence, philosophical inquiry and critical analysis.
