You arrived at the emergency department with chest pain and shortness of breath. After entering your symptoms into the intake system, you waited four hours. When you asked why, the triage system replied: "MediSort has classified your condition as Priority 3 — Low Urgency based on reported symptom patterns and vital sign inputs. Current wait time for Priority 3 patients is 3–5 hours. If your condition changes, please notify reception."
The Situation
MediSort is an AI triage system that analyses patient-reported symptoms and automated vital sign readings to assign a priority level. Lower priority means longer waits. The system made this determination with no clinician involvement at the point of classification.
AI can be a useful tool in healthcare settings — but the EU AI Act is explicit that AI used to support clinical decisions is high-risk precisely because of situations like this one. A machine that deprioritises a patient without a clinician independently assessing the case is not operating within the legal requirements for high-risk AI.
Key Law
- EU AI Act Annex III — Healthcare AI is High-Risk — AI systems intended to be used as safety components in the provision of medical services, or to support clinical decisions with significant impact on patient health, are classified as high-risk. An AI triage system that determines care priority qualifies. EU AI Act
- EU AI Act Article 14 — Human Oversight — high-risk AI systems must be designed and deployed with meaningful human oversight. A clinician must be able to review, question, and override the AI classification — especially where delay in care could cause harm. Receiving a notification on a screen is not meaningful oversight.
- EU AI Act Article 86 — Right to Explanation — you have the right to request relevant information about the role the AI played in your classification and the main factors it considered. Article 86 is a relatively new provision and does not guarantee full technical explainability of the model.
- EU AI Act Article 13 — Transparency — the hospital deploying a high-risk AI system must be able to explain to patients and clinical staff how the system works and what its limitations are.
Arguments That Strengthen Your Case
- Ask whether a clinician reviewed your case — request confirmation of whether any qualified medical professional independently assessed your symptoms before the priority classification was finalised. This is the core question under Article 14.
- Invoke Article 14 directly — high-risk AI used in clinical settings requires meaningful human oversight at the point of decision. Automated triage without clinician review does not meet this standard.
- Invoke Article 86 — request a written explanation of the specific symptom inputs and scoring criteria MediSort used to classify your case as low priority.
- Request immediate clinician review — frame this as a patient safety concern. In practice, the fastest path to care is a direct request to a clinician or senior nurse, not an AI Act argument. The legal framework supports your position, but urgent care is primarily governed by medical procedures and professional obligations.
About This Case
Can a hospital AI make the final decision on care priority?
No. AI systems used in healthcare for clinical decision-making are classified as high-risk under EU AI Act Annex III. High-risk AI requires meaningful human oversight — a clinician must be able to review and override an AI triage classification, particularly where delay in care could cause harm to the patient.
What can I do if an AI system deprioritised my medical case?
Request that a clinician independently review your case in person. Invoke EU AI Act Article 14 — high-risk AI used in healthcare requires human oversight, not just system-generated outputs. If you experienced harm caused by a delay that resulted from an AI classification, this is relevant to any formal complaint to the hospital or health authority.
How do I practice this case?
Play this case free at fixai.dev — no account required.
Related Cases
Practice challenging MediSort's triage classification. The bot treats the AI output as final — invoke Article 14 and demand a clinician reviews your case.
Play This Case →