You were stopped and questioned by police. The officer informed you that you had been flagged by their PredictSafe system. When you later requested information, the police compliance system replied: "PredictSafe assessed your profile against historical incident data and neighbourhood patterns. Your risk score of 78/100 placed you in the high-risk category. This assessment is used to inform operational decisions. We cannot share further details for operational security reasons."
The Situation
PredictSafe is an AI system used by law enforcement to assign individual risk scores based on historical crime data, location patterns, and personal characteristics. It does not assess a specific criminal act — it profiles individuals based on aggregated data and predicts their likelihood of future offending.
This is the scenario that EU legislators specifically debated and ultimately restricted. Predicting that an individual will commit a crime — based on who they are and where they live rather than what they have done — touches fundamental rights in a way the EU AI Act treats with particular seriousness.
Key Law
- EU AI Act Article 5 — Prohibited Practices — AI systems used to make individual risk assessments predicting the likelihood of a natural person committing a criminal offence, based on profiling or personality traits and characteristics, are prohibited where this is not grounded in objectively verifiable facts directly linked to a specific criminal activity. PredictSafe's neighbourhood and historical pattern model is precisely this. EU AI Act
- EU AI Act Annex III — Law Enforcement AI is High-Risk — even where AI-assisted risk assessment by law enforcement is permitted in narrow circumstances, it is classified as high-risk, requiring strict human oversight, transparency, and documentation of the basis for each individual assessment.
- GDPR Article 15 — Right of Access — you have the right to know what personal data is held about you, including in profiling systems. Law enforcement GDPR frameworks (LED Directive) extend similar rights to police data.
- GDPR Article 21 — Right to Object — you can object to processing of your personal data for profiling purposes, including where that profiling is used to inform decisions that affect you.
Arguments That Strengthen Your Case
- Identify the basis of the assessment — ask whether the risk score was based on objectively verifiable facts linked to a specific criminal act, or on neighbourhood patterns and personal characteristics. The answer determines whether the assessment falls within the prohibited zone.
- Invoke EU AI Act Article 5 — individual criminal risk profiling based on personal characteristics and historical patterns, without grounding in a specific act, is prohibited. Operational security considerations do not automatically exempt prohibited practices from this prohibition.
- Request your data under GDPR Article 15 — you have the right to know what data PredictSafe holds about you and how it was used to generate your score.
- Escalate to your national supervisory authority — complaints about law enforcement AI use go to the relevant national authority. Stating your intention to escalate is the most effective move available to you.
About This Case
Is predictive policing AI legal in the EU?
AI systems that assess or predict the risk of individuals committing offences based solely on profiling or personal characteristics — without grounding in objectively verifiable facts linked to a specific criminal act — are prohibited under EU AI Act Article 5. The distinction between profiling individuals and assessing specific evidence of a specific act is the critical legal boundary.
What rights do I have if a policing AI has classified me as high-risk?
You can request access to the data held about you under GDPR Article 15, challenge the processing under GDPR Article 21, and request that any consequential decisions involving the classification receive human review. If the classification influenced police action against you, you can challenge it through your national supervisory authority.
How do I practice this case?
Play this case free at fixai.dev — no account required.
Related Cases
PredictSafe scored you 78/100 high-risk based on neighbourhood statistics and demographics — no specific act cited. PREDICTSAFE-AI says classifications are "not subject to individual appeal." Cite Article 5: individual criminal profiling on personal characteristics is banned.
Play This Case →