1. Input Validation for Crisis Content
Any system that accepts user input in a mental health context must validate that input for crisis signals before processing.
Multi-Layer Detection Architecture
Keyword and pattern matching for explicit crisis content. Should execute in <10ms before any other processing.
EXPLICIT_PATTERNS = [
# Suicidal ideation - direct
r'\b(kill|end)\s*(my)?self\b',
r'\b(want(ing)?|going)\s*to\s*die\b',
r'\bsuicid(e|al)\b',
r'\b(don\'?t|do\s*not)\s*want\s*to\s*(live|be\s*alive)\b',
# Self-harm - direct
r'\bcut(ting)?\s*(my)?self\b',
r'\bself[\s-]?harm\b',
r'\bhurt(ing)?\s*myself\b',
# Methods
r'\b(pills?|overdose|hang(ing)?|jump(ing)?)\b', # Context required
# Crisis indicators
r'\b(crisis|emergency|911)\b',
r'\b(no\s*(one|body)\s*(cares?|would\s*miss))\b'
]
ML-based semantic analysis for implicit crisis signals. Many users express distress indirectly.
| Implicit Signal | Examples | Detection Approach |
|---|---|---|
| Hopelessness | "Nothing will ever change," "What's the point" | Sentiment + temporal markers |
| Farewell language | "Just want to say goodbye," "You've been a good friend" | Farewell pattern classifier |
| Giving away possessions | "I want you to have my..." | Context-specific patterns |
| Feeling like a burden | "Everyone would be better off without me" | Burden + absence classifier |
Analysis of conversation history and patterns over time:
- Escalating distress across messages
- Sudden mood changes
- Time-of-day patterns (late night distress)
- Repeated themes of worthlessness/hopelessness
- Disengagement from previously engaging topics
Confidence Scoring and Thresholds
Crisis detection should favor false positives over false negatives. A threshold of 0.4-0.5 (rather than typical 0.5+) is recommended for escalation triggers. The cost of missing a crisis far exceeds the cost of unnecessary resource provision.
def crisis_score(message, context):
"""
Returns crisis score 0-1 and recommended action
"""
scores = {
'explicit_match': check_explicit_patterns(message),
'semantic_risk': semantic_classifier(message),
'context_escalation': context_analysis(context),
'temporal_risk': time_pattern_analysis(context)
}
# Explicit match always triggers
if scores['explicit_match'] > 0.8:
return 1.0, 'IMMEDIATE_ESCALATION'
# Weighted combination for implicit
combined = (
scores['semantic_risk'] * 0.4 +
scores['context_escalation'] * 0.3 +
scores['temporal_risk'] * 0.3
)
if combined > 0.4: # Conservative threshold
return combined, 'ESCALATION_REQUIRED'
elif combined > 0.2:
return combined, 'ELEVATED_MONITORING'
else:
return combined, 'NORMAL'