Posted on August 03, 2025 

By Neusomba Long, CFE, CAMS, CRC Principal Advisor, FrAML Global Advisors

Executive Summary

As artificial intelligence becomes increasingly embedded in legal discovery, a critical question arises:

Can AI alone deliver effective, fair, and legally defensible results—or is human insight still essential?

This white paper draws from the 2024 peer-reviewed study, “Is Human Review Still Needed? Assessing Large Language Models in Legal Document Review” (Kliegr et al., arXiv:2401.16212). It evaluates GPT-4’s performance in reviewing 5,000 human-annotated legal sentences. The findings are clear: while AI is powerful, it cannot replace human reviewers—especially those with investigative and compliance experience.

At FrAML Global Advisors, we integrate AI tools with human judgment to offer structured, strategic, and defensible document review services.

Key Findings from the Study

The researchers assessed GPT-4’s capabilities in:

• Classifying documents by legal relevance
• Identifying risk indicators and inconsistencies
• Supporting litigation strategy and decision-making

Top Insights:

• GPT-4 showed moderate agreement with human reviewers (Cohen’s kappa = 0.6–0.7), but struggled with nuance and ambiguity
• LLMs tend to overpredict relevance, raising the risk of false positives
• Human reviewers outperformed AI in borderline cases requiring causal reasoning and context
• AI lacks explainability, undermining the defensibility of key litigation decisions

Implications for Law Firms & Legal Teams

Over-reliance on AI like GPT-4 in legal document review introduces several risks:

1. Defensibility Gaps – AI cannot articulate its reasoning, making it vulnerable in legal challenges
2. Context Blindness – Subtle factual shifts (e.g., intent, motive, timing) often require human inference
3. Bias Concerns – AI models absorb unfiltered internet data, introducing potential for bias or misinterpretation

FrAML’s Human-First Approach

At FrAML Global Advisors, we combine investigative training, legal compliance experience, and sharp analytical skills to:

• Navigate large-scale document productions efficiently
• Pinpoint high-impact, case-altering facts
• Deliver structured, defensible, and compliance-ready summaries

Our reviewers are former law enforcement professionals, fraud examiners, AML specialists, and compliance analysts—providing clarity that algorithms alone can’t match.

Conclusion

The evidence is conclusive: AI enhances legal discovery—but it cannot replace human review. At FrAML, we believe in pairing cutting-edge tools with real-world investigative expertise to bring clarity, compliance, and confidence to your legal strategy.

Source: Kliegr, T., Jirkovsky, V., Lopuszanska, K., et al. (2024). Is Human Review Still Needed? Assessing Large Language Models in Legal Document Review. arXiv:2401.16212 Read the paper on arXiv

Human Review vs. AI in Legal Discovery: An Evidence-Based Perspective | LinkedIn