Substantive quality
Correction of seeded errors, preservation of correct propositions, identification of uncertainty, and appropriate qualifications or escalation.
The professional judgment study
Project Rainmaker
Research into human judgment, professional responsibility and AI-assisted transactional legal work.
Research platform under construction
- Planned participants
- Approximately 50 experienced Canadian transactional lawyers
- Study tasks
- Two fictional Ontario private-company acquisition matters
- Comparison
- Staged review policy and ordinary professional discretion, both with AI
The broader inquiry
What makes human oversight defensible?
Project Rainmaker examines professional judgment alongside the practical review of AI-assisted work.
The broader questions concern professional responsibility: which checks should a lawyer make before relying on AI-assisted work, when should uncertainty prompt escalation, and what supervision can support a defensible final decision?
The proposed empirical comparison can inform these questions by examining what reviewers do and the work they produce. It cannot, by itself, establish a professional standard or determine legal responsibility.
The question
How should people review AI-assisted legal work?
Does a staged, risk-sensitive AI review policy improve the quality of transactional legal work and change the human effort required to produce it, compared with ordinary professional discretion?
The current proposed design compares review approaches within an AI-assisted workflow. Both groups use AI. The question is how the human review process changes the resulting work and the effort it requires.
Two review approaches. A common starting point.
- Match and assign. Approximately 50 experienced Canadian transactional lawyers will be matched on relevant transaction experience and AI-use frequency, then randomly assigned to two groups.
- Complete two matters. Each participant will work on two comparable fictional Ontario private-company acquisition matters.
- Keep the inputs consistent. Both groups will receive identical AI-assisted starting drafts, source materials, tools, and a fixed AI configuration.
- Compare the review policies. One group will follow a staged policy; the other will review using ordinary professional discretion.
What the study will measure
Quality and human effort, considered separately.
Better work and faster work are distinct outcomes. The study will assess each without collapsing them into a single “quality per minute” score.
Human effort
Active time spent producing the final work.
Professional judgment needs professional review.
Blinded peer assessment is planned for professional judgment, prioritization, usefulness, handoff quality, and client communication.
The streamlined proposal envisages four peer reviews per participant, providing two ratings for each completed artifact.
A supporting role for AI.
AI may assist structured measurement. Assessment of professional judgment will remain human-led.
Keeping quality and time separate will make the tradeoffs visible: improved work may require more effort, while faster work may still contain substantive problems.
The platform’s intended role
A controlled environment for the experiment.
Juridical AI’s MVP is intended to provide the research environment for Project Rainmaker. The participant experience below is planned and under development.
Read the planned participant workflow and safeguards
Consent and orientation
Assigned review workflow
Two simulated matters
Locked submissions
Follow-up judgments and blinded peer review
Consistent conditions, observable work.
The future environment should support consistent materials, condition-specific instructions, work records, time measurement, submission locking, and blinded review.
Work records will focus on observable actions and explicit decision checkpoints: authorities consulted, verification steps, revisions, and reviewer feedback.
Protecting the production phase.
Confidential case keys must remain inaccessible until participants’ own production submissions are locked.
Current status
Research platform under construction
This page will become the entry point for study participants. The research environment is currently under construction. Participant access and study instructions will be added when the study is ready to launch.
The protocol and timing remain subject to refinement and pilot testing. Any live study launch will depend on the required research approvals.
Express interest in participating
Prepare an email to request a place on the lawyer interest list or discuss participation through a law firm. You can review and send the draft in your email app.
Sending an interest request is not enrollment or research consent. Participation arrangements and study instructions will be provided when the study is ready.