ATI Evaluation — Decision Brief · Toronto Fire Services

The dashboard

The tool, whichever path is chosen

A working demonstration on entirely fabricated data — no real instructors, evaluators, or evaluation records. Tap a day for who/when/where, browse the ATI and evaluator records, and see coverage by discipline.

ATI Evaluation — HomeSynthetic demo data

1Are we on track?

—of ATIs evaluated

On track

—

evaluated

—

disciplines covered

2What’s the schedule? — tap a day for detail

Covered Needs an evaluator Completed Operations Technical

Tap a day above to see its sessions.

3Browse the data

Click an ATI’s name to see their evaluation history.

Discipline	Coverage	Evaluated

⚠ All data above is fabricated for demonstration — read-only, no real personnel.

Standardizing how we evaluate our 692 instructors.

What: Toronto Fire Services delivers training through 692 Acting Training Instructors across 26 disciplines, with no standardized, defensible way to evaluate their instruction against NFPA 1041. Why now: NFPA 1041 requires documented, continuous evaluation of instructor performance, and recruit surveys repeatedly flag inconsistent training as a result — today nothing measures it. The proposal: a shared dashboard that schedules who evaluates whom and when, tracks coverage by discipline, and flags results below standard — built once, and run under whichever of the three delivery models below is chosen.

Three paths forward

	Path ADedicated ATI Evaluation DisciplineRecommended	Path BATI Leads evaluate across disciplines	Path CManagement evaluates
Cost per hour	● Comparable	● Comparable	● Comparable
Staff to manage	● More	● None added	● None added
Credentialing	● Guaranteed	● Not guaranteed	● Not guaranteed
Skill development	● Strong	● Moderate	● Weak
Data accuracy	● Strong	● Variable	● Weak
Bias	● Low	● Higher	● Mixed

The trade-off: Cost per hour is comparable across all three paths — the real trade is staffing overhead against consistency. Path A scores highest on accuracy, bias, skill and credentialing, at the cost of a new cadre of evaluators to manage. Paths B and C avoid that overhead but trade away consistency, bias control, and guaranteed qualification. The dashboard behind this brief works the same under any of the three. (Hover a rating for detail.)