V1, the form trap

When we shipped V1 we thought of check-ins as a data collection feature. We modeled it on a Typeform: 12 fields, validation, a "save" button. The launch numbers were embarrassing.

Trainee completion rate at week 1: 87%. Week 2: 64%. Week 6: 31%. By the time we had data the feature had taught coaches that check-ins were a data-collection chore. The fix was not adding fields. It was deleting them.

V2, the un-replied check-in

V2 dropped to 3 questions and shipped on a Sunday-evening schedule. Trainee completion rates jumped: week 1 at 91%, week 6 at 72%. We declared victory. We were wrong.

The number we did not measure was reply rate from coaches. 38% of coach replies came within 24 hours, 22% within 12 hours, 40% never. The 40% never group accounted for almost all the trainee churn at month 3.

The check-in feature is not a data feature. It is a conversation feature. V2 built the question half of the conversation and left the reply half as an honor system. Honor systems do not scale.

V3, the reply loop

V3 ships the 3 questions plus a structured reply UX: when a coach opens a check-in to reply, the UI prompts a 4-part template (acknowledge, reflect, change, question). A small "reply by" countdown reminds the coach of the 12-hour window. Coaches who reply within the window get a small achievement badge.

Within 60 days of V3 ship: coach reply rate at 12 hours moved from 22% to 71%. The check-in feature finally did the thing it was supposed to do.

What I learned about building for behavior change

Build for the behavior, not the feature. The first two versions of check-in were features. The third was the behavior. You cannot ship the behavior in V1, because you do not know what it is yet. You ship V1, watch what people do, and rebuild for the actual behavior you observe.

Most coaches I have spoken to about this nod, because the same pattern is true of programs. The first program you write for a client is not the right program. It is the program you needed to write to find out what the right program is.