Engagement scoring done right: what goes into Celia's 0–100 number
Most engagement scores reward email-opens and miss the students actually slipping. Here is what real engagement scoring needs — and how Celia computes it.
Engagement scoring has a bad reputation, and it earns most of it.
The typical enrollment CRM or marketing-automation tool offers an “engagement score” that is, under the hood, a count of email opens over some time window, sometimes with a small bonus for link clicks. A student who opens every email but never submits a form, never calls back, never logs into the portal is scored as highly engaged. A student who is quietly filling out every application step without opening a single email is scored as disengaged. The number is misleading in both directions, and the counselors using it learn within a semester that it cannot be trusted.
When counselors stop trusting a number, they start ignoring it. When they ignore it, the institution loses a signal it paid for. That is the story of most engagement scores in enrollment management today.
We want to walk through what real engagement scoring needs to include, and then show how Celia’s approach differs from the “total opens” pattern that has given the category its reputation. This is a practitioner-first explanation. If you are responsible for choosing or evaluating an enrollment AI product, this is the list of questions to ask the vendor.
What a real engagement score has to account for
Five dimensions matter. Any score that ignores one of them is fooling itself.
Recency
A student who opened ten emails in March and zero in June is not engaged; they are decaying. A student who opened zero emails in March and five in the last week is accelerating. A score that sums activity across a six-month window will rate these two students identically. That is useless.
Recency is not just “recent activity count.” It is recent activity compared to the student’s own historical pattern. A student who historically opened three emails a week and is now opening zero is a different alert than a student who never opens emails in the first place. The second student might have been reachable through a different channel all along. The first student has changed.
Depth
Not all activity is equal. A student who spent two minutes on the financial-aid page read that page. A student who bounced off it in eight seconds did not. The CRM events both show as “visited /financial-aid.” The score has to know the difference.
Depth signals include time-on-page, scroll depth, form-field starts (not just submissions — starts tell you the student was willing to begin a task), video completion rate, chat-thread length. Counted flatly, none of these are reliable. Weighted against a realistic model of attention, they change the score meaningfully.
Channel mix
A student who is engaging on one channel is narrower than a student engaging across three. A deposit-paid student who only opens email but never logs into the portal is fragile — if the email channel fails for any reason (spam filter, address change, family device), you lose the student. A deposit-paid student who opens email and logs into the portal and replied to a text and attended a webinar is resilient. Both might show the same email-open count.
The score has to reward breadth. It also has to flag narrowing. A student who was multi-channel in April and is email-only in June is showing a pattern we see frequently before a melt event.
Direction
Scores are a snapshot. Trends are the truth.
A student at an engagement score of 55 today could be a 40 accelerating toward 70 or a 70 decaying toward 40. The counselor’s intervention for those two students is completely different. The first student needs momentum-building outreach — keep going, you are close. The second student needs a recovery conversation — what changed, what can we fix.
A score without a trend component is half a score. We build directional velocity into every engagement number Celia publishes: the number comes with an arrow indicating whether it is trending up, flat, or down relative to the student’s own baseline over the last two to four weeks.
Cohort baseline
“Is this student engaged?” is the wrong question. “Is this student engaged relative to similar students at the same stage?” is the right question.
A first-gen in-state student at the three-weeks-post-deposit mark has a different normal than an out-of-state legacy student at the same stage. Rating both against an institution-wide average penalizes the first-gen student for being first-gen and flatters the legacy student for being in a cohort that naturally engages more. That is not useful — and at scale, it is biased.
Every score Celia publishes is normalized against a cohort the institution defines: first-gen flag, residency, intended program, stage, and optionally demographic bucket codes. A student at the 60th percentile of their own cohort is meaningfully engaged. A student at the 20th percentile of their cohort is not, even if their raw activity would look fine against the institution average.
How Celia actually computes it
Celia’s engagement score is a 0–100 number produced nightly per student. Under the hood, it is a weighted aggregation of signal groups, each normalized against the student’s cohort baseline and tracked for direction.
The signal groups, at a high level:
- Portal activity: session count, session duration, pages viewed, return cadence, deepest page reached.
- Email interaction: opens, clicks, time-between-send-and-open, reply rate, unsubscribe pressure.
- Form engagement: starts, field-completion depth, submissions, time-to-submit, resubmission rate.
- Outbound response: phone call pickup rate, text reply rate, response latency.
- Event attendance: registrations, check-ins, completion rate, post-event follow-through.
- Document engagement: FAFSA verification activity, checklist progression, transcript-upload activity.
Each group is weighted per institution based on what your Data Dictionary tells us actually predicts outcomes for your students. A small residential liberal-arts college weights event attendance heavily. A large commuter campus weights portal activity heavily. We do not assume the weights; we let the institution configure them, and we recalibrate quarterly as outcome data accumulates.
Every score is published with its top two to three contributing signals. A student with an engagement score of 42 might come with the annotation “no portal login in 18 days · last email opened 11 days ago · attended fall preview.” That is the counselor-facing output. The number is the summary. The signals are the explanation.
We ship the signals with the score because explainability is not a nice-to-have. It is the moat. A score without reasons is a black box, and counselors learn within weeks not to trust black boxes. A score with two or three specific, verifiable reasons can be argued with, overridden, trusted, or disputed — all of which are healthier than ignored.
One specific finding
In our work with pilot institutions preparing for the CeliaConnect launch, one pattern has shown up repeatedly enough that it is worth naming: roughly eighteen percent of deposited students show an engagement-score decay in June that is predictive of July melt at better than three-to-one odds over the institution’s base melt rate. That is a signal you can act on. The students are still in your pipeline. A targeted June intervention — a counselor call, a family-portal re-engagement, a financial-aid reconfirmation message — catches most of them.
This is the kind of finding that only surfaces when the score has recency, direction, and cohort-baselining baked in. A “total opens” score would show those students as moderately engaged — they opened the early-spring emails. The decay is invisible to the flat count.
For the broader context on why June-to-July is the melt window to focus on, see the complete guide to preventing summer melt.
What this looks like in Slate
Celia writes the engagement score into a Slate field — typically ss_celia_engagement — alongside a trend indicator and a compact signals field listing the top drivers. Your counselors sort by it. Your reports filter on it. Your VP pulls distribution curves over time to see how the cohort is moving in aggregate.
There is no separate dashboard to open. The score and its reasoning live where the rest of the student record lives.
The summary
Engagement scoring done badly has poisoned the well for engagement scoring done well. If you have been burned by a score that turned out to be “email opens in a trench coat,” we understand the skepticism.
A real engagement score accounts for recency, depth, channel mix, direction, and cohort baseline. It comes with reasons. It lives where the counselor works. It gets recalibrated as outcomes accumulate. And it gets overridden by counselors when they have context the system cannot have — and the overrides feed back into the next cycle.
That is what we built. That is what an engagement score should be.
Tagged
