How we actually measure character consistency

What "consistency" actually means, technically

Every character on ToonMate has a real reference image — the version everyone agrees is "the character." Every new generation of that character is compared against it using CLIP embeddings, a real, established computer vision technique that converts an image into a numerical representation capturing its visual meaning, not just its raw pixels. Two images of the same character, even in different poses or scenes, produce embeddings that are mathematically close together. A different character — or a drifted, off-model version of the same one — produces embeddings that are further apart.

That distance becomes a real, calibrated similarity score, checked against a real threshold. Above the threshold, the generation passes. Below it, it's flagged before you ever have to catch the drift yourself.

A real mistake we made, and how we found it

The honest version of this story is more useful than a clean one. Our first calibration used a single character and produced what looked like a solid, single-number threshold. Once we tested a second character — visually very different from the first — with a controlled test comparing generations made with a reference image against generations made without one, the single threshold broke. One confirmed, visually-verified correct match scored below several confirmed incorrect ones.

The real bug wasn't the threshold value — it was pooling two genuinely different populations of data together. A generation made with a reference image has something real anchoring it to that reference; a generation made without one has no such anchor, so its similarity scores spread wider even when the actual content is genuinely correct. Averaging both populations into one number was mathematically guaranteed to be wrong for one of them.

The real, current thresholds

Once we split the calibration by whether a reference image was actually used, the data separated cleanly:

• Reference-conditioned generations (the default, real path most users take): a threshold of 0.89 — genuine matches scored between roughly 0.90 and 0.96 in testing; genuine mismatches topped out around 0.88.
• Unconditioned generations (an explicit testing mode, not the default): a real, separately calibrated threshold of 0.82, reflecting that wider, real spread.

Both thresholds come from actual, human-verified test data — a real person confirming whether a given generation genuinely matched the reference character — not a number picked because it felt right.

What this doesn't claim to solve

One character against its own reference is a real, tractable problem, and it's the one this whole system is built to answer well. Multiple characters interacting correctly in the same scene — each one staying independently consistent while appearing together — is a real, harder, related problem the wider industry has openly said remains largely unsolved. ToonMate checks each character in a multi-character scene independently, which is real, genuine progress on that specific problem, not a claim that it's fully solved.