Read playtest data
A higher completion rate does not prove a better game: check denominators, counts vs. shares, and sample limits first.
Check the denominator first
Every rate on the playtest panel uses browser sessions with a recorded start as the denominator. A session is browser activity, not a unique person: the same player in another browser, or after clearing site data, appears as a new session. Events missing a start are excluded from every rate.
Counts and shares answer different questions
Action counts accumulate per session and can exceed the number of sessions; they describe intensity. A session share describes how widespread a behavior is. Neither number is a percentage of players.
What small samples can say
A version experiment shows a comparison only after each side reaches its sample threshold (10 starts by default). Even then the result is directional evidence, not statistical significance. Publishing two versions one after another is not a randomized A/B test: the time windows, audience mix, and player practice all differ.
- If samples are insufficient, keep collecting before drawing a conclusion.
- If the direction regressed, review the version change or record a rollback intent.
- If the direction improved, record the decision and keep collecting.
- If data was trimmed or is missing, reread the coverage note before interpreting trends.