Creator Rating Desk

Reviews written from memory after a collab closes aren't comparable, and they aren't fair.

Part of my role is rating creators so the team knows who to rebook, retrain or replace. Those reviews were written after the collaboration closed, from memory. Recent events crowded out the rest: one late reply could outweigh three on-time deliveries, and two people rating the same creator would use different words for the same thing.

The facts were all in the thread: when deliverables arrived, how many revisions it took, whether nudges were answered. Nobody had time to reread it.

1. Paste the full thread from the creator platform, with the creator and campaign name.
2. Claude returns a verdict (worked well or had problems), 1–5 scores for quality, creativity and professionalism, two to five tags, and a 2–4 sentence summary ready to paste into a review box.
3. Each tag comes with one sentence from the thread as evidence. Tags and scores can be changed; anything added by hand is marked as such.
4. Save it to the shared rating log, which the team can scan and reopen.

1. A fixed vocabulary

   Tags come from two closed lists: 13 strengths (fast turnaround, strong hooks, follows the brief…) and 9 problems (needed many revisions, unresponsive, needed a reshoot…). Claude can't invent a tag. That's what makes the log searchable and two reviews of the same creator comparable.

2. No tag without evidence

   Every tag needs its own line from the thread naming the deliverable, the commitment or the date. If Claude can't point to it, it can't claim it. When I add a tag by hand, the evidence slot says so instead of staying blank.

3. Scoring rules that stop grade inflation

   A 5 needs the thread to show it. A 3 is ordinary, and also the score when there's no evidence either way. A missed deadline or an unanswered nudge caps professionalism at 2. If the thread is thin, Claude is told to say so and score conservatively.

4. Judge against what creators actually owe

   The prompt carries the standard deliverable: three videos with different hooks and the same body and CTA, each in Edited, No Music and RAW versions, named per the brief's convention. So "follows the brief" means something specific.

5. Works without the AI

   If Claude isn't available, the page switches to manual mode: stars, tags and summary all work by hand and save to the same log. The tool is a rating form first and an AI assistant second.

3

scores per creator, each with a written definition

22

fixed tags, so every review in the log uses the same words

1

shared log for rebooking, retraining and replacement decisions

Free-form AI ratings read well and compare badly. Constraining it, with closed tag lists, a rubric and an evidence requirement, made the output less fluent and far more useful. The constraints are the product; Claude just fills them in faster than I can.