By Mehdi HassanWhat should you evaluate during a design agency trial?
Evaluate how the team works as well as what it produces. A one-week trial should be scored on five criteria: speed, communication, copy, design craft and code quality. Together they cover the questions a portfolio can't answer: how fast the team moves on your problems, how it talks to you, and whether its work holds up on your real product.
This scorecard is a template you can copy into a doc or spreadsheet. It works for any studio, not only ours. If you haven't planned the trial itself yet, start with our hub on how to run a paid trial with a design agency, which covers choosing the task and writing the brief.
The scorecard is most useful when you set it up before the trial begins. Deciding the criteria in advance stops the final impression, good or bad, from overriding everything you saw during the week.
What are the five criteria in the design trial scorecard?
The five criteria are speed, communication, copy, craft and code quality. Each gets a score from 1 to 5, a weight, and a line of evidence. The suggested weights below suit a website project. Adjust them for your next engagement.
Criterion | What it measures | Suggested weight | Evidence to collect |
|---|---|---|---|
Speed | Work in progress early, definition of done met on time | 20% | Dates of first draft and final delivery |
Communication | Updates without chasing, questions early, risks flagged | 20% | Messages and review summaries |
Copy | Writing specific to your product and buyers | 20% | The headline and body copy produced |
Craft | Typography, spacing, hierarchy, responsive states | 20% | Figma files and mobile screens |
Code quality | Works across devices, accessible, editable | 20% | Staging link, a phone test, a quick accessibility check |
If the trial is design only, with no build, drop code quality and spread its weight across the other four. If you're choosing a team for a retainer, raise communication, because you'll work with them every week. For a broader comparison of agencies before any trial, our agency evaluation scorecard covers portfolio, pricing and fit.
What does each score from 1 to 5 mean?
Each score should mean the same thing every time you use it, so write the definitions down. The anchors below describe a 1, a 3 and a 5 for each criterion. Scores of 2 and 4 sit between them.
Criterion | 1: poor | 3: acceptable | 5: excellent |
|---|---|---|---|
Speed | Nothing seen until the last day, or done missed | Done met, with one draft mid-week | Early draft, quick revisions, done met with time to spare |
Communication | You had to chase for updates | Regular updates, some questions late | Proactive updates, early questions, risks flagged before they hit |
Copy | Generic lines that fit any company | Accurate and clear, a little safe | Specific, uses buyer language, answers real objections |
Craft | Inconsistent spacing, weak hierarchy, broken mobile | Clean and consistent, few surprises | Clear hierarchy, careful detail, strong on every screen size |
Code quality | Breaks on a phone, hard to edit | Works on main devices, mostly editable | Fast, accessible basics covered, easy for your team to edit |
How do you fill in the design trial scorecard?
Fill it in on the day the trial ends, alone first, then compare with anyone else who reviewed the work. Each score needs a short line of evidence: a date, a quote from a message, a file. Without evidence, the score is an opinion.
Set it up before the trial
Copy the template, agree weights with your decision-maker, and write the pass line.
Take notes during the week
Note when drafts arrive, which questions the team asks and how feedback is handled.
Test the output yourself
Open the staging page on your phone, tab through it with a keyboard, and read the copy aloud.
Score each criterion with evidence
Give a 1 to 5 score and one line of proof for each of the five areas.
Calculate the weighted total
Multiply each score by its weight and add them up for a total out of 5.
Apply the decision rules
Check the pass line and any deal-breakers before you talk to the agency.
Hypothetical example: a filled-in scorecard
For example, suppose a startup scores a studio after a trial week on its homepage hero, using equal weights of 20%.
Speed: 4. A first copy draft arrived early in the week, the final build a day before the deadline. Communication: 5. Updates came daily, and the studio flagged a missing product screenshot on day one. Copy: 4. The headline used the exact phrase customers use for the problem, though the subhead was a little long. Craft: 4. Strong on desktop and mobile, with one spacing issue on tablet. Code quality: 3. Works well on phones, but one image was not compressed.
Weighted total: (4 + 5 + 4 + 4 + 3) × 0.2 = 4.0. With a pass line of 3.5 and no score of 1, the startup moves on to scoping the full project.
This is an illustration of how to use the template, not a real trial result.
How do you decide after scoring a design trial?
Decide with rules you set before the trial: a pass line for the weighted total and a deal-breaker rule for any single criterion. One workable setup is a pass line of 3.5 out of 5 and an automatic stop if any criterion scores a 1. These are suggested thresholds, so set your own.
- Above the pass line, no deal-breakers: move to scoping the full engagement.
- Near the pass line: name the weakest criterion and ask the team about it directly. A good answer can move you forward.
- Below the pass line or a deal-breaker: stop, pay for the trial, and keep the output.
- Unclear because of your side: if feedback was slow or the task too big, the trial didn't test the team fairly. Consider a second, tighter piece of work.
Also weigh what you'll need next. A team that scores 5 on craft but 2 on speed may be fine for a careful brand project and wrong for a fast launch. If speed is the priority, read building a website in one week to check what's realistic.
What should you not judge a design trial on?
Don't judge a trial on volume, on taste alone, or on things the team couldn't control. A trial that produces one excellent, well-reasoned section is a better sign than one that produces five rushed ones.
- Amount produced: more screens isn't better if none of them are finished properly.
- Personal taste with no reason: if you dislike a color, say why it doesn't fit the goal.
- Delays you caused: late access or slow feedback shouldn't count against the team.
- Price alone: compare the trial to what the next engagement needs, not just to its cost.
Finally, remember why you paid for the test. Our opinion piece on paid trials vs spec work explains why real stakes give better signal, and our trial week guide shows how we scope ours. If the trial goes well and you need to make the internal case for the full project, see how to justify a website budget.
