Tier 3: Full-size Playtests

1. What It Is

Full-size Playtests have more of everything: more players, more tailoring, and more insights.

There are two options:

• The Essentials with six (6) players

• The Deluxe with twenty (20) players

Like the Mini, players who enjoy games of a similar genre to yours will be required to play your game for at least 15, 30, or 60 minutes, depending on what makes sense for your build and budget. (Players can play longer than the minimum if they want!) It’s done remotely without a moderator, so players play from the comfort of home while recording their gameplay and voice as they think and react aloud while playing.

Unlike the Mini, if your build could benefit from a bit of player guidance — say it’s lacking onboarding, tutorials, or there’s a known issue that players need to be guided around — I design instructions that help players be aware of how to get through or around it.

Also unlike the Mini, I add custom tasks if there are high-priority learning objectives that would benefit from more targeted focus. For example, if you’re interested in whether players properly understand a move, I might design tasks near the end of the play session that ask them to demonstrate that move and describe what they think it does. After playing, players also fill out a survey about high-priority topics, with some additional questions about the player’s gaming preferences and habits to see how closely they match the team’s “ideal player”.

In both the Essentials and Deluxe options, six (6) players’ footage and survey responses are closely analyzed to create Player Briefings summarizing top insights from sessions. However, in addition to Player Briefings, you’ll receive a Comprehensive Playtest Report that also includes insights on patterns across these players. If you choose the Essentials, you double the amount of opinion data insights compared to the Mini, and if you choose the Deluxe package, you’ll get graphs and charts of player opinions, with percentages and margin of errors so you can get a more confident sense of how players were feeling and why.

Although Full-size Playtests are larger, you start getting insights close to the same time that you would in a Mini Playtest — and I continue to share new insights throughout the analysis and synthesis process. I make sure to send you regular bite-sized updates throughout the second half of the project duration, so you’re not just sitting and waiting for a big report that drops at the end.

2. Why It Matters

• See all the reasons under Mini Playtests, plus…

• The Essentials package helps find most usability issues: Supported by usability research — particularly usability specialist Jakob Nielsen’s work (see article for more details) — five (5) users is often enough to identify most critical usability issues while maintaining a good cost-to-value balance. (This assumes iterative testing will be done throughout development, since new issues are often introduced as changes and fixes are added.) I default to one more player for a total of six (6) because I want to increase the chances that any major issues are caught. From experience, I’ve found that one additional player can be a source of valuable insights!

• The Deluxe package allows you to more confidently describe opinions of playtesters: With smaller playtests, you can understand what and why players felt a certain way, but you can’t make any generalizations about how many players would feel similarly — groups of 3 or 6 are too small because the margin of error is too large for them to tell you anything helpful. With 20 players, that starts to be enough people where you can use statistics to more confidently describe player opinions. For example, if 75% of the 20 players liked the level of challenge, you can be fairly sure that most players of a similar background would agree — even with the margin of error, the % is over 50%.

• Full-size Playtests come with rolling insights drops that increase how much you learn: I make sure to share insights as I complete my analysis and synthesis. This is valuable and ideal for busy teams, since you can digest insights in small quantities over multiple days, rather than just one huge drop of a report at the end of the project. Leaving insights sharing only till the end, a common approach, can leave teams overwhelmed and only taking away a fraction of the insights.

3. Who It’s For

• Proactive teams who want to catch the majority of usability issues before they become baked into the foundation of the build

• Larger teams who want designers, producers, engineers, and stakeholders aligned around real-player behaviors and feelings – instead of assumptions – for more informed planning and decision-making

• Teams working on complex or multi-loop games who want to understand how the combined experience holds together in the hands of real players

• Publishers seeking player-centered insights for their studios to increase the chance of positive reception and reduce the risk of frustration or surprises at launch

4. What You Get

• REPLACE - USE SAME FORMAT [[Kickoff meeting to confirm scope and what you want to learn

• 3 players who enjoy games in a similar genre recruited via trusted games user research platforms like Antidote.gg

• Player answers to a post-playtest survey plus gameplay footage, including player voiceover of what they’re thinking or feeling as they play from home, giving you insights you can’t get just from watching gameplay

• A thoughtful analysis of the 45+ mins of gameplay footage and survey data that I synthesis into Playtester Briefings that summarize the most valuable insights from each playtester’s session, so you can quickly learn without spending time combing through the raw data yourself

• Issues prioritized by severity of likely player impact, each with a concise explanation and annotated screenshots, with short video clips for high-impact issues to make the problem clear

• Debrief meeting over video chat to go through the findings together so you can ask questions and get responses in real time

• Fast turnaround – typically 10 business days including kickoff day]]

5. How It Works

1. We have a Free Discovery call so I can better understand what you want to learn

2. You book a Full-service Playtest – choose either the Essentials or the Works package and whether players are required to play for at least 15, 30, or 60 minutes before they fill out the post-test survey 

3. You deliver a WIP “stable enough” build or send a video walking me through your game that I review on Day 1 of the project so I can come prepared to our kickoff

4. We have a kickoff meeting on Day 2 to confirm learning goals and I start work on designing the playtest

5. xxx

6. I carefully analyze the playtest footage and survey responses and synthesize it down into the most important takeaways and insights per playtest session

7. I walk you through the findings in a live video debrief call so you can ask me questions and get answers live

8. You get a Full Playtest Report [add more detail to differentiate] that you can refer to over the coming weeks and months, with issues prioritized by potential player impact so you have a better sense of where iterating will have more value