One of the tenets of Design Thinking and Design Sprints is a bias for action – starting somewhere, and generating insights and learning that ultimately leads you to the creation of new or enhanced value (a.k.a. “innovation”).
Prototyping and testing are at the heart of innovation because this is where the real learning, and if approached properly, the even more valuable iterative learning, takes place.
We are absolutely thrilled to have sat down for an extensive Q&A session with prototyping trailblazer J Li, Co-Founder of Prototype Thinking Labs who gave us the real scoop on prototyping and testing. In this insightful interview, J shares her vast prototyping knowledge and wisdom, some pretty awesome stories and examples, as well as tips about how every team (no matter the size or experience) can bring rapid prototyping into their day-to-day — immediately.
We’re also very excited that both Founders, J Li and Olivia Wong will be coming to Toronto on November 20 to share their very popular “Prototype & Iterate 10X Faster” workshop. Consistently voted “best workshop” at conferences around the world, you don’t want to miss this one. More information on the event follows the Q&A.
Q&A with J Li from Prototype Thinking Labs
Q: Can you tell us a little about Prototype Thinking Labs? How did it come to be and how did you and Olivia decide to take the leap to build this company?
I spent years working in Design Thinking, systems thinking, and organizational development, but always felt that something was missing.
The missing piece was a question something like this: I love the approach of placing users at the heart of the design process and working with perspective on the entire interrelated system in mind– but how do you truly, practically, get things done with limited time and resources if you aren’t at the top of the hierarchy? How can we take human-centered design and make it really practical for teams of any size to use it in their everyday work?
I found the missing piece after learning about rapid prototyping at GoogleX. The idea was not to iterate once or twice, but to iterate dozens or hundreds or times.
Iteration is often the least understood, least systematized part of the design process, but it’s actually a superpower. Iteration allows us to rapidly build and navigate an arbitrarily complex space. If you think about the journey a baby goes through from knowing absolutely nothing to being able to understand complex human behaviors, it’s a process of thousands of iterations, not 2-3 “pivots”.
Olivia came from the nonprofit world, where she was a grant writer. There, she saw organizations routinely tackle massive, thorny human problems and expend immense resources on the wrong solution.
Prototype Thinking Labs is the art of iteration – of turning “Let’s Try” into something that will inevitably succeed because we learn enough through the process. We’re both obsessed with sharing it with the world because we see the incredible impact that it has on realistic problem solving.
Q: What common problem(s) are you helping your clients solve?
The most common problem we solve is how to create something new.
It sounds simple, but it’s actually not. It’s in no small part because we’re brought up in society as high performers, with problem-solving habits optimized to reproduce or expand on something old. It’s often the opposite of what we should do.
We’re taught to have high standards for delivery, think and plan carefully, try to get an “A”, raise buy-in through strength of reason and debate.
If I need to build a bridge across a river, I can lock an engineer in a room and give them the specs, and they can come up with several bridge plans that clear my requirements for load, size, budget, etc.
This is because all bridges, all cement, all physics is the same everywhere.
But if you’re creating a new product or service or process or tool or strategy or other type of solution for a particular group of people, you’re solving a problem that humanity has never solved before.
It sounds dramatic, but it’s true: all people are different, so a process inside your organization is going to be different from a process in a different organization. Many people create software products, but yours has a specific design and a specific audience that’s unique.
In order to do it well, and reliable, given that there is no pre-established way to approach it, requires a completely different approach and set of skills.
We help teams who make new things unlearn the habits of reproducing old things and master the art of making new things.
Q: How would you define a “prototype”? Is it something any organization should be thinking about, no matter product/service or other?
A prototype is anything that you make or try to get a quick picture of how something is likely to work before producing the fully formed version.
There’s often a belief that a “Prototype” is some sort of software or hardware mock-up that you need formal skills and tooling to produce, but that’s not the case at all! No matter what your idea is, there’s a way to try it out and learn more about it before committing.
- You can prototype an HR policy by writing up some quick talking points / scripts and a memo, and role playing through some critical interactions with people.
- You can prototype a new process by just making a diagram, translating it into an instruction checklist for each of the key stakeholders, and asking them to simulate through how it will impact them.
- You can prototype software by drawing out screens of the software and having the user react to the functions therein.
- You can prototype a new hardware product before investing R&D into actually building it by creating a mockup of the packaging, specs, sample reviews, and approximating shape or appearance with other tools. Then show those to potential customers and see how valuable it is to them.
- You can prototype a partnership by mocking up a few key things that could come out of it, and bringing both the partner and users into the room at the same time so everyone can iterate on the possibilities quickly together.
More broadly, Prototype Thinking refers to literally thinking with prototypes. In any situation where you’re not sure, ask yourself, “what can I easily try to do to get a little more understanding and perspective here?”
When trying to figure out how to easily make a prototype, ask yourself: “What part of the final interaction is actually most crucial to the user?” and then build only that. The rest of it doesn’t need to be hooked up and functional for you to answer your most important questions.
Q. What is the core purpose of prototyping and rapid iteration? Does that change depending on what you’re trying to achieve in terms of business goals?
Learning. Everything comes down to learning.
The goal of prototype and iteration is to make the right decisions because you deeply understand what’s going on and why – in a fundamental way.
Have you heard of the difference between Precision and Accuracy? Precision is working in a polished, to-spec way and accuracy is working on the right thing. Here’s a diagram:
Have you ever been in the upper right corner? Had a team build something out in detail and turn out to be the wrong thing?
No matter what you’re working on, and what level, the purpose of prototyping is finding the center of the dartboard before firing the rest of the (much more expensive and labor-intensive) shots.
Q: I’ve heard you mention “Minimum Unviable Product” – that’s really intriguing, given how much we hear about “Minimum Viable Product.” Can you explain what you mean by that, why it’s important, and how it differs from an MVP?
Aha!
A Minimum Viable Product is a concept from Lean Startup. It refers to building a functioning, live, bare-bones version of something that you can run so you can see how it works.
In Prototype Thinking, a Minimum Unviable Product is the easiest thing you can build that doesn’t even function, but can still give you about 85% of the information you need to know.
You do this simply by simulating the core functions one at a time, instead of all together. This also allows for extra scrutiny and ease of revision. There’s a great story about this:
In 2017, Dick Costello, the former CEO of Twitter, raised several million dollars to create a startup called Chorus. The idea was that people would sign up on the app, set a goal, and be put in a community of other people with similar goals so that doing it together would help them reach it.
He hired an entire development team, launched his MVP to fanfare, and after about 8 months the startup crashed and burned because they discovered that the core design ran up against a fundamental roadblock in human psychology, the Abstinence Violation Effect, where essentially people don’t want to check in with the group if they’re not doing well and falling behind. Since having contact with peers when things got rough was supposed to be the entire point of the app, they essential didn’t have a product.
With a Minimum Unviable Product, Costello could have essentially figured the same thing out with about 2 weeks and $200. Instead of building an entire functioning platform, you can capture the core experience of the design using hacked-together simulated functions.
For example, they could have taken the core design of the app and run a quick pilot for a small group of volunteers using only existing tools— text messaging or Slack, a Google photo album or shared document, having someone doing manual updates for all the automated components, and so on. This would have allowed them to test all of the fundamental components of the central group-support-and-motivation-experience itself (and quickly change them around) without any expensive engineer labour.
The nice thing about a Minimum Unviable Product is that it’s also trivial to make changes. Instead of hooking up and launching each new thing you want to try, you can just have the actor behind the curtain try a different behaviour, or draw a different paper prototype or Google doc.
Q: What is a common pitfall that people/organizations fall into, and how can rapid prototyping help?
Broadly, I think the pitfall is not being aware that you just need more information.
All the time, I see teams who are planning to roll out something– anything, could be a plan, a product, a solution, etc– and debate with themselves and one other over how to do it. It can feel like a very heavy, involved decision making process, and they spend a long time considering different angles of approach and weighing pros and cons.
We have a saying: Your team is smart. If you can’t solve it in 20 minutes of discussion, you’re not going to solve it in 20 hours.
Almost always, in cases of agonizing uncertainty and debate (even if it’s just one person with themselves), it’s because there’s information about the situation that’s missing.
THIS is the best time to prototype: call a PAUSE, stop the debate, and everybody go mock up quick designs of what they’re talking about. Half the time, literally just having everyone concretely diagram their own point of view separately will already immediately solve the problem. The rest of the time, it takes just a couple hours to put it in front of a few prospective users and get the context you need to make the decision obvious.
Q: What are some of the myths you come across that pertain to prototyping and testing that you’d like to help bust here and now?
Myth #1: “I’m not ready to test / to show this to real users yet”.
We often conscientiously don’t want to come out and share something that is incomplete: we feel like we want to put our best work forward, and that the user deserves a good experience.
However, the 
entire point of testing is to make it complete: the best time to test is during maximum uncertainty, not after confidence is reached.
You won’t disappoint your users by putting something awful / unfinished / unviable in front of them. Instead, just set expectations clearly: this interaction is about showing how much you truly value their feedback and involvement in the creative process, not about what you can already do for them with what you’ve made.
When expectations are set properly and appreciation is shown, user testing should give you more social capital than you spent.
If Branding is worried about your putting out something incomplete / foolish under the organization’s name, just meet in an off-site location and hide your identity under a fake/imaginary brand “similar to X, Y, and Z”.
If your client is internal, just be very clear that you’re early in the exploratory process and value their input before you get too far.
Myth #2: “I need to find a great idea”
Killer ideas don’t really come out of ideation sessions: they come out of experience and data.
If you’re prototyping and iterating a lot, you don’t need to start with a single great idea because you’ll be revising the idea constantly through user contact. The deeper your understanding of the situation, the more of the user’s life / beliefs / experiences / internality around the challenge space you’re exposed to, the more likely you will eventually come up with the right idea.
If you try enough different things fast enough, it’s impossible not to get the perspective you need to find the right one.
Q: Is it possible to prototype operational processes? How does that differ than products or even services?
Absolutely. To prototype an operational process, as with anything else, we start by identifying the biggest uncertainties:
- What are the biggest questions you have about the design?
- What are the biggest broad unknowns?
- What are the most important things to go well that you’re not sure will work as intended?
Rank all of them by how important they are. Then, as with anything else, we get concrete:
- What are the stages? What goes in and out at each stage?
- Who is involved?
- What behaviors need to be changed?
- What information needs to be understood?
- How does it all fit together?
Then, we simply conduct a series of experiments at each of the major risk points:
- Will this team successfully adopt? Let’s put it in front of them
- Will the numbers for all the moving pieces add up? Let’s design a quick simulation. And so on…
Q: What about more intangible things, like employee engagement or group interactions? Or exploring running new business models?
Employee engagement is one of my favorite things to test, because you have a group of very accessible, very engaged users who are highly committed to helping you succeed!
The truth is that nothing is really intangible — it’s only intangible if you haven’t made it concrete yet. If you have an initiative for improving employee engagement and group dynamics, for example, then you need to have the following:
- A really specific plan (tangible! See directions in the last question on how to test)
- Metrics of success (tangible! Just go set something you can measure at least qualitatively)
- Human reactions (tangible! Just watch closely for them)
More broadly, I think what this question is getting at might be more like this:
Let’s say that an initiative will lead to a series of outcomes where the true metrics can only be assessed months or years down the line, and in the meantime there’s an emergent, complex system that can’t be easily broken down. How can you test it in a reasonable time?
For this type of challenge we look for secondary metrics in the form of leading indicators. These are things that we know are highly correlated to the ultimate outcomes, but can be measured much sooner.
For example, when we evaluate the likelihood that a business model will succeed, we look for a high amount of value given compared to the status quo, using a particular scale that we’ll explain in the workshop. In our experience, a 3-point offset on this scale just seems to be the tipping point for success.
.png)
Team members spending time listening to one another and sharing airspace is a leading indicator for more effective outcomes.
You may have to spend some time thinking about what the indicators are for your particular outcome. Ask yourself: what are the “red flags” for failure or “green flags” for success that you’ve seen in the past?
Then we look for those correlated outcomes in early testing.
Q: What’s the strangest or most unusual prototype you’ve ever built?
One of my favorite stories is when we were testing an enterprise software solution for HVAC companies to manage their business processes. That may sound dry to some, but it was actually a fascinating project!
We were running a sprint during business hours, and actually had a really hard time getting technicians who were able to come test during that time. First of all, it’s not like there’s a database of HVAC technicians you can just email to recruit users, and companies aren’t willing to give out the contact info of their employees. The ones who are free want to use their precious spare time doing something else.
Finally, we realized that literally just hiring a technician to do an inspection walk-through of our building was cheaper than some of the elaborate incentive structures we’d set up to convince technicians to come in off hours. So we lined up a row of HVAC inspections one after the other.
That meant that we quickly got really good at explaining that we wanted their expertise in a different way, and can they try out these paper prototypes we made and talk through their thoughts as they were inspecting through our office?
So we’d have this guy bent down next to our boiler, checking the equipment on one hand and flipping through a bunch of index cards that we’d drawn up to be a mobile app on the other hand, telling us why at that moment he didn’t care about any of the stuff on the card.
.png)
Through that process, we learned we were actually completely wrong about what capabilities and knowledge technicians prioritized, and redesigned the entire app on the technician side.
As an added bonus, we also learned that, having recently moved into that office, we’d accidentally set up the boiler closet as a major fire hazard: it was a really good thing we had them over to inspect in July or we might have burned the building down once winter came!
Q: What kind of team should you have when building and testing prototypes?
We get asked this question a lot. The answer is, whoever’s involved in making decisions on the design should directly experience user feedback. And ideally one person with legible handwriting should help build the prototype. The end!
You don’t need expert designers or engineers to build prototypes until relatively late in the process — in the first ⅔ of your design process, the real point of a prototype is to ask questions, not to guess at answers. So anyone can build a concrete mock-up of the idea manually.
However, getting to actually see a user react to a prototype is transformative. This is actually also our #1 recommendation for getting executive buy-in: once you see emotions play across a customer’s face, hear excitement or curiosity or frustration in their voice, the level of understanding you suddenly get about their experience and needs is impossible to ignore.
Q: What advice do you have to ensure the team is getting the most out of user testing?
This is funny, but my #1 piece of advice is: DO IT.
The truth is that most common reason teams fail to get the most out of user testing is that they worry so much about “doing it right,” that they never get around to doing it at all.
Once you actually get in front of your user, there are a great many techniques I can share to get the most nuanced, accurate, useful, on-point, etc information out of them (and you will learn them in the workshop). However, all of that essentially boils down to two things:
- Show Them Something Concrete
Even if it’s just a drawing or diagram of what you’re talking about. show them something! Don’t verbally describe a hypothetical situation. This will help them give feedback from a literal, accurate place rather than sort of making up opinions in their heads. - Listen, Don’t Talk.
In a good user test, the experimenter should be talking less than 20% of the time. Don’t lead the user, find out how they naturally think about it on their own. (The guidance is already given by that concrete example you just made.)
Q: What’s the best way to select your user tester group? How many users should you be testing with?
The most important first piece to knowing who to test with is knowing what question(s) you’re asking. Begin by listing out your biggest uncertainties, and rank them in order of how important they are compared to how uncertain you feel about them. Then take the very top one and ask, “What users do we need to get in front of in order to understand this?”
Then, just go talk to a handful of them. Studies show that talking to just 5-6 users will give you about 85% of the information you need on whether something is workable. And we’ve found that usually if you just try 3-4, you’ll already get a pretty good picture.
Most importantly, you don’t have to worry about having your entire user testing plan in place before you start: Your first few tests will give you the context you need to make an educated decision about the next batch of tests.
The testing process itself is iterative.
About Prototype Thinking Labs
At Prototype Thinking Labs, we are obsessed with the point where the moonshot meets the practical. We’ve invented sophisticated shortcuts to make design sprinting, prototyping, and testing genuinely doable — by any team, in any industry, at any experience level, on any timeline, with any budget.
Just a few of Prototype Thinking Lab’s many clients include: Google, Fitibit, Intuit, Vanguard, Mastercard, Capital One, Kaiser Permanente, Humana, L’Oreal, Bang & Olufsen, Lululemon, Telus, US Cellular, Tencent, Stanford University, and Unicef.
Prototype Thinking Labs is created by J Li, Olivia Wong, and Wes Wakefield and are based in San Francisco and Vancouver. Learn more about Prototype Thinking Labs here.
Design Sprint & Innovation Masterclass: Prototype & Iterate 10x Faster
Prototype Thinking Lab’s workshop is Day 2 of a premium three-day Design Sprint & Innovation Masterclass that includes the Official Design Sprint Bootcamp with John Zeratsky (co-founder/co-author of the process and book), and a third day focused on advanced facilitation.
We can’t wait to work with Prototype Thinking Labs on November 20!
Sign me up for the Prototype and Iterate 10x Faster workshop: http://bit.ly/sprint-prototype
