A high-fidelity user prototype is still a "smoke and mirrors" style simulation, however, now it looks very real. In fact, with many good high-fidelity user prototypes you need to look close to see that it’s not real. The data you see is very realistic, but it not real – mostly meaning it’s not live. For example, if I do a search for a particular type of mountain bike, it always comes back with the same set of mountain bikes, but if I look close, it’s not the actual bikes I asked for. And I notice that every time I search it’s always the same bikes no matter what price or style I specify. Now if you are trying to test the relevance of the search results this would not be the right tool for the job, but if you are trying to come up with a good overall shopping experience this is probably just fine and very quick and easy to do.

[...] The big disadvantage of a high-fidelity user prototype is that it’s not good for proving anything – like whether or not your product will actually sell. It is good for testing out usability, and it’s great for communicating the proposed product to key stakeholders, and it’s great for rapid learning. Where a lot of people go sideways is when they create a beautiful user prototype, and put it in front of 10 or 15 people that say they love it, and then they declare victory. Unfortunately, that’s not how it works. People say all kinds of things and then go do something different. We have a much better tool for proving whether something actually works (described next).

But my favorite use of a high-fidelity prototype is not to see if users like it, but rather, to try to learn why they don’t. When you test with a high-fidelity user prototype, you don’t get your answer from any one user, but every user you test with is like another piece of a puzzle, and eventually you see enough of the puzzle that you can see where you’ve gone wrong.

Fidelity has to do with how realistic the prototype looks. A low or even medium-fidelity user prototype doesn’t look real – it is essentially an interactive wireframe. “User prototype” means that it is a simulation and not something real. In other words, you can enter your credit card info as many times as you want, you won’t actually buy anything.

Balasamiq is pretty much everyone’s favorite low-fidelity user prototyping tool as they have made it so quick and easy to create this form of prototype. Low–fidelity user prototypes are typically created by the interaction designers. Many teams use these as a way to think through the product among themselves. For certain situations you can also identify usability issues right at this very early stage.

As great as they are, low-fidelity user prototypes only represent one dimension of your product – the information and the workflow – there’s nothing there about the impact of visual design, or the differences caused by the actual data, as just a couple important examples.

These are written by engineers in order to address technical risks during product discovery before we decide whether something is feasible or not.  Sometimes the engineers are trying out a new technology.  Sometimes it’s a new algorithm.   Lately it is often about assessing performance (think mobile app transitions, or Big Data).  The idea is for the developer to write just enough code to be able to answer the feasibility question.  It may be throwaway, or it might turn out to be something you can leverage if you decide to proceed.

Live-data prototypes are a little tougher to explain, but they are absolutely critical and the cost of producing them is dropping rapidly so I’m loving them more all the time. The main purpose of a live-data prototype is to actually prove something – normally it’s to prove whether an idea (a feature, a design approach, a workflow) really works. In order to know this, we typically need to do two things. First, we need the prototype to access our real data sources – like actually search our live inventory and show products that are really available right now. Second, we need to be able to send live traffic, in quantity, to the prototype.

The key is that we sure don’t want to have to build, test and deploy a real product in order to do this. That would take far too long, and cost far too much, and yield huge waste. And we don’t. A live-data prototype is a very limited implementation – typically just the critical use cases, and none of the “productization” that’s normally required like full use cases, test automation, SEO work, internationalization and localization, performance and scalability, etc. A live-data prototype is a fraction of the effort, but you get big value. You do have to keep in mind two big limitations. First, this is code so it requires your developers to create the live-data prototype and not your designers. Second, this is not a product, and you can’t run a business with it, so if the tests go well, you’ll still need to allow your engineers to take the time to productize the code.

[...] Normally we’ll test a live-data prototype in an A/B test, but we can also do an opt-in test or an invite-only test. The key is that real users will use the live-data prototype for real, and this will generate real data (analytics) that we can compare to our current product to see if this new approach actually performs better.