Home » Blog » Archive

Posts Tagged ‘A/B testing’


Creating simple usability tests with IntuitionHQ

Posted by john on August 20th, 2012

A website usability audit can be a great marketing tool

Everyone wants to be billing their clients and I’m sure thinking up some good ways to justify this is a major occupation. Well we here at IntuitionHQ have a solution: consider a usability audit of a client’s website that includes a few simple suggestions on how to improve the usability of their website.

So where are the billable hours coming from? Not too many are going to come from the usability testing itself. IntuitionHQ makes usability testing incredibly easy. Within a few minutes you can create a number of tests on the usability of a website and send it to a group of respondents. Results are compiled instantly and you can gain immediate insights from the usability test. Usability testing will highlight your web development skills and is a great marketing opportunity for you to leverage implementing the improvements to your clients website.

Here is our guide to a usability audit that is easy to put together, easy to interpret, will give you billable hours and some great reasons to pick up even more work from your client. Don’t forget that your client and their website visitors are the ultimate beneficiaries, as Amazon’s much fabled $300 million dollar button proves.

Step 1: Spend a few minutes analysing the website

Websites have differing functions. An ecommerce site should make it easy  to register, navigate, and purchase goods and services. A traditional store may need to build a community while it provides simple advice of location and purpose. Most companies want to give customers information to avoid costly call centre calls.

Whatever the purpose, look for a few obvious things on the website that look a little dated. Usability thinking has come on hugely in the last few years and a website more than 3 years old will need few tweaks. For some really useful insight on how to get started check out these 7 great tips for writing usability questions

Step 2: Develop some easy website usability tests

Did you know that people are much more likely to click on a contact button with a picture of a person on it? This has been widely tested in usability studies and you can prove it to your client with an easy usability test run using IntuitionHQ:

Does the website have a main picture in the website? You can run a quick A/B preference test on whether the current picture is preferred or an alternative:

A simple example we ran was on whether the cat or dog was cuter:

If a website requires a specific task of visitors, then testing how long it takes visitors to accomplish this task compared to an improved design can show how website usability can be improved.

In the example below, the respondents are shown either the website on the left or the right. They are asked where to click to find the blog and IntuitionHQ automatically times how long it takes for respondents to click on the blog button. You can see that the respondents shown the website on the right found the blog button almost twice as fast as those shown the website on the left.

Conclusion

Taking the initiative and proactively using a usability audit on a customer website will provide a great marketing opportunity for you.  You’ll be able to engage them on a range of ways you can improve their website with the benefits sure to follow for their website users.

 

 

12 Website Usability Testing Myths

Posted by Jacob Creech on July 12th, 2011

 
The internet is a wonderful, magical place that is filled with more amazing content than you could shake a stick at; it has an almost unimaginable wealth of resources on a huge array of different topics, and more or less anything you can think of exists on the internet.

The problem though, is not that there is too much content, nor that there are too many sites, it’s just that the vast majority of sites and services suffer from a number of different usability issues that make using them anything from difficult and frustrating to downright unpleasant to use. I’m sure you can think of a number of sites off the top of your head that fit into these categories.

Unfortunately there are a number of different myths floating about saying that improving usability takes too long, costs too much or doesn’t really do anything useful to these sites and services. As someone who works on a website usability testing tool I hear these myths far too often, and I’d like to dispell them permanently.

Read on to see 12 Website Usability Testing Myths, and why they are wrong:

12 Website Usability Testing Myths


Usability testing is pointless because we won’t make changes anyway

Change ahead

I’ve heard this very depressing argument a number of times, and while I understand that you may not have all the development resource at your disposal to implement required changes, you might still find some things that don’t require much time or effort to change, and that could make a substantial difference to your site and user experience.

I’ve encountered people who initially thought that the business wouldn’t see the value of making changes, but upon being presented with testing results saw how a few relatively inexpensive and relatively fast changes could make a big difference to their bottom line. It’s pretty hard to argue when you have results in front of you showing exactly what is wrong.

Even if there is no chance that you can make changes in the near future, at least you have some idea of what is going wrong, and if you ever do get that development resource, you can implement the required changes.

It will just get overruled through ‘design by committee’

Design by committee

This is a myth I’m very happy to dispel. If design by committee is a designers worst nightmare, then usability testing is the solution to it. If you have been told to put this button there and that button here, and you can present results showing why one location is clearly better than another then it’s very difficult for anyone to argue with those results.

If people start suggesting changing you text to comic sans, and using sky blue text on an azure background, then you can provide testing results showing just how laughable this idea is. Of course, the ideas may not seem so ridiculous, but you get the idea.

Present testing results that showcase a few different options, and it’s very difficult to disagree with the one that works best.

It takes too long

A long way to go

This is a very frequent excuse; many people are under the impression that running a usability test requires weeks of time, and a number of dedicated staff members in order to get any results whatsoever.

The truth is that with modern website usability testing tools that you can create and share a test in just a few minutes.

For example, when creating a test with IntuitionHQ, you simply upload some screenshots or designs, write the questions, and you are good to go. Share you test via email, Facebook, Twitter or any other medium you see fit and you get great results in no time. If you can write an email, you can create a usability test. Simple, and very quick.

It costs too much

Usability Testing can be low cost

Many people are under the mistaken impression that running usability testing costs thousands or even tens of thousands of dollars. The truth is with remote testing tools you can run tests for a pittance.

Tests with IntuitionHQ cost just $9, and include unlimited questions and respondents. There are other tools out there that enable you to capture feedback in different ways and don’t cost much more.

Whatever your budget, there is a tool to suit it.

It’s impossible to convince management to run tests

Convince management to usability test

We were recently working with a local government agency who were very interested in testing, but couldn’t see how they could sell management on the concept. We showed them how cheaply, quickly and easily they could set up a test, and just by testing with internal staff they got over 200 respondents on their test.

Not only were management blown away with the results they received, but by involving staff in the process the enthusiasm levels went through the roof.

If management can see the value that some simple testing can provide, they will be very quick to get on board. Management love metrics; show them some testing metrics, and they will love it.

My site is perfect, there is no need to test

How to build a perfect website

I’ve yet to see a site that couldn’t do with a little tweak here or there, but even if you think your site is perfect, wouldn’t you rather have the evidence to back that up?

Also, as I’ve already mentioned, people change, trends change. Your site may be perfect now, but I can guarantee it won’t remain perfect for long; look at sites made even a few years ago, and I’m sure you will understand what I mean.

The point here is even if it’s perfect now, it may not always be. With a little testing you’ll catch any problems as and when the crop up.

It’s impossible to show the value of testing

Show the value of Usability Testing

Over on UIE.com they have a remarkable story about a $300 million button. The long and short of it was that by forcing users to do something they weren’t interested in, that company was costing itself $300 million. Obviously $300 million is more than most sites can hope to gain but the point is, every little improvement can help.

How can you know what changes you should make and what things will add value for you company? Usability testing. We’ve done some interesting example website usability tests of Gizmodo, TED and the iReddit iPad app and found some very interesting results. Check out our analysis of the Godaddy User Experience for a fantastic example.

Our testing showed some simple changes that each site or app could make that would dramatically improve user experience – a different navigation label, moving around some key social media icons or following design conventions can all make a big difference to your site and your users. In todays competitive environment you simply can’t afford to miss out on these improvements.

Users don’t care about usability

Involve your users

We’ve run tests on a number of different sites, and every time we get feedback from users saying how great it is that that site cares about usability, that they care about their users, and they want to get users involved in developing the site.

It helps users feel involved in the future direction of the site, and builds up passion and community around a site. The more users are involved, the more committed they will feel to your site.

If users feel you are responding to their needs they will keep on coming back, they will recommend you to their friends, and they will be a great advocate for you. In a day when the switching cost of changing from one site to another is so low, building this passion and commitment is what helps you stand out from your competition.

You need an Human Computer Interaction degree to understand usability

HCI Degree

Again, looking at our website usability testing examples you can get an idea of how simple setting up usability tests can be. Think of some important points you would like you users to accomplish on your site, take some screenshots and get ready to go.

When looking at your results, you get a very quick idea of what is going right and wrong. If it takes users 20 or 30 seconds to find your RSS feed subscription button, you know you have a problem. If only 50% of users can find your signup page, you’ve got a problem.

None of this is rocket science, and if you ever do get stuck or have a question, we’d be more than happy to help point you in the right direction. The truth is though, it’s really not (or at least, doesn’t have to be) that hard.

Designers already know what they are doing, they don’t need to run usability tests

Different users and different personas

While I completely agree that there are a number of fantastic designers out there, even the best designer in the world can’t be expected to understand the needs of an entire user base without a little feedback.

Usability testing gives them that feedback, and helps them understand how their users think and what sort of things their users are looking for. If you think Apple released the iPod or iPhone without any kind of testing, then you really need to think again. To make a great site, service or product, you need that feedback.

Remote testing tools enable you get that feedback quickly, easily and effortlessly, and as designers, we can tell you this is something designers understand and appreciate.

I’ve already tested my site in the past, there is no need to test again

Try try try again

Congratulations; you have taken your first step in the right direction, but this logic is the same as saying something like ‘MC Hammer was the height of fashion in the 80′s, therefore my parachute pants are still in fashion’.

The truth is your audience changes, trends and fashion change, even design conventions change. Testing you site on a regular basis ensures that you are always improving your user experience, that you are keeping up with design conventions, and that users will continue to use your site, and come back in increasing numbers.

Steve Krug advocates testing once a month to ensure you are up to date and know and understand how your users are interacting with your site or service.

Testing on a regular basis means there won’t be any nasty surprises further down the track; it’s easy to make a few small changes on a regular basis, than massive changes a little less frequently. Sensible, isn’t it?

It’s too difficult to get started

Getting started with usability testing

While it used to be true that usability testing required a lot of time and effort – recruiting participants, getting them along to usability testing labs, hiring expensive equipment and so on, times have certainly changed.

I’ve set up tests on IntuitionHQ in five minutes flat, and received hundreds of responses in just one or two hours time. All you need to do is write a few questions, upload a few screenshots, and you are really good to to. Send your test to your site stakeholders, put it on Facebook, Twitter or Google+ and watch the results roll in. It really is that simple.


Where to next?

If your interested in taking your next steps in usability, we’ve got a few posts you might like to check out:

There are also a whole bunch of great sites out there for learning more about usability and user experience that anyone interested in the topic should check out:

If you’ve got any website usability testing myths you’ve heard before, any questions about usability testing, or anything else we can help you with, please let us know in the comments below. We’d love to hear your experiences.

Thanks for dropping by!

Don’t forget to sign up for an IntuitionHQ account while you’re here. Signing up is free, and publishing your tests only costs $9.

You can also subscribe to our RSS feed and follow us on Twitter or Facebook to keep up with the latest news in the world of website usability.

 

Website Usability Test: Gizmodo.com

Posted by Jacob Creech on June 23rd, 2011

 
Welcome back to the second in our series of Website Usability Tests. It’s a great way to learn more about the usability testing process, and understand a bit more about about the thinking behind some very popular sites.

This time around we are looking at Gizmodo and seeing how the design and usability of the site stack up.

Update: I’m unsure if it’s related to this post or not, but Gizmodo seems to have removed all the old versions of the site. If anyone has a link to the old version, please let us know in the comments.

Read on to learn more about the site, see how we formulate our questions for this website usability test, have a look over our results, and see our recommendations for the site.

What is Gizmodo?

 
According to Wikipedia:

Gizmodo is a technology weblog about consumer electronics. It is part of the Gawker Media network run by Nick Denton. It’s known for up-to-date coverage of the technology industry and the personal, humorous, sometimes very inappropriate writing style of the contributors.

Basically, Gizmodo is a hugely popular site that reports on a range of different news about technology, gadgets (including one particularly well known post about iPhones) and a whole range of other interesting stories as they pop up.

The audience, according to Alexa, is largely male, and between the ages of 18 and 34. Considering the topic of the site, you can also imagine the users are pretty tech savvy folk.

The website

 
Gizmodo actually has several different designs across the different Gizmodo properties; Gizmodo.com uses the a newer design, and has an option to toggle between the usual, fluid design and a more blog styled version.

The older version (UK, Australia, Canada etc.) have retained an older design (that the main Gizmodo site also used to use). You can see the two versions below – although you should note we’ve cropped the international version because the page is really very long:

The New Version

Gizmodo - New Version

Gizmodo - American Version

The Old (International) Version

Gizmodo - Old Version

Gizmodo International Version

As you can see, the old version really is very long, which I imagine is one of the reasons they may have changed the design. The newer design certainly has a much more modern feel about it as well.

In order to better understand the site, we’ve made this an A/B test, looking at both the newer and older versions of the site. That way we can make a fairer assessment of how the two sites stack up, and the strengths and weaknesses of each one. If you are interested, you can even take the test yourself.

The Questions

 
As I talked about in our recent website usability test of the TED.com website, when writing questions, you should consider the key tasks that users are looking to complete on a site. The following are tasks that I think are important for users visiting the Gizmodo site:

How would you login to the Gizmodo site?
Getting members on the site is a great way to encourage community participation. Logging in is obviously an essential part of this.

How would you subscribe to the RSS feed?
Subscribing to a sites feed means updates get pushed out to you, and you are more likely to read the content they are publishing. For content based sites such as Gizmodo, this is obviously a great thing.

Your friend told you about a story called ‘Academics on why trolls troll’ – where would you find it?
Finding content on the site is another crucial factor. Whether you’ve been directed there by a friend, or come across it in some other way, it’s important to be able to find the interesting stories. It’s also useful to see if uses find the post on the page, or by using the search function – in which case they should ensure that search results are well optimised.

Which interface do you prefer?
For this question, users are shown both designs of the site and asked on their preference. This is a bit of a popularity test, but it’s good to know how users will react to your designs.

How would you follow Gizmodo on Twitter?
Getting a social tie in from users is another great way to encourage participation, and to lower the chance of them shifting to another website. It’s also a useful stream for users to access content.

How would you search the Gizmodo site?
As previously mentioned, finding content is a key for this site. The search function is something that we probably all assume to be extremely simple, but it’s worth testing just to make sure it’s easy to find.

How would you advertise on this site?
Writing content is all well and good, but the way this site makes money is through advertising. The more advertisers and the more competition the better. Hence, finding out how to advertise.

The results:

 
For the results, we’ve cropped down the screenshots to the crucial areas to save some space, but the test was taken on full screenshots of each design. On to the results:

How would you login to the Gizmodo site?

Gizmodo - Where would you login?

Where would you login - old version

Gizmodo - Where would you login - new version

Where would you login - new version

In the old version of the site, 79% of users clicked the login area correctly, compared to 100% on the new version. Interestingly, the users who clicked on the wrong area in the old version, all clicked on the search bar which is directly beside the login button.

This shows that users were confused, and thought perhaps that they needed to enter their details in that box. 100% for the new login button is a great result, and shows the benefit of following conventions – as login buttons are most typically located in the top right hand corner of a website.

How would you subscribe to the RSS feed?

Gizmodo - How would you subscribe to the RSS feed - old version

How would you subscribe to the RSS feed - old version


Gizmodo - How would you subscribe to the RSS feed - new version
How would you subscribe to the RSS feed - new version

How would you subscribe to the RSS feed - new version

On the old design, it takes users an average of 16.38 seconds to click, but they have a success rate of 100%. Obviously the time is longer than they might like (I would say due to the length of the page), but a 100% success rate is very good.

On the new design we have a shorter average click time of 12.67 seconds, but a terrible success rate of only 30%. If you look at the results above for the new design, you can see a large number (around 60%) of people clicking the flame sort of icon in the upper right corner. That actually links to the hot stories on the site, but the lack of understanding shows they need to make some sort of modifications to make this a lot more understandable to their users.

For a site like this where sources such as RSS are so important, they really need to do a lot more to pull it out and make it more visible to their users.

Your friend told you about a story called ‘Academics on why trolls troll’ – where would you find it?
Where trolls toll - old version

Where trolls toll - old version

Where trolls toll - old version

Where trolls toll - new version

Where trolls toll - new version

Where trolls toll - new version

For both the old and new version of the site we had the same success rate of 100%; users either went for the search box, or scrolled around the page until they found the article. Interestingly the average click times were quite divergent; on the old site, users clicked on average after 14.6 seconds. On the new site, the users took 21.3 seconds on average.

Of course a quick glance shows us that in the old design articles were more prominently feature, but Gizmodo needs to carefully consider what are important goals for their new design. If they make it a lot more difficult for users to find content on the site, they will eventually begin to turn to other sources.

Which interface do you prefer?

Preference Test - old and new design

Preference Test - new on the left, old on the right

As I said, preference tests are a useful gauge for your users feelings. This test showed almost 65% of respondents prefer the new design, 32% prefer old design, and 3 percent clicked either in the middle or skipped this question, showing a neutral vote. This is a pretty good result for the new design considering it hasn’t faired quite so well in the testing so far.

How would you follow Gizmodo on Twitter?

Follow on Twitter - old design

Follow on Twitter - old design

Follow on Twitter - new design

Follow on Twitter - new design

Follow on Twitter - new design

In this test, the old design has a success rate of 87% and an average click time of 10.4 seconds. Although the time could be improved, an average click time of 87% is pretty good on a long page like this.

The new design however shows the same problems we saw when looking for the RSS feed. Only 29% of people correctly clicked the subscribe button (which doesn’t even correctly link to the subscribe area on the about page) at the bottom of the page, with 42% clicking the flame icon on the upper right, and the rest of the clicks dispersed around the page.

If Gizmodo wants to push its social presence, it really needs to bring this information up the hierarchy. Even if it’s not concerned about it’s social presence, it needs to clarify the meaning of the hot stories button because this test has shown us a huge number of users have been very confused by this icon.

How would you search the Gizmodo site?

Search the site - old design

Search the site - old design

Search the site - new design

Search the site - new design

In this question the old design had an average click time of 5.15 seconds, and a success rate of 92% – you can see there were a couple of clicks on the ‘share a tip’ bar, and a couple further down the page. They should really consider the ‘share a tip’ bar design because it does look awfully like a search dialog box, and is in a common location for search bar.

The new design did better on this question with a 100% success rate, and an average click time of just 4.84 seconds. Really a very good result. One thing to mention though, when you click the search icon, the search box actually appears below the advertisement which is rather counter-intuitive. If they wanted to optimise this page more they could consider the position of the search bar.

How would you advertise on this site?

Since the ‘advertise’ text was so far down the old design, I’ve just looked at the new design for this question, you can see the results below:

Advertise on this site - new design

Advertise on this site - new design

There was a 91% success rate on this page, and an average click time of 11.1 seconds. Not too bad. I think this reflects the fact the many people expect to look to the bottom of a page to find certain information such as advertising details. Following conventions such as these is always a smart thing to do.

Interestingly, the 9% that clicked in the wrong locations were all clicking on different ads around the page.

Recommendations for the Gizmodo site:

 
Hopefully after reading through that you can see some of the flaws in the Gizmodo site. Based on this test, and my own observations there are a few simple changes I would suggest. These are changes I would make to the newer design:

  • Pull the links to social media sources further up the page
  • Pull the RSS link further up the page
  • Think of the value of links and icons at the top of the page; how many people use the hot stories link? Is the icon clear enough?
  • Rather than having a text talking about blog formatting and a funny icon to show the blog view, just use the text to change to the blog view; screen real estate at the top of the design is very valuable
  • Consider popping out a search box immediately below the search icon, or creating a separate search box entirely – look at analytics to see how many people use search
  • Try and follow website design conventions; in the areas where conventions are followed, users responded both much faster and much more accurately
  • Make sure the subscribe text leads to the right place

There are obviously more changes that could be made to improve the site, but these are some good starting points that could make an immediate difference to a users experience of the site.

Conclusion

 
Considering how popular this site is, I’m surprised to see how many problems have cropped up during our testing. Although by themselves, none of these issues are enough to make someone stop using the site, the more issues that crop up, the less enjoyable the user experience will be.

With sites like Gizmodo, the switching cost of changing sites isn’t awfully high, and there are a number of other sites competing in this space. I imagine it would be well and truly worthwhile for them to invest some time in making some simple usability improvements across their sites.

Final Score: 6/10

While Gizmodo has a huge amount of interesting content, there are a number of simple usability issues which prevent the site from reaching its full potential, and contribute to a less than perfect use experience.

What do you think of the Gizmodo site? What changes would you make? Are there any issues holding you back from using the site more? Be sure to let us know in the comments below. You can also run your own tests on IntuitionHQ.com and see how your results stack up.

Don’t forget to subscribe to our RSS feed, and follow us on Twitter and Facebook to share your thoughts on what we should test in the future, and to keep up with all the latest usability news.

 

Start user testing with a homepage healthcheck

Posted by Courtney Johnston on June 1st, 2011

Sometimes it feels like you only get a chance to do user testing as part of a development project, and you haven’t really managed to make it part of ‘business as usual’.

An easy place to start with making user testing a regular part of your work is with a six-monthly or yearly healthcheck (Steve Krug recommends monthly checks) for the parts of your site that have to do the most work (such as your homepage, important section pages, and your search and search results).

A homepage healthcheck is also a good place to start if you’re using an online usability testing tool like IntuitionHQ for the first time.

You’ll learn how to pose questions to elicit useful answers from testers, and also get information you can use to make both quick tweaks and longer term plans for changes to your homepage.

Test the basics

Could you find our 'Pricing' page?

Could you find our 'Pricing' page?

Ask yourself ‘What are the five most common things people do when they come to my homepage?‘ and then write tests around these tasks. You’re likely to come up with questions around things like:

  • Finding contact details
  • Going to news or job vacancies section
  • Finding event information
  • Finding pricing information
  • Getting to your most popular products/offering/tools (depending on what kind of website you run).

Now write test questions to uncover how well people can figure out how to start these tasks from your homepage. Remember not to influence your testers by tipping them off with keywords in your questions.

For example, if contact details on your website are available from a link titled ‘Contact Details’, don’t ask ‘Where would you click to to find our contact details?’. Instead, try something like ‘Where would you click to find our phone number?’.

Likewise, if you have job listings under a tab called ‘Vacancies’, don’t ask ‘Where would you click to find out about current vacancies?’. Instead, try putting testers into a scenario like ‘Imagine you’re interested in working at ‘company name’. Where would you click to find out whether there were any jobs available?’.

Test first impressions

What part of this page catches your eye?

What part of this page catches your eye?

Why not try being a bit more creative with your questions? You could start your test by asking people to ‘Click on the first thing that catches your eye on this page’ and get a feeling for what people focus on when they first visit your website.

Ask testers to ‘Click on what you think is the most important piece of information on this page’ to find out if your information hierarchy is working.

Test for speed

Test for speed - unsurprisingly, Google is pretty good at this

Test for speed - unsurprisingly, Google is pretty good at this

Hopefully, the people you’re testing will find your questions pretty easy. That’s a good thing – the whole job of your homepage is to get people to where they want to go as quickly as possible.

So as well as looking at how successfully people accomplish the tasks you set them, look at how quickly they accomplish them. IntuitionHQ records how long it takes people to answer questions on tests, and presents this as an average.

This means you can spot underlying problems. It’s great if 96% of people can successfully find your contact details; it’s not so great if it takes them nearly a minute to do so (imagine how frustrating that is for a person who just wants to call you).

Follow-up testing

Your first test will tell you one of two things:

  • Your homepage is working really well just as it is;
  • Or, there are some areas that people have problems with.

Either of these findings are great. The first means you can move on to testing the other parts of your site. The second means you can start thinking about how to make people’s experience on your homepage better.

Before you make changes, you could run another round of testing using the improved homepage and the original questions; that way, you’ll be able to compare previous performance to performance on the new design or wording.

Useful links

If you’re just getting started with user testing, you might find some of these articles helpful

Questions?

Do you have questions about getting started with usability testing? We’d love to help answer any questions you might have, and we’ve got lots of great resources we can direct you to as well.

We’d also love to hear your stories about your own experiences with usability testing, and how your homepage healthcheck worked out, so be sure to let us know in the comments below. Happy testing!

Don’t forget to subscribe to our RSS feed or follow us on Twitter or Facebook to keep up with all the latest news on website usability testing. Thanks!

 

Usability testing: What to test

Posted by Jacob Creech on May 24th, 2011

 
When you have decided that you want to usability test, one of the first things you have to consider is what you are going to usability test. Although this may sound like a very straightforward question, there are a number of facets you need to consider to make your test successful and to ensure you are testing the right things.

Of course, what you are trying to test can vary a lot depending on the product or service you are working on, but there are some simple rules that you can follow to ensure you get the best, most effective results you can from your testing process.

What to consider when usability testing:

What do users do on your site/service?

Do you know what your users do when they come to your site or are using your service? How do they get from place to place? What sort of information are they trying to find? Do they follow the path that you’d imagine?

There a number of different ways you can work this out, but one of the easiest is looking at your analytics data. Not using analytics? Check out Clicky (which has free and premium options) for a great, user friendly analytics experience.

Clicky analytics - what are users looking for?

Use your analytics data to see what's popular.

From your analytics data you can get an idea of where users are going, how they are getting there, and how long they spend in each place. If you discover all of your users are looking for your contact information, then you might want to make your contact information more prominent. If you find everyone is using your search box in order to find a certain your blog or your about page, you might consider making those areas more prominent.

Try this on for an exercise: ask a friend or family member to use your site or service, and see if they can complete some common tasks. If they can complete the tasks, did they use the method that you would have thought? If they didn’t complete it, what tripped them up? You’d be surprised at the huge array of different ways people complete seemingly simple or obvious tasks.

Chrome - searching for google

Google-ing for Google. Yes, it happens.

To this day, I never cease to be astounded by the amount of people who type ‘Google’ into the Chrome address bar, or even into the Firefox ‘Search’ bar. You will find the same astounding things by testing on your own site.

Once you’ve seen how your users really use your site, and the sort of information that they are looking for, you can then think of testing questions that will cover these points, and using your test results, you can streamline the process to make it as efficient and enjoyable as possible. You will be surprised how large a difference even a few small changes can make to your users.

What do you want users to achieve?

What are you goals for your site or service? What do you want users to do when they arrive? What do you want them to achieve?

Goal: sign up for IntuitionHQ

We want users to sign up to IntuitionHQ. Hint.

Write down a list of goals, and think of all the things that are really important for your site or service. It might be subscribing to your RSS feed, it might be finding your blog content, it might be a link to your LinkedIn profile. Whatever it is, however many points there are, write them all down and then go ahead and test them.

If your users can manage to do all these simple tasks in good time, then you don’t need to worry, but in my experience 99.9% of sites and have some sort of tweaks they could make that would improve the overall experience of using the site.

Google vs Bing Usability Test

Look at the average click time and the location of clicks. A clear win for Google.

In our recent comparison of the Google and Bing search engines (the UI, not the actual search results) we found a number of small tweaks that even major search engines like these could make. It may not sound like much, but each second you shave off, or each time you make a small tweak that makes things easier to find or understand, a user is that much more likely to return to your site, or to use your app.

Of course, the more that users can achieve your goals, that happier you should be as well, so it’s definitely worth testing to ensure that this is happening.

What are the important features of the site/app?

As well as the important goals that you’d like people achieve, there are also probably a number of things you’d like users to notice on your site. Can they even tell what the site is about when the arrive there? Can they find your pricing page? Can they find the sign up or sign in button?

Easy website usability testing - IntuitionHQ

We want people to remember our tagline, and to have easy access to important pages

You know best what are the most important features, that along with your previously mentioned goals that you would like users to notice. If even a small percentage of people can’t find the pricing page, this might prevent them from signing up and have an impact on your bottom line. There are many apps and services that I’ve come across that seem to be lacking a pricing page, or the link to it is impossible to find – it’s enough to prevent me signing up, and I’m sure I’m not the only one. A quick test will show just how visible your pricing page is.

What about the message you want users to take away after viewing your site? Why not ask them what element of your site stands out the most? When users come to the IntuitionHQ site, we want them to remember that we are a quick, easy usability testing service. What do you want your users to remember? Do a test to make sure the message they are taking away is what you want it to be.

What next?

 
Once you’ve considered all the important points of your site, what the users are looking for, and what you’d like them to achieve, make a list and think of questions you can use to test each point. If you’ve got a couple of different ideas you’d like to test against each other, why not run an A/B test? Want to see what your users prefer? A preference test will help you out there.

Our experience shows that tests with 15 or less questions are much more effective and less likely to have a drop off in respondents. Any more than 15 questions and users start to get distracted. Think of the most important points and try and fit them all into one test. If you have many more than 15 points, run a second test.

Think carefully about the wording of your questions, and keep an eye out for our upcoming post on writing great questions for usability tests. Try not to lead your users in a certain direction with the way in which you word your questions, or your results will lose some validity.

Hopefully this post gives you some ideas about what you should be testing and why it’s important. If you have any questions, please ask in the comments below and we’ll do our best to help you out. Happy testing.

Don’t forget to subscribe to our RSS feed for more great usability testing tips and advice, as well as our upcoming article on how to write great questions for usability testing. Thanks for dropping by!

 

A/B and Preference Testing for Usability

Posted by Jacob Creech on April 14th, 2011

One of the most frequently asked questions we receive here at IntuitionHQ is from users who are interested in A/B testing, preference testing and usability, but who don’t know what kind of test is right for them, what kind of results they could expect with each kind of test, or even how to get started with usability testing in the first place.

We addressed the last issue in a previous post, ‘How to get started with usability with usability testing in seven simple steps‘, but the first two questions can still be quite confusing. It’s worth addressing the differences between these two types of tests, because both can be very useful for gathering certain types of information. Read on to read what kind of information each test is best for, and help learn what is the best kind of test for you.

A/B and Preference Testing for Usability

A/B Usability Testing

A/B Testing is a very popular method for testing variations in live sites; you can have two different variations of a text, button or whatever and find which one works best. This is achieved by sending roughly 50% of the sites traffic to the different vartiations (either A or B – not both) and seeing which one works best. Two useful options for this are Google Website Optimizer and Visual Website Optimiser – both of these tools are very handy for testing on live sites.

A/B usability testing is a very similar concept. Whatever ideas you have to test, whatever variations you can think of, you just upload them, set a task, and see which one works best. The great thing about this is you don’t need to make changes to your live site, and it’s easy to add multiple tasks to test different variations.

A/B usability testing is great for testing alternatives, for example navigation structure, button designs, button locations – basically anything where even a small variation could make a difference to the end user and the usability of your site, service or product.

Preference testing

Preference testing is when two images/wireframes/screenshots are shown side by side, and users can make a choice on which one they prefer based on the test criteria that you set for them – generally along the lines of a ‘which design do you prefer’ kind of basis.

Preference testing is really useful for testing a range of different things: it can help you to better understand conventions in design (as shown in our ‘The User Experience and Psychology of Colour’ article on Spyre Studios), for looking at preferences across cultures, for understanding how small differences can affect your users and so on. As with A/B usability tests, there is a huge range of different things you could test in this way, and ways that your site could benefit from this kind of testing.

Some examples: A/B and Preference tests

Of course, it’s easy to talk about these different testing methods, but the most effective way to understand what they really mean is to see them in action:

A/B testing

One great example of this is a test we did comparing Bing and Google. Obviously the objectives of the two sites is very similar, but the way they are structured is quite different. We wanted to find which one worked better – not about search results, but in terms of usability and optimisation.

Bing heatmap results

We looked at some of the more common goals that both of the search engines would be trying to achieve. In the example above, we’ve used finding advertising as part of our test – advertising being a crucial part of the business for any search engine. In the first part of the test, users are either directed to either a screenshot of Bing or Google, and try and find the link to advertise with that provider.

A/B test heatmap results - Google vs Bing

When looking at the results (shown above – click to enlarge) we can see that not only is the average click time for Google much lower in this example (just over 7 seconds to almost 11 seconds), but also that the success rate of users finding the correct button is much higher (77% for Google versus 65% to Bing).

What this means is that Bing should really focus more on the location of this button, and trying to make it more visible. Obviously the easier, more straight forward it is to advertise, the more people will advertise. Even a 1 or 2 percent bump to their advertising spend would mean significant revenue for the company.

If you are interested you can also take the test or view the results.

Preference testing

We recently wrote an article over at Spyre Studios talking about The User Experience and Psychology of Colour. It was based on the colour of labels they use at Clicky, a web analytics tool – where they often use red to display confirmation/positive messages as well as for failure/warning messages.

Colours and Psychology - heatmap testing

What our testing showed with quite an overwhelming majority is that most people (88% in our tests) associate red with failure.

Colours and Psychology - heatmap testing

It also showed that red was the colour that stands out the most, although as I argued in my post, this doesn’t make it a good reason to use it for all types of messages. Certainly if it’s overused, it will stop having much effectiveness.

Colours and Psychology - heatmap testing - red warning text

And indeed, what we can see for the above image is that most users think red makes the most sense for warning text – a huge 97% of users chose red for warning text. Another question in the test had 74% of users choosing green for their success message. You can take the whole test here, or view the results here.

While these might be the results you’d expect in this example, there are many times where based on culture, gender, age or a number of other factors that people would choose different options in this type of testing. If you want to make your website as effective as possible for your audience, you should really perform this type of testing.

What next?

So now you (hopefully) know the difference between A/B usability testing and Preference testing, you can work out which is the best option for you next time you are getting ready to test.

This brings us to the last point, which is frequency of testing; obviously in the earlier stages of design and development there is more testing to be done, but a key point to remember is constant, consistent usability testing to monitor for small changes in the internet psyche, and to adjust to the ever changing network. If you ever have an inspiration on how you could improve your site, test it and see how it works in reality. It’s never a bad time to test.

As I’ve quoted time and time again, ‘Build it and they will come; build it well and they will come back’. The better, more usable, more enjoyable your website, the better it will be for you and your users, and the more successful you and your site will be in the long run. How can you afford not to test? Why not head over to our homepage and sign up for your free IntuitionHQ account today. It’s always a good time to get started with testing.

Do you have questions or comments about this article, or usability testing in general? Feel free to ask – we love to help. And don’t forget to subscribe to our RSS feed to keep up to date with all the latest news in usability.

 

Users’ Wants Versus Clients’ Needs

Posted by Jacob Creech on September 14th, 2010

There is an interesting article over at UXBooth.com discussing those times when you are in a client meeting, awaiting final approval, and all of a sudden the big wig comes in, scrunched up piece of paper in hand with new directions on the site for.
more »