What is Usability Evaluation?
Usability evaluation is a term to describe the ways in which we investigate the usability of a design or project. Allowing users to interact with a version of the product in a controlled environment in order to assess the efficiency, effectiveness and desirability of that product to complete given tasks.
About the project
Total Gaming is a platform for gamers to share all of the content they are interested in from their gaming console or mobile device. It is a part of the interaction design module.
We started each session by reading participants an adapted version of Steve Krug’s Usability Test Script.
Usability Test Script – Total Gaming Application
Tip
Ask open-ended questions:
How would you…?
What do you understand this to be…?
Is this what you expected?
Usability Evaluation Summary
Participant #1 (P1)
We asked Participant #1 (P1) to search reviews for Overwatch for PC on the Total Gaming mobile app through a face-to-face moderated interview. P1 was passionate about gaming a long time ago and knows the ecosystem very well, but said they don’t have a lot of time to play now.
We asked P1 to think aloud whilst completing the assigned tasks and then we followed up with a debrief interview on the main challenges or functionalities they would have expected to find. P1 went straight to the search box without any hesitation and then immediately into reviews. P1 got confused about the radio button on the left side, asking what it would activate if clicked.
P1 liked the way in which users reviews were presented because they immediately show if the game is worth buying or not, but P1 was not clear about the meaning of the ‘Points’ in the user page review. What is the scale? Is 3000 a good score? Also, P1 asked if the score was related to the user only or to the game as well. After that, P1 asked why user points were not present on the review page too.
The app has been confirmed to have a good level of learnability – the user completed the task easily, even though the ‘Points’ caused some confusion. The user also confirmed the importance of having a multi-platform system where they could find unbiased reviews created by experienced and trusted users.
Participant #2 (P2)
User testing for Participant #2 (P2) was face-to-face, moderated and followed thinkaloud protocol. When initially viewing the console home page, P2 suggested that having a trial of the games would be a nice feature. They were enthusiastic about the layout and visual design of the screen, and commented that it was useful to have the controller icon and instructions showing how to navigate through the system. They easily found the fan art section from the home page. When probed to explain what they thought clicking the ‘+’ icon beside the heading would do, P2 expressed confusion and guessed that it would expand into a submenu.
P2 also tested the Total Gaming mobile application. They liked that the software was available cross-platform, and said they would use both versions of the application in different contexts. P2 liked the close similarity in terms of design between mobile and console. P2 was not immediately clear what they were looking at on **screen 4**, but understood when it was explained to them. Again, P2 didn’t understand that the ‘+’ icon was to add content. They believed it would expand the review when clicked on. P2 liked the flow of the game reviews screens, commenting that it was useful to determine what level of detail they viewed the review in. P2 wasn’t sure what the thumb icon beside the user’s name (**screen 7**) meant. When they expanded the review and saw the thumbs up and down icons appear, they understood what these icons would do but remained confused if they somehow related to the previously mentioned thumbs up icon. When the facilitator explained that the top thumbs up icon was a “Top Contributor” badge, P2 commented that this was a nice feature, but that the icon should be changed to differ in appearance from the upvote/downvote icons. Overall, P2 expressed that this was a system they could see themselves benefiting from with the aforementioned minor changes.
Participant #3 (P3)
We conducted a remote evaluation with Participant #3 (P3), providing the user with a link to clickable wireframes on her own smartphone. P3 was a casual gamer who played PC games 10 hours a week. We asked P3 to complete three findability tasks. In the first task, P3 was able to search for Overwatch PC game reviews but suggested that there should be an option in the menu for reviews. In the second task, on the review page for Overwatch PC, P3 said that it is not necessary to add a video alongside the review because the video would just discuss what the comment already says. In the final task, P3 said it is good to have the option to click through to John’s profile so they can see how reliable of a user he is. P3’s overall experience using the Total Gaming mobile application was described as positive.
It is a qualitative feedback – gives us the how and the why.
Resources
I found this excellent YouTube video by Steve Krug. This Youtube video is a great usability demonstration.
Another excellent YouTube video Usability Testing with kids
=