Aude Lerner
UX writer | Portfolio |

Handshake App Content Testing and Performance

Dataset Analysis, User insights and Content and Copy Recommendations

Tackling the Handshake App Content performance


The handshake app (fictitonal) is an app for for both freelancers and business owners to manage their project together.

For the Handshake project, a project I worked on to get certified in Content Research and Testing, I had to get user insights and come up with content and copy recommendations to improve the overall user experience and enhance the app's performance.

This involved processing the dataset handed over by the UX research team as follows:

  1. Analyse quantitative data
  2. Turn qualitative data into quantitative data
  3. Derive insights from data analysis
  4. Draw conclusions

Business challenge

To solve usability problems through content, copy and design improvements in order to meet business goals, i.e., to increase the total number of sign-ups for the Handshake app and keep users moving along the flow.

Tracked Metrics and factors used for defining usability:

  • Completion Rate
  • Drop-off Rate
  • Ease of Use
  • Confidence

User Testing

The product team conducted a series of usability tests for the first version of the Handshake app.

The purpose of the tests was to get insights on how the “Business Owner” users perceived information within the app and how successful they were at completing tasks on the app.

Each user performed the tasks on a mobile device.

Test subjects

Number of test persons: 5

  • Age : 21- 41
  • “Business Owners” User type hiring freelancers
  • 2 Male
  • 3 Female
  • Industry: Fashion, Copywriting, Landscaping, Accounting, Beauty

Tasks

  1. Create an account
  2. Set up a project
  3. Navigation tabs
  4. Make payment
  5. Payment dialog

Methodology

Qualitative research with think aloud protocol, follow-up questions and comments.

The performance measured by:

  • Successful task completion (Y/N)
  • Ease of use (Self-assessed on 1-10 scale)
  • Confidence (Researcher assessed on 1-10 scale)

Overview of individual user performance

Overview of individual user performance

Graph Data Interpretation

The data on the graph should be interpreted with caution. We noted a disconnect between how some users feel doing the tasks and the result or the actual amount of time spent on the task.

Discrepancy

User 3 had a high level of confidence for nearly all the tasks and found the copy easy to understand, but failed to achieve some of the tasks.

User 5 showed a weak level of confidence although she found the copy easy to understand.

➜ Because a task was completed, this does not necessarily mean that the process of completion was a smooth one.

➜ Being diligent in doing a task does not mean that the user succeeded in completing it.

Incomplete Data

Moreover, not all the participants completed all the tasks - due to late arrival, accidentally skipping tasks - so the rates are to be considered with an error margin.

Findings and Recommendations

Key Findings

  • All users failed at completing the Sign-up process
  • 75% of users had no problems Setting up a Project
  • None of the users completed the Payment Confirmation step

Recommendations

Critical

> Improve critical points in the flow to improve conversion rate for sign-up and to lower drop-off rate for payment flow. Use concise and clear wording for CTAs, A/B test for variants.

Redesign Sign-up flow and use preferred terms.

Medium

> Fine-tune design, content and copy for Set up a Project, Navigation Tabs and Make a Payment tasks.

- A/B test for tab labels.

-Use more accurate wording, provide additional text for better user guidance.

Content research and performance for the Handshake app

Create Account Task: Findings

The users didn’t know whether they had to click on Sign up or Sign in since both were displayed on the same line or if they first had to go to the bottom part as a “first user” to create their account.

> Most users did not know what to do or what to expect by clicking on either of the tabs and what was the difference between Sign in, Sign up and First time user.

>  4 users out of 5 were confused by the terms used and the way the info was displayed

Recommendations for Create Account

Investigate how to improve Create Account screen so that users can complete the task effortlessly. Look into both copy and design.

Copy: use consistent words as to not confuse the users, make a list of prohibited terms. Run a user survey to identify which terms make sense, create a terms bank and document terms in a style guide for future use.

Design: Simplify and review the lay out. Cut any redundant flow. Clearly separate Sign up from Sign in.

Payment confirmation dialog: Findings

All the users were confused by the "Cancel" and "I’m good" CTAs because of the lack of contextual information. The trust of the users happens to be low when clicking on either of the buttons.

3 users out of 5 found the “I am good to go” weird in wording.

Some worried that the Cancel button would close the dialog box but not cancel the payment. 

>None of the users completed the task

Recommendations for payment confirmation dialog

Copy

Provide contextual info to reassure the users about what they can expect next when clicking on the CTA buttons. Avoid vague CTAs that don’t mean anything.

Watch for the right tone of voice in relation to the nature of the task performed. A relaxed tone is not appropriate for a financial transaction, it does not foster trust in users.

>Perform A/B Testing with B variant: "Cancel Payment" "Confirm Payment" against version A "Cancel" "I'm good".

Ease of Use per Task Rating

Ease of Use per Task Rating

Ease of Use for Navigation Tabs

3/4 of the users felt that they could need more info under the respective tabs although they managed to complete the task.

They expressed their wish to have more detailed information as to what kind of data it is possible to enter under the respective fields for the “Time” and “Pay” tabs to be able track their budget and quickly get an overview of their situation.

Recommendations

  • Fine tune content and copy for Time and Pay Tab
  • Further Investigate users’ needs and add missing info according to findings. Provide more context for the user to immediately understand the function of the Pay tab
  • Define what is the best word choice “Pay” vs “Payment”. Run survey to find out what is the preferred term by users.
  • Break down the content and organise it better visually for enhanced readability and clarity
  • Use categories, sections, headings, sub headings.
  • Improve the information architecture, do card sorting exercice to find out what structure makes most sense to the users.

Insights lead to further testing

The data synthesis performed enabled me to detect patterns in the behaviour of the users and their emotional responses while interacting with the app. The most severe pain points and their root cause could be identified. These insights build the stepping stone for recommendations on content/design updates.

The next step would be to do further testing to come up with a new design, fix weak spots and retest the app again to validate those choices.