UX/CX Research for
Newsday Pt. 2 - Offboarding

Newsday is a daily print and online news publisher serving the New York City borough of Queens and Long Island.

I worked with the House of Kaizen agency on this project as a Mixed Methods Researcher using live session recordings to analyze behavior patterns, find friction, analyze re-engagement with existing value propositions, and make recommendations on optimizing the offboarding process for unsubscribers who have made up their mind about leaving.

Overview

Goals

OVERVIEW - Session Recordings

50 unique recordings were analyzed over six days

Overview

Methods

Overview - Semiotic Mouse pattern Analysis

Semiotic analysis revealed insight into the user's behavior and cognitive load

Mouse pattern analysis can provide a more complete picture of user behavior under high cognitive loads like decision making tasks. Metrics like random and straight movements, mouse hovers, etc., can be associated with perceived ease use, perceived usefulness, self-efficacy, willingness to learn or risk-perception.

Hesitation patterns

Hesitation patterns occur when users are unsure where to click or which option to choose. There are two kinds of hesitation patterns: multi-target, and single-target.

Random patterns (Wild mouse)

Random mouse movement is a reaction to either slow UI elements, or high cognitive loads. In both cases, it manifests the same way: rapid back-and-forth or circular movements without any functional intent and with no relation to targets on the page.

Overview

Objectives

Process - Affinity mapping exercises

Affinity maps helped track behavior for quantitative analysis

Each of the 50 flows were broken down into six distinct maps – give or take, depending on the individual users’ progress. 

Finally, a mega map was created.

INSIGHTS - STEP 2: "Here's what will happen"

Most users spent up to eight seconds on this step, which is an average of three seconds longer than step one.

94% of the sessions that began the flow made it this far. Longer time on card and mouse movement behavior indicate users are consuming the content. 

About 30% of impressions displayed reading patterns, including resting or fixed, vertical and horizontal. This could indicate interest in the value proposition messaging on the card.

27.6% displayed hesitation patterns on “Continue to cancel”, possibly signaling anxiety, confusion, or a high cognitive load

12.7% displayed hesitation patterns on “I’ve changed my mind”, possibly mistaking it for the “Continue” CTA in some instances.

INSIGHTS - STEP 2: "Here's what will happen"

Opportunity to immediately engage with users who have selected “I’ve changed my mind”

Users who select the “I’ve changed my mind” CTA land on an end of journey that does not encourage any further action to be taken.

Optimizing this experience would be beneficial for users who in fact did change their mind as well as users who accidentally selected the wrong CTA.

This is another touchpoint to further enforce the value of Newsday and make subscribers feel good about their decision!

INSIGHTS - STEP 3: “Please tell us why you are canceling”

User behavior indicates impatience is beginning to set in at this step

Most users spent up to nine seconds on this step. 27.9% displayed hesitation patterns on the survey or individual survey options — which is not unusual, because they are being asked to make a choice.

Opportunity to switch radio buttons for checkboxes and obtain more outgoing data:

  • 13.6% tried to provide multiple answers on the survey but the format doesn’t allow it. 

Some users may be rushing, feeling impatient, or dealing with slow UI elements:

  • 6.8%  tapped or clicked a target but missed, suggesting that they may have been rushing, anxious or frustrated.

INSIGHTS - STEP 3: “Please tell us why you are canceling”

Reduce uncertainty by re-thinking CTA hierarchy, which could be confusing for some users

Some users are mistaking the blue “I’ve changed my mind” CTA for the way forward. 

 9% selected “I’ve changed my mind” by mistake and had to re-start the flow again.

READING PATTERNS

  • 15.9% displayed a resting or fixed reading patterns.
  • 6.8% displayed a vertical reading pattern over survey options.
  • 2.3% displayed a horizontal reading pattern on “feedback”
.
  • 2.3% displayed a horizontal reading pattern over “please tell us”
.

18.8% displayed hesitation patterns on or near “I’ve changed my mind” indicating difficulty or doubt.  

2.7% almost selected the “I’ve changed my mind” call to action by mistake.

2.3% displayed a multi-target pattern between “I’ve changed my mind” and “Continue to cancel”.

6.8% displayed hesitation patterns on or near “Continue to cancel”. 

OTHER PATTERNS

  • 2.3% displayed wild mouse or random mouse patterns
  • 2.3% clicked randomly

INSIGHTS - STEP 4: “Give us another chance?”

Investigate an error stopping the user from progressing in their journey after accepting the offer

Overall errors were sparse. However, one user dropped off because of an error after accepting the deal.

HESITATION PATTERNS

  • 9.3% displayed hesitation patterns on or near “Accept Offer”
  • 20.9% displayed hesitation patterns on or near “Continue to cancel”.
  • 2.3% displayed a multi-target pattern between “Accept offer” and “Continue to cancel”.

2.3% displayed a horizontal reading pattern on the T&C fine print.

16.2% displayed a resting or fixed reading pattern.

11.6 % displayed wild mouse or random mouse patterns indicating possible  indecision, a high cognitive load, or anxiety

Key Takeaways

Summary of insights

STEP 1

  • Most users spent about 1-5 seconds on this step, typically rushing through it. 
  • 96% of the sessions began on this step.
  • 4% of the sessions skipped this step entirely.
  • 6.25% got stuck or dropped off before continuing to the next step.
  • 10.4% of the users went through this step more than once, because of u-turns, or wrong turns, further along in the flow.

STEP 2

  • Most users spent about 1-8 seconds on this step. 
  • 27.6% of users on this step displayed hesitation patterns on “Continue to cancel”
.
  • 12.7% of users on this step displayed hesitation patterns on “I’ve changed my mind”.
  • 8.5% of users on this step displayed wild mouse or random mouse patterns, possibly signaling anxiety, indecisiveness or difficulty.
  • Users who change their goal and select one of the “I’ve changed my mind” CTA’s landed on an end of journey.

STEP 3

  • Most users spent between 3 and 9 seconds on this step.
  • 88% of the sessions made it this far.

    Of those:
    • 95.5% moved from this step towards their goal. 9% passed through this step twice because of a wrong turn
    • 9% selected the “I’ve changed my mind” CTA then restarted the flow, a behavior that suggests they may have done it by mistake. 
    • 2.3% got stuck or dropped off.
    • 13.6% tried to provide multiple answers on the survey but the format doesn’t allow it.

STEP 4

  • Most users spent between 2 and 8 seconds on this step.
  • 20.9% displayed hesitation patterns on or near “Continue to cancel”
.
  • 11.6 % displayed wild mouse or random mouse patterns indicating possible  indecision, a high cognitive load, or anxiety for some.
  • 9.3% displayed hesitation patterns on or near “Accept offer”

  • The majority of users are not swayed by the deal. 4.6% changed their goal and only 2.3% were successful on their new journey.

STEP 5

  • Most users only spent between 2 and 4 seconds on this step. Many rushed though. Some users may not have realized this was the final step in the cancellation. Opportunity to let the user know this is the final step.
  • 26.8% displayed hesitation patterns on or near “Continue to cancel”
.

STEP 6

  • Most users only spent about 5 seconds on this step.
  • Most users took between 20 and 40 seconds to get to the final step. A small few blazed through in under 10 seconds. 
  • 25.6% displayed hesitation patterns on “Return to Newsday”.
  • 12.8%  displayed wild mouse or random mouse patterns, indicating higher than normal anxiety, indecision or confusion. Possibly because they did not realize the last step was the final one before cancelation.
  • 36% of users dropped off at the end of the flow, deciding to end their journey with Newsday. 
  • 64% remained engaged, clicking “Return to Newsday”.

Want to work together?

Adam Inglis – UX/CX Design Researcher and Consultant

adaminglis.design@gmail.com

Adam
Inglis