Overview
This report collates findings from three rounds of CX research conducted as part of the Authentication Uplift project and provides a comparison on models tested.
Round 1 was conducted in September of 2022 and benchmarked the existing ‘Redirect with One Time Password (OTP)’ model. Round 2 research focussed on ‘App/Web-to-App with Biometric’ and ran in November of 2022. Round 3 research focussed on ‘Decoupled with QR Code’ and ran in March of 2023.
The purpose of the research was to identify consumer experience considerations to support and inform an expanded approach to Consumer Data Right (CDR) authentication. The objective of uplifting authentication in the CDR is to give consumers more choice and freedom when authenticating themselves with DHs (data holders), while maintaining financial grade security.
In total, over 150 consumers participated across the three rounds of research; which involved 90-minute 1:1 interview sessions and 30-minute unmoderated prototype tests. Various prototypes were used to facilitate discussion and generate insights in relation to the authentication models shown, as well as to authentication more generally.
More detail on context can be found in each of the research reports, and in Noting Paper 280 – The CX of Authentication Uplift.
Research goals
This research project aimed to:
- Identify appropriate authentication models to support in the CDR;
- Provide CX input to the authentication framework to assess incoming/supported models;
- Strike a balance between security, consumer experience and value delivery;
- Help organisations provide intuitive, informed, trustworthy consent experiences with positive outcomes.
Research Objectives
- Understand current consumer behaviours, pain points and needs regarding authentication;
- Identify appropriate consumer experience criteria and metrics to assess authentication models;
- Inform the development and proposal of new standards, and/or the revision of existing standards;
- Identify appropriate models to be considered for adoption that are interoperable, flexible and adaptable;
- Uplift authentication standards to offer improved experience, choice, convenience, inclusivity and security as well as alignment to consumers' existing digital experiences;
- Understand how consumer behaviour/attitude may shift for different use cases (e.g. banking vs energy) using the same authentication method;
- Explore the impacts of different elements and mechanisms.
Hypotheses
Standard hypotheses tested over all 3 rounds:
In addition to the above 5 hypotheses, Round 3 research also looked to validate the following:
Research approach
In order to meet research objective #7 “Explore the impacts of different elements and mechanisms” the following 4 components of authentication were explored:
- Channel: This is the channel where authentication is performed. For example: mobile, desktop, kiosk etc.
- Modality: Modalities are the inputs used for authentication. For example: Biometric, Pin code etc.
- Authentication method: This is the method by which an authentication is performed. Out of many factors of authentication methods, these 3 are mostly recognised:
- Knowledge-based: Something the user knows, such as a password or the answer to a security question
- Inherence-based: Something that the user is, as represented by a fingerprint or iris scan
- Possession-based: Something the user possesses such as a one-time password generator, certificate, or smart card
- Notification method: This is the different ways a user is alerted about the authentication requirement. For example: Push notification, Email notification etc.
These elements were tested in various combinations across each round.
Channel | Modality | Notification method | Authentication method | Authentication method |
App to browser | One time password | SMS | Possession based | Something the user knows (Customer ID)
Something the user has (Phone/OTP) |
Channel | Modality | Notification method | Authentication method | Authentication method |
App-to-App | Biometric only | Push | Inherence | Something the user is (FaceID) |
App-to-App | Biometric + OTP (step-up) | SMS | Inherence + possession | Something the user is (FaceID)
Something the user has (Phone/OTP) |
Browser-to-App | Biometric only | Push | Inherence | Something the user is (FaceID) |
Browser-to-App | Biometric + PIN (back-to-back) | Push | Inherence + knowledge | Something the user is (FaceID)
Something the user knows (PIN) |
Channel | Modality | Notification method | Authentication method | Authentication method |
Decoupled:
Browser-to-Browser | One Time Password | SMS | Possession based | Something the user Knows (Customer ID) |
Decoupled:
Browser-to-App | Biometric + PIN Code | Push | Inherence + knowledge | Something the user has (Phone/OTP) |
Models tested
Round 1: Redirect with One Time Password
Currently, Redirect with One Time Password is the only model supported by the CDR. It involves redirecting the user to their DH’s website in a new tab on their browser. There, they enter their Customer ID and a one-time password (OTP) is generated and sent to the user via a different communication channel (such as email or SMS).
- Getting indicative interest rates for a car loan through a fictional non-bank lender called “Lendify” (ADR). The participant was told they bank with a real-world DH.
Round 2: App/Web-to-App with Biometric
App/Web-to-App is an authentication model where a user is redirected from either their ADR mobile application or mobile browser to their DH app. It involves a redirection from the ADR app or browser to a new browser, and then the DH app is triggered to open, where authentication takes place.
The research team developed 2 use cases that would be tested across 4 different user flows. The use cases included:
- Getting indicative interest rates for a car loan through a fictional non-bank lender called “Lendify” (ADR). The participant was told they bank with a real-world DH.
- Comparing telco plans from various providers using a fictional comparator service called “TelCompare” (ADR) to get a better deal on home internet. The participant was told their provider was a real-world telco DH.
These two use cases were each then extrapolated into two user flows, the first saw the ADR experience on a website (”Browser-to-App”), and the second had the ADR experience in a mobile app (”App-to-App”). This resulted in a total of four prototypes which were tested with research participants. These have been broken down below:
NB: All of the following flows involve a user authenticating in their Data Holder’s app, and assume the DH app has been pre-installed and previously used on the device.
App-to-App
Banking: User gets indicative interest rates using the Lendify app and shares data from a real-world DH.
Telco: User compares their home internet plan with more competitive telco providers on the market, using the TelCompare app.
Browser-to-App
Banking: User gets indicative interest rates using the Lendify website and shares data from a real-world DH.
Telco: User compares their home internet plan with more competitive telco providers on the market, using the TelCompare website.
Use case 1 (Banking)
Use case 2 (Telco)
Round 3: Decoupled with QR Code
Decoupled authentication requires the authentication of the user (or ‘challenge’, such as a PIN, password, biometric) to occur outside of the service/channel being accessed. This method verifies the user’s identity and authenticates the transaction via a separate channel — for example, a push notification to their banking app or via an email.
Round 3 research also tested fall-back methods. Fall-back (or waterfall) authentication is a mechanism that allows for an alternative authentication method/s to be used if the primary authentication method fails. This can be useful in decoupled authentication scenarios where the primary authentication method (app) is unavailable and a fall-back (browser) is required to complete the authentication and authorisation process.
The research team developed 2 use cases that would be tested across 2 flows. The use cases included:
- Getting indicative interest rates for a car loan through a fictional non-bank lender called “Lendify” (ADR). The participant was told they bank with a real-world DH.
- Comparing energy plans from various providers using a fictional comparator service called “Switch” (ADR) to get a better deal on energy. The participant was told their provider was a real world energy DH.
These use cases were tested across two prototypes. The first prototype tested a decoupled scenario where a participant began their journey on Lendify’s desktop website and scanned a QR code using a mobile phone which opened the DH’s app installed on the mobile device. They then authenticated and authorised in-app on the mobile device and were then prompted to return to their desktop browser to complete the journey.
In the second prototype, the participant began the journey on Switch’s desktop website, and scanned a QR code using a mobile phone. In this use case, no energy app was available on the device, so the fall-back was triggered. The browser was automatically opened with the DH’s log in page, where a participant authenticated with an OTP before returning to the desktop.
Use case 1
Use case 2
Findings
Several recurring themes were identified and observed throughout all rounds of research. These recurring themes are significant to the overall research findings and offer valuable insights to the research project as a whole.
- Friction is multifaceted
The research found the principle of friction to be multifaceted, with factors manifesting in various ways; friction can occur both online and offline. Online friction can include extra authentication factors, and offline friction could be the requirement to switch between devices, for example. Friction can be viewed by participants as negatively or positively impacting on an authentication experience, i.e. there are ‘positive’ or ‘negative’ levels of friction in a given flow. One may hypothesise that higher levels of online friction create more frustrating experiences for users, however the research does not support this. While some participants experienced frustrations when accessing devices (such as, to receive one time passwords or access an app), they generally appreciated lengthier processes when accessing sensitive data.
- Users look for, and rely on, visual trust markers to assess risk
- Checking URLs for suffixes such as “.gov.au” or “.com.au”, or assessing whether it compares to a known website for a brand to ensure legitimacy;
- The presence of a padlock icon in the URL bar as well as a ‘HTTPS’ connection;
- Looking for SSL certificates and security badges on websites;
- Pixel-perfect user interfaces which match user expectations of formatting, such as colour palette, typography, branding;
- Correct spelling and use of grammar;
- No slow loading time or suspicious redirections;
- Corporate information such as ABNs or phone numbers;
- Apps downloaded from trusted sites such as the App Store or Google Play;
- Assessing links in SMS text messages which have come through, including when the SMS has come through in the text-thread from their bank.
Consumer participants across all age demographics were conscious of the risks involved with using the internet and implement practices and habits to ensure their safety online. The research found participants heavily relied upon visual cues to determine whether a platform was trustworthy. Each research round saw an uptick in participant awareness of the potential for data breaches, and an increased understanding of scams. This may be attributed to the increase in highly publicised data breaches. Those who had been impacted by previous security breaches are proactive in their approach to online safety and actively seek out information on how to protect themselves.
This list is not exhaustive. We note that many of the trust markers listed rely on visual assessment. Consideration is being taken to better understand how those with specific accessibility needs assess trustworthiness.
- Extra authentication factors are appreciated
Across the board, consumer participants appreciated extra authentication factors even when they were not expected. Although two or more factors were expected for high-risk scenarios such as banking or health related data, participants also appreciated extra factors for actions they deemed as slightly risky. Even when a participant did not expect a second factor, they did not feel negatively toward the increased level of friction. On the contrary, participants perceived the extra layers of security as the brand or corporation’s effort to prioritise consumer privacy and data safety. Implementing extra factors provided participants with a sense of security and comfort. Research indicated that the extra factors or increased friction should be in context and relevant to the use case. A low-risk use case such as social media log in does not warrant multi-factor authentication (MFA).
- Meeting consumer expectations helps build trust
The research highlighted participant opinions on the importance of corporate responsibility in order to build trust. Participants noted they generally only create accounts out of necessity and believe more needs to be done by businesses to protect customer data. Consumer participants expect businesses to treat customer data as securely as possible, remain up to date with cybersecurity best practices to prevent hacking, never share their data with third parties, direct adequate funding to building strong back-end systems and hire talented teams. Interestingly, participants inherently placed more trust in larger and more established brands, though they recognised that their data is not guaranteed safety. Participants cited Optus, Medibank and Latitude as examples of companies whose recent data breaches have shaken consumer trust.
In order to assist consumers feel more in control, DHs should regularly communicate with their customers about data security and methods to keep accounts safe, as well as swiftly advising of any data breaches or any compromises to data.
- Step-up authentication is perceived as the norm
The research found that consumer participants expected authentication to adapt and become more rigorous as the sensitivity of their data increased, as this is what occurs in their present digital experiences and matches their mental models. Participants were familiar with risk-based step-up authentication because it is common in industries such as banking. Participants generally had a decent understanding of the requirements of step-up authentication, and the friction, or “extra layers”, present were considered positive. Step-up authentication aligned with participant expectations of security and demonstrates the importance of security measures that are tailored to meet individual user actions.
- Importance of protecting vulnerable customers
The research reiterated the importance of accessibility and protecting vulnerable customers. All users across the spectrum of human diversity should have access to robust and easy-to-use authentication methods, which match their expectations of security and take measures to protect their privacy. Both permanent and temporary disability impact how users prefer – and are able – to authenticate online. Further findings included people who can not read or write, or those with English as a second language, who may find it hard to comprehend complex information presented to them, reiterating the importance of providing alternative ways to authenticate where possible, which conform to the latest Web Content Accessibility Guidelines (WCAG).
An alarming finding from the research was the risk malicious intent poses to vulnerable users. Cases are varied in nature, but regularly involve Domestic and Family Violence, or elder abuse, particularly in communities where English is a second language. This highlights an issue far greater than the need for secure authentication, and identifies a systemic and widespread societal issue, one which secure authentication can’t solve, but can do its part in reducing potential suffering.
Outcome Summaries
Redirect with One Time Password
The research found One Time Password to be a generally well-performing authentication method. Consumers were typically familiar with the verification requirements, having regularly used the OTP model in various contexts; with banking platforms being the most frequently cited. Consistent exposure to this method meant consumer participants were confident with the flow and aware of what to do at each step, making it a fast and easy process to complete. OTP offers a level of convenience to users by removing the need to recall lengthy and complex passwords, and quickly auto-filling OTPs from SMS text messages on some newer devices. From a security perspective, consumer participants appreciated the OTP expiration window and preferred entering a one time password in place of their actual password; subsequently reducing drop-off rates. However, OTP did not match participant expectations contextually; as most participants were familiar with the model as a second factor of authentication and did not perceive it as strong enough when used as a primary, standalone model.
More detailed findings can be found in the Round 1 report.
App/Web-to-App with Biometric
The research found App/Web-to-App with Biometric to be a generally well-performing authentication method. The majority of participants tested were familiar with biometric methods of authentication and cited using them on a regular basis. The highly automated process of the App/Web-to-App flow and use of biometrics meant participants had very little information to recall or input throughout the flow. Participants appreciated the ease with which they could authenticate with this method, and although they like authenticating with biometric means, they believe it is not always the most appropriate method for sensitive use cases when used as a single factor. While the method was familiar and found to be very easy to use, many participants expected a standardised approach to authentication; with consistent and strong authentication required to access any type of data, no matter the sensitivity. Consumer participants reported greater feelings of control and confidence when more than one factor was required as a ‘confirmation’ of action. The participant expectation is that multi-factor authentication would be in place across all sectors and types of data, regardless of the sensitivity.
More detailed findings can be found in the Round 2 report.
Decoupled with QR Code
The research found that decoupled authentication was accepted in some use cases but not others. There was a strong preference to be taken to an existing, pre-installed app which had been downloaded from a reputable source, as users would have a pre-established level of trust and confidence. Consumer participants were not as comfortable with being redirected to a website in their browser, as they perceived it to carry security risks. When being redirected to a website, it was not immediately clear to participants why they couldn’t simply continue the process on the originating device (desktop in the instances tested), adding to the lack of transparency and trustworthiness. Many consumer participants had their banking provider’s mobile app installed on their phones. This contrasts with less digitally mature sectors, where the use of mobile apps is less common. As such, decoupled experiences that require switching from an originating device to an app may be more successful for the financial sector in the interim, but this may improve over time if app adoption increases in other sectors. Decoupled authentication could be supported with focus on educating users on the process, safety and validity of QR codes, and avoiding device-switching if no DH app is available. There was also a strong desire from consumer participants for extra authentication factors.
More detailed findings can be found in the Round 3 report.
Global Performance: Radial Graph
Global Performance was developed by the research team to define success for various authentication models, made up of five separate measures. Each measure consists of 3 different metrics collected throughout the research sessions. The metrics are then collated to determine a quantifiable outcome for each measure. These 5 measures are then reflected on a five-point radial graph, demonstrating the global performance for the respective authentication model.
- Information a user needs to recall: how much information is a user required to recall to successfully authenticate themselves (eg. Customer ID, lengthy and complicated passwords)
- Users perception of length of time: how long did the user perceive the length of time it took them to authenticate, and, did they find it appropriate
- Number of user inputs: how many fields were users required to successfully input throughout the authentication process
- Familiarity: how familiar a user is with a specific authentication model, and, do/have they used it frequently
- Brand influence: is user trust influenced by the brand they are authenticating with (eg. do they place more trust in a Big 4 bank than they do a smaller player)
- Current authentication models: what model/s does the user currently use
- User feeling in control: what element/s of the authentication method gives the user the feeling of being in control
- Awareness of next step: could the user accurately anticipate what would happen at each step based on the information provided to them in the flow
- Trustworthiness: how trustworthy did the user find the authentication method
- Benefit awareness: was the user aware of the benefit of the authentication method in conjunction with the use case in which it was applied
- Sensitivity of value proposition: was the user influenced by the value proposition (e.g. did they feel more likely or less likely to authenticate with the method due to the value they derived)
- Level of positive-friction: did the user feel the authentication method was easy enough for them to complete and hard enough for someone else who was wrongfully trying to access their data
- User security expectations: how did the authentication method meet or exceed the user’s expectation of security, if not, why did it fail
- Perceived security: how secure did the user perceive the security of the authentication model and what elements contributed to this perception
- Sector: was the user influenced by the sector in which the use case occurred (e.g was the user more or less trusting of a specific authentication model when accessing banking data vs energy data)
Recall &/input | Familiarity & completion | Comfort & control | Purpose & outcome | Expectations |
Information a user needs to recall | Familiarity | User feeling in control | Benefit awareness | User security expectations |
Users’ perception of length of time | Brand influence | Awareness of next step | Sensitivity of value proposition | Perceived security |
Number of user inputs | Current authentication models | Trustworthiness | Level of positive-friction | Sector |
Overall, the authentication model with the highest performing scores was App/Web-to-App. Though, out of all five measures, App/Web-to-App scored highest in only two (Familiarity & Completion and Comfort & Control). One Time Password also performed best in two (Recall & Input and Expectations) and they both tied in one measure (Purpose & Outcome), so it was an even split of winners across the five measures.
One Time Password | Moderated | Unmoderated | Combined |
Recall & input | 3.93 | 4.08 | 4.01 |
Familiarity & completion | 3.73 | 3.39 | 3.56 |
Comfort & control | 3.97 | 3.08 | 3.53 |
Purpose & outcome | 3.97 | 3.08 | 3.53 |
Expectations | 4.10 | 3.25 | 3.68 |
App/Browser-to-App | Moderated | Unmoderated | Combined |
Recall & input | 4.02 | 3.83 | 3.93 |
Familiarity & completion | 3.95 | 3.72 | 3.84 |
Comfort & control | 3.54 | 3.94 | 3.74 |
Purpose & outcome | 3.46 | 3.60 | 3.53 |
Expectations | 3.35 | 3.77 | 3.56 |
Decoupled | Moderated | Unmoderated | Combined |
Recall & input | 3.88 | 3.63 | 3.75 |
Familiarity & completion | 3.52 | 2.98 | 3.25 |
Comfort & control | 3.48 | 3.56 | 3.52 |
Purpose & outcome | 3.39 | 3.33 | 3.36 |
Expectations | 3.62 | 3.58 | 3.60 |
Note: 0.00 to 2.99 is Bad; 3.00 to 3.24 is Poor; 3.25 to 3.74 is Good; 3.75 to 3.99 is Very Good; 4.00 to 5.00 is Excellent. Anything with a 0.40 or greater difference between unmoderated and moderated testing cohorts for a measure is considered significant.
- One Time Password was the best performing with an excellent score of 4.01.
- App/Web-to-App followed very closely behind with a very good score of 3.93 and lastly Decoupled at 3.75, also considered very good.
- Though not significant, One Time Password was the only model which performed better with the unmoderated cohort.
- App/Web-to-App was the best performing with a very good score.
- One Time Password scored 3.56 and Decoupled came in last with a score of 3.25.
- Decoupled results differed significantly with unmoderated scoring 0.54 lower than moderated. It was the only cohort across all models which scored below 3 for Familiarity & Completion.
- This measure saw the greatest variation in outcomes across the three models.
- App/Web-to-App performed the best for this measure with a very good score of 3.74, there was a variation of 0.40 points between the cohorts with moderated performing better than unmoderated.
- One Time Password and Decoupled had very close scores of 3.53 and 3.52 respectively.
- Notably, despite App/web-to-App taking the top-spot, the One Time Password moderated cohort was actually the highest scoring group at 3.97 (0.03 points more than App/web-to-App moderated) and a significant difference of 0.89 between the moderated and unmoderated groups.
- One Time Password and App/Web-to-App tied with good scores of 3.53 for this measure.
- One Time Password reported a higher variation (0.89) between the moderated and unmoderated cohorts with scores of 3.97 and 3.08 respectively. This may be attributed to the nature of the testing sessions, with 1:1 interviews requiring more critical thinking from participants.
- Decoupled also resulted in a good score of 3.36 and very similar results for both moderated and unmoderated testing groups.
- One Time Password performed the best for this measure with a score of 3.68.
- This measure had the closest outcomes for each model with only a 0.13 point difference between the best and worst score, Decoupled scored 3.60 and App/Web-to-App scored 3.56. This was also the only measure where Decoupled didn't score last.
- The moderated cohort for OTP was the best performing group across all models and measures with an excellent score of 4.10.
System Usability Scale
App/Web-to-App was the best performing model when it came to System Usability with a score of 82.88, however it only marginally beat Redirect with One Time Password which scored 82.61. This difference of 0.27 points is marginal and there is not much differentiation between the usability performance between these two models. Decoupled scored slightly lower at 74.29, but this is still an above average score.
The average SUS score is 68 for technology in general (while that may indicate 68% of the total maximum score, it’s actually more appropriate to call it 50%). Usability scores of 80.3 or higher are well-performing and bode very well, scores of 68 or thereabouts are average and need some work to improve and anything under 51 is a problem and needs addressing.
All models were well performing when it came to System Usability.
Consumer Behavioural Archetypes
Consumer archetypes help segment and succinctly describe different drivers, behaviours, and needs observed through research. The archetypes have been developed by the DSB to represent common behavioural and attitudinal themes relating to data sharing. There are four identified CDR archetypes; Sceptics, Assurance Seekers, Sense-makers and Enthusiasts. Each archetype has specific needs for how authenticating to share CDR data should work to be trustworthy and comprehensible.
Interestingly, no model tested had any Enthusiast consumer archetypes. Enthusiasts are excited to get the benefits of authenticating to share CDR data and generally value simple experiences once trust is established.
Assurance Seekers were the highest represented group of consumer archetypes across all three models. Characteristically, Assurance Seekers want to read additional information. They generally value familiarity and external reference/support, and are apprehensive to new experiences. They made up just over half of all consumer participants for both App/Web-to-App and Decoupled models.
Redirect with One Time Password saw just under half of all consumer participants fit into the Sceptic archetype. Sceptics are less trusting of organisations and/or technology. They generally value control, and are averse to sharing data based on experience with current practices.
Decoupled had the smallest representation of Sense-makers. Sense-makers need to understand how the process works. They generally value details, and can trust the process if given enough valuable information. This is consistent with the Decoupled qualitative findings and consumer perceptions of authenticating with a QR Code.
Opportunities
The findings from all three rounds of research support the opportunity for a combination of step-up and waterfall authentication frameworks. Many participants were familiar with step-up authentication, and expected corporations to implement 2FA and step-up models regardless of the sensitivity of the data being accessed. This awareness and desire for tighter security may be related to recent high profile data breaches but might also indicate a general increase in data literacy and privacy awareness among consumers. The research found variables such as the authentication platform, sensitivity of data, sector and macro environmental impacts (such as data breaches) all had a bearing on consumer participant perception of the security and trust in authentication models.
No models tested were completely rejected, rather they were accepted with caveats or with areas identified for improvement. There was a clear desire for App/Web-to-App to be supported in the CDR, affording consumers the option to authenticate within their DH app. The research on Redirect with One Time Password identified several key opportunities and improvement areas and could be uplifted to continue being a supported model. Decoupled could also be supported to allow the user to authenticate securely with their known device no matter how they interact with the CDR. This may potentially see a separation of Decoupled from QR Code as the mechanism which connects the two channels.
This suggests all models tested could be supported within the CDR, as part of step-up and waterfall authentication frameworks, with clarity around how they can be implemented and adopted. A waterfall approach to authentication could be considered that supports App/Web-to-App, Redirect with One Time Password, and Decoupled authentication; and facilitate a framework of fall-back options. This approach could give consumers and DHs alike more optionality and flexibility while allowing for consistent authentication experiences.
The step-up framework should also consider Credential Level pairings, recommendations from both the PwC IC Accessibility and Independent Security Review reports, and also uplift the Redirect with One Time Password model with research.
The DSB are now working on a Decision Proposal to consult on the step-up and waterfall authentication frameworks.
Quick links to CX Guidelines:
