Updated @January 18, 2023
This page is about how the Data Standards Body's CX team measures consent quality, comprehension, characteristics, and other research-related information.
Comprehension and informed consent
The CDR Rules require the consent given by a CDR consumer to be voluntary, express, informed, specific as to purpose, time limited, and easily withdrawn. The OAIC's Australian Privacy Principles Guidelines on consent were used to inform how we defined and assessed consent quality. The following criterion has been used to assess participants' ability to comprehend and remember the terms of their consent. Commonly, participants have been asked to recall their consent:
- Immediately after completing the 'original consent' flow
- Sometime after their 'original consent' (typically this has ranged from several days to several months after completing the 'original consent' flow)
- Immediately after completing the 'amending consent' flow(s)
CDR consents must be:
- voluntary
- express
- informed
- specific as to purpose
- time limited
- easily withdrawn
Measures
DSB criteria | CDR Rule 4.9 criteria | Measure of success | Example 'Pass' responses ✅ | Example 'Fail' responses ❌ |
Data Holder (DH) | Informed | Participant is able to recall who they were sharing data FROM. | • "My bank"
• [DH name] | • "Not sure" |
Accredited Data Recipient (ADR) | Informed | Participant is able to recall who they were sharing data TO. | • "A budgetting app"
• "Budget"
• "Budget Guide" | • [DH name]
• "Not sure" |
Purpose/benefit | Specific as to purpose;
Informed | Participant is able to recall why they are sharing data or the ADR value prop. | • "To track and manage my budget"
• "Review, categorise spending across multiple accounts" | • "So others could access it"
• "Understand customers needs"
• "Not sure" |
Type of data shared | Informed | Participant is able to recall datasets, permissions, uses or accounts. | • "Account Balance and type, transaction details, direct debit and schedule payments"
• "My bank account balances and payments made from each account"
• "Bank account numbers, different accounts i.e. savings" | • "All it asked for"
• "Bank preferences" |
Sharing period | Time limited;
Informed | Participant is able to recall the duration of their data sharing arrangement. | • "A year"
• "24th July 2020 • 23rd July 2021" | • "Until I say otherwise"
• "6 months"
• "Don't remember" |
Risks of sharing data | Informed | Participant is aware of potential and perceived risks of data sharing. | • "Hacking, illegal use of my account by others"
• "Privacy"
• "The app assured me that their were no risks and that I could opt out at any time" | • "No risks" |
Risks of not sharing data | Voluntary;
Informed | Participant is able to identify potential consequences of not sharing data with the ADR. | • "I might not reach my desired goal and have to manually figure out how to budget myself."
• "Incorrect information (or incomplete) may distort what you are trying to achieve by using the application." | • "If you didn't share your data you couldn't move on to the next task"
• "Nothing" |
How to manage or revoke data sharing | Easily withdrawn;
Informed | Participant is able to recall how they might manage or stop sharing data with the ADR. | • "Writing to [ADR] email"
• "Stop it in the app"
• "Through the app, unclicking what I choose to share."
- "notifying the Bank" | • "I would delete the app" |
How redundant data is handled | Informed | Participant is able to recall what will happen to their redundant data after the 'sharing period.' | • "Deleted"
• "Destroyed"
• "Removed from the database"
• "[ADR] will delete my data when I no longer consent [ADR] to collect or use it"
• "They say deleted but who knows" | • "No idea"
• "Data still available from the Cloud" |
Assessment method
Pass (1) and Fail (0). Assessment and scoring were by proxy.Please note, our approach varied between rounds; Language and questions have been slightly iterated and participants have responded to certain tests verbally and/or in a survey form.
- For verbal interviews, participants have responded to questions.
- For survey forms, participants have responded to 'fill in the blank' style fields with assistive text.
Individual participant scores are calculated based on the criteria listed above.
Aggregated scores and findings
While criterion has been marked to 'Pass' or 'Fail' for scoring purposes, some of the results have been elaborated on to produce more qualitative detail in research reports.
Common findings
- Purpose/benefit
- How redundant data is handled
- Data Holder (DH)
- Accredited Data Recipient (ADR)
For example, participants were not able to recall the sharing period (of 12 months) but were able to recall that they were sharing their data for a limited time.
For example, participants were able to recall the types of data shared. While responses varied, most participants recalled datasets and some permissions. Some participants struggled and were only able to recall at an account-level.
For example, participants did not see any risks in data sharing with the ADR. While they believed there's "always a risk" with data sharing in general, they felt that the CDR process offered constant assurance and reduced risk. Generally, those participants with low privacy awareness did not think there were any concerns or risks with data sharing with accredited or non-accredited companies.
- Risks of not sharing data
- How to manage or revoke data sharing
Scoring examples
DSB criteria | 'Higher accuracy' response
Phase 3, R3P02 scored 8 out of 8 | 'Average accuracy' response
Phase 3, R3P10 scored 5 out of 8 | 'Lower accuracy' response
Phase 3, R6P20 scored 5 out of 9 |
Data holder (DH) | ✅ ”my bank [data holder name]” | ✅ ”my bank accounts” | ✅ ”[data holder name]” |
Accredited data recipient (ADR) | ✅ ”a budget planning app” | ✅ ”the budget app and potentially others - looks like data is shared with market researchers etc” | ❌ ”?? Prototype app” |
Purpose/benefit | ✅ ”increase my financial wellbeing and address any recurring bad financial habits I may have and identify patterns of spending behaviour” | ❌ ”Use the app” | ✅ ”To consolidate financial information so it can all be seen in one app.” |
Type of data shared | ✅ ”All my account details, including transactions, balance, direct debit, scheduled payments” | ✅ ”transactions, accounts, balance info - not sure if other data types e.g. age, address etc are also shared” | ✅ ”Bank account/s information, budget” |
Sharing period | ✅ ”1 year” | ❌ ”until I delete the app I think and then data is deleted as per my selection” | ✅ ”One year” |
Risks of sharing data | ✅ ”Not wanting to have my ex-partner notified that I am using this app” | ✅ ”because I feel there's enough data on me as it is in the world” | ✅ ”Risk of app being hacked.” |
Risks of not sharing data | (not addressed) | (not addressed) | ❌ ”No” |
How to manage or revoke data sharing | ✅ ”go to the appropriate section of the website and follow the steps to revoke my authorisation” | ❌ ”deleting my account” | ❌ ”Opt out at any time.” |
How redundant data is handled | ✅ ”Either deleted or shared in a de-identified way depending on which option I choose” | ✅ ”deleted - or I can be de-identified” | ❌ ”Unsure” |
Other assessments
Criteria (Participant can identify...) | OAIC consent criteria | Likert scale |
How capable did you feel to stop sharing your data with ADR? | Easily withdrawn | 1-Not very confident
2-Not confident
3-Neutral
4-Confident
5-Very confident |
How difficult/easy was it to share your data? | Voluntary | 1-Very difficult
2-Difficult
3-Neutral
4-Easy
5-Very easy |
Measuring literacy and other characteristics
A broad and diverse range of participants have been recruited to help reduce bias and research out risk. The recruitment process strives to reflect the demographic percentages outlined by the Australian Bureau of Statistics (ABS), and explicitly includes those who may be experiencing vulnerability or disadvantage.
Vulnerable people can include:
- children and seniors
- people with impaired intellectual or physical functioning
- people from a low socio-economic background
- people who are Aboriginal or Torres Strait Islanders
- people who are not native speakers of the local language
- people with low levels of literacy or education
- people subject to modern slavery, which involves human exploitation and control, such as forced labour, debt bondage, human trafficking, and child labour.
Vulnerability may be either temporary or ongoing.
This definition is from the Australian Charities and Not-for-profits Commission [1].
Financial literacy and experience
The following criterion has been used to assess participants' financial literacy.
- Participant response to recruitment screener are used to establish a baseline understanding.
- "When your bank sends a bank statement, do you find it:"
- "What kind of bank accounts do you have in your name?"
- Participant response to artefacts and interview are used to validate or amend the baseline.
- Researcher observed attitudes and behaviours are used to validate or amend the baseline.
Screener questions such as:
Ability and confidence when defining banking data language.
Interaction with banking sector and use of other financial tools.
Characteristics | Participant response to screener | Participant response to artefacts/interview | Researcher observed attitudes/behaviours |
Low financial literacy | Bank statements are...
• Difficult to understand
• I don't receive bank statements | Can't define datasets or permissions | |
Medium financial literacy | (Mix of 'low' and 'high' criteria) | ||
High financial literacy | Bank statements are...
• Easy to understand
• I don’t read through my bank statements | Can confidently and accurately define datasets or permissions | • Multiple types of accounts
• Monitors/manages own finances regularly |
Energy literacy and experience
The following criterion has been used to assess participants' energy literacy.
- Participant response to recruitment screener are used to establish a baseline understanding.
- "When you get your energy bill, do you find it:"
- "I switched energy providers:"
- Participant response to artefacts and interview are used to validate or amend the baseline.
- Researcher observed attitudes and behaviours are used to validate or amend the baseline.
- Interaction with own energy data.
- Installation of other energy utilities, such as gas and solar.
Screener questions such as:
Ability and confidence when defining energy data language.
Interaction with energy sector, including:
Characteristics | Participant response to screener | Participant response to artefacts/interview | Researcher observed attitudes/behaviours |
Low energy literacy | Energy bills are...
- Difficult to understand
- I don’t read through my energy bills
- I don't receive energy bills | - Can't define datasets or permissions | - Does not monitors/measures own energy usage and consumption |
Medium energy literacy | Energy bills are...
- I don’t read through my energy bills | - Can define datasets or permissions | - Monitors/measures own energy usage and consumption |
High energy literacy | Energy bills are...
- Easy to understand | - Can confidently and accurately define datasets or permissions | - Monitors/measures own energy usage and consumption |
Digital ability
The Australian Digital Inclusion Index 2019 (ADII) has informed our research and approach. ADII assess digital ability based on three areas:
- Attitudes: including notions of control, enthusiasm, learning, and confidence
- Basic Skills: including mobile phone, banking, shopping, community, and information skills
- Activities: including accessing content, communication, transactions, commerce, media, and information.
Similarly the following criterion has been used to assess participants' digital ability.
- Participant response to recruitment screener are used to establish a baseline understanding.
- "Which of the following applies to your use of the internet and online technologies?" to understand 'Basic Skill'.
- "What kinds of digital apps or services do you use?" to understand 'Activities'.
- Researcher observed attitudes and behaviours are used to validate or amend the baseline.
Screener questions such as:
Familiarity and confidence with universal interaction/UI patterns, to understand 'Attitudes'.
Characteristics | Participant response to screener | Researcher observed attitudes/behaviours |
Low digital ability | Basic Skill
• I don't use the internet often, only if I have to. My main form of communication is not online
• I use the internet to check my email and read news, but not more than an hour a day
Activities
• Currently uses few digital apps or services
• Currently uses similar digital apps or services | Attitudes
Not familiar or confident with universal interaction/UI patterns, e.g. toggles, expandable panels, etc. |
Medium digital ability | Basic Skill
• I am internet-savvy and spend a lot of time online
• I couldn't live without the internet | Attitudes
Familiar and confident with universal interaction/UI patterns, e.g. toggles, expandable panels, etc. |
High digital ability | Basic Skill
• I couldn't live without the internet
• I spend a lot of time on social media
• I use online apps frequently throughout my day
Activities
• Currently uses many digital apps or services
• Currently uses a variety of digital apps or services | Attitudes
• Familiar and confident with universal interaction/UI patterns, e.g. toggles, expandable panels, etc.
• Suggests improvements to experience based on other digital interactions |
Privacy importance
The following criterion has been used to assess participants' privacy importance.
- Participant response to recruitment screener are used to establish a baseline understanding.
- Participant response to artefacts and interview are used to validate or amend the baseline.
- Researcher observed attitudes and behaviours are used to validate or amend the baseline.
Screener questions such as: "How important is the privacy of your information and data when using a digital app or service?”
Task recap survey question, "I am aware that by sharing this data I could risk the following ..."
Perceived risk and concerns and use of privacy preserving tools.
Characteristics | Participant response to screener | Participant response to artefacts/interview | Researcher observed attitudes/behaviours |
Low privacy importance | • Not important
• Don’t know | For example: "There is no risk with data sharing" | • Does not use any privacy preserving tools
• Did not mention any privacy preserving behaviours or actions |
Medium privacy importance | Quite important | (Mix of 'low' and 'high' criteria) | (Mix of 'low' and 'high' criteria) |
High privacy importance | • Very important
• Extremely important | For example: "There's always the risk of hacking or breaches" | • Uses privacy preserving tools, e.g. special browsers or plugins
• Mentions privacy preserving behaviours or actions, e.g. checking 'from' email addresses (phishing) |
Willingness to share data
The following criterion has been used to assess participants' willingness to share data.
- Participant response to recruitment screener are used to establish a baseline understanding.
- Participant response to artefacts and interview are used to validate or amend the baseline.
- Researcher observed attitudes and behaviours are used to validate or amend the baseline.
- Perceived risk and concerns
- Perceived benefit and trust.
Screener questions such as: "What kinds of digital apps or services do you use?"
Task recap survey question, "How willing would you be to share your data if this was the new way of doing things?"
Factors such as:
Characteristics | Participant response to screener | Participant response to artefacts/interview | Researcher observed attitudes/behaviours |
Low willingness to share data | • Currently uses few digital apps or services
• Currently uses similar digital apps or services | Likert response:
1-Very unwilling
2-Unwilling
Open response example:
"I'm old fashioned and I would not share my data" | • Believes "risks outweigh the benefit"
• Prefers to use 'offline' tools or approaches
• Low trust in the process and company |
Medium willingness to share data | Likert response:
3-Neutral
Open response example:
"I'll get used to it with time and exposure” | (Mix of 'low' and 'high' criteria) | |
High willingness to share data | • Currently uses many digital apps or services
• Currently uses a variety digital apps or services | Likert response:
4-Willing
5-Very willing
Open response example:
"I've used similar data sharing methods before" | • Believes that "benefits outweigh the risks"
• High trust in the process and company |
CDR awareness
The following criterion has been used to assess participants' CDR awareness.
- Participant response to artefacts and interview are used to validate or amend the baseline.
- Viewed CDR landing page prior to Consent Flow; and/or
- Exposure to CDR (over multiple occurrences) and their ability to explain/recall the term CDR.
If participants had:
Characteristics | Participant response to artefacts/interview |
Low CDR awareness | • Didn't view CDR landing page
• Completed the CDR process once only |
Medium CDR awareness | Able recall and define CDR with low fidelity |
High CDR awareness | • Viewed CDR landing page or completed the CDR process multiple times
• Able recall and define CDR with high fidelity |
Quick links to CX Guidelines:
