UX Reaserch for Website Cookie Pop-up

1. Introduction

Many of us have encountered sudden pop-ups requesting cookie consent while browsing websites . These web pages require your agreement to set cookies—small files saved to your device—to enable certain functionalities, such as recording your preferences and login status. This often occurs when we open a specific webpage for the first time, register a new account, or click on an advertisement. The cookies you agree to may store some of your privacy settings and track your behavior while you browse the webpage.

The occurrence of these pop-ups is primarily due to the General Data Protection Regulation (GDPR) issued by the European Union in May 2018. The GDPR categorizes cookies as part of user data, leading many websites to implement cookie alerts spontaneously to enhance transparency for users. However, for most of us, we may not read the privacy policies; instead, we mechanically click “accept” to ensure access to the content we genuinely want to browse.

With the increasing number of these pop-ups—regardless of whether they truly inform users about the potential misuse of their data privacy—they undeniably diminish our browsing experience. It seems that no matter which website you visit, there’s always a cookie consent pop-up, which often feels no different from annoying pop-up ads.

To enhance the smoothness and security of users’ web browsing experience, I propose optimizing cookie consent pop-ups on websites. My goal is to explore better ways to ensure that users’ browsing experiences are not disrupted and that they effectively understand the purpose of these cookies and their impact on privacy.

2. NEEDFINDING

2.1 Survey

The task involves understanding users’ thoughts and experiences when facing cookie setting pop-ups. The term “user” refers to anyone using electronic devices to browse the internet and encountering these pop-ups. Essentially, all internet users are subjects of this survey.

I am conducting an online questionnaire using a peer survey to research the following aspects:

User Attributes: Age, gender, devices, and browsers used.
Context of Encounter: Situations when users encounter cookie pop-ups.
User Awareness: Understanding of what cookies are.
Support from Websites: Whether websites help users understand the purpose of cookies.
Impact on Browsing Experience: Effects of cookie pop-ups on users’ overall browsing experience.
Privacy Protection: Whether cookie settings effectively protect user privacy.
Personalization of Experience: Whether cookie settings help personalize the user experience.
Suggestions for Improvement: Ideas for better implementation of cookie settings.

The questionnaire will be distributed, with half sent to peers in an HCI course and the other half to friends, family, and classmates, most of whom do not have relevant knowledge in HCI. Participants will first view an image of cookie settings to jog their memory, followed by a series of questions related to the aspects mentioned above (see Appendix 12.1: Needfinding Survey).

2.2 Reuslt

The survey was distributed using PeerSurvey, targeting primarily students from Georgia Tech, along with some of my relatives and friends. A total of 47 responses were collected, close to the anticipated target of 50 responses (see Appendix 12.2: Survey Responses).

1. Demographics of Respondents

Age Distribution: The majority of respondents (70%) fell within the age range of 18 to 29 years old.
Gender: A predominantly male group represented 60% of the respondents.
Technology Usage: Approximately 70% reported using Chrome as their primary browser, while nearly 70% preferred using a PC as their device.

2. Familiarity with Cookies

When asked about their familiarity with cookies, 85% of respondents indicated they were either somewhat familiar or very familiar with them. This level of familiarity is understandable given the prevalence of digital technology and online activities among younger demographics.

3. Encountering Cookie Pop-Ups

Circumstances: When asked about the circumstances in which users encounter cookie pop-ups, 98% stated they encounter them when first visiting a webpage. Additionally:
- 40% encounter them when logging in from a different location.
- One-third encounter them during login or registration processes.
Frequency: In terms of frequency, 95% of respondents indicated they encounter these pop-ups very frequently or frequently, while the remaining 5% responded occasionally.

4. Clarity of Information

When asked whether websites clearly inform users about the purpose of cookies:

50% of respondents believed that websites perform poorly in this regard.
30% maintained a neutral stance.

5. Perceptions of Cookie Pop-Ups

In response to three agree-disagree statements:

70% of users agreed or strongly agreed that website cookie pop-ups reduce their willingness to browse the web.
Conversely, when asked if cookies enhance the personal user experience, only 28% agreed, with the remainder expressing neutrality or disagreement.
76% of respondents believe that rejecting cookies can protect their privacy.

6. Summary and Insights

In summary, users have a negative perception of cookie pop-ups. This approach by websites fails to adequately inform users about the purpose of their cookies, leading to concerns about privacy settings and hindering a smooth browsing experience. Today’s cookie pop-ups, rather than enhancing privacy and personalization, seem more like disclaimers aimed at meeting minimum legal requirements.

Question	Option 1	Option 2	Option 3	Option 4	Option 5	Option 6
1. Age	under 18	18-29	30-39	40-49	50-64	65+
0	33	11	3	0	0
2. Gender	Male	Non-binary	Female	I prefer not to say
31	0	14	2
3. Browser	Chrome	Edge	Firefox	Safari	Others
29	2	9	5	2
4. Device	Phone	Tablet	Personal Computer	Game console	Others
12	2	32	0	1
5. Familiarity	Very	Somewhat	Neutral	Not very	Not at all
19	21	0	4	3
6. When it happens	First time website visit	Login or registration	Clicking on advertisement	Access website in different place	Comments or saving preferences	Others
45	15	6	21	5
7. Frequency	Very frequently	Frequently	Occasionally	Rarely	Never
21	22	4	0	0
8. Understanding	Very good	Good	Neutral	Bad	Very bad
2	6	15	18	6
9. Reduce will	Strongly agree	Agree	Neutral	Disagree	Strongly disagree
10	17	9	11	0
10. Personal experience	Strongly agree	Agree	Neutral	Disagree	Strongly disagree
3	10	19	10	5
11. Privacy	Strongly agree	Agree	Neutral	Disagree	Strongly disagree
10	26	10	1	0

3. INITIAL BRAINSTORMING

3.1 Brainstorming Plan

The core problem of this brainstorming is:

“How to clearly convey to users the impact of cookies on personalization and privacy without causing disruption to users’ browsing experience.”

My plan is to start with independent brainstorming, attempting to spend 10 minutes several times over one or two weeks to note down one-sentence bullet points. Then, I will explain the objectives and purpose to ChatGPT and ask for creative ideas. By comparing its suggestions, I will try to integrate or separate each point and then identify three actionable plans for prototypes.

3.2 Brainstorming Results

I brainstormed some highly conceptual solutions (Appendix 12.3: Brainstorming results) and compared them with suggestions from GPT. After integration, I had GPT generate more creative solutions based on them (Appendix 12.4: ChatGPT dialogues).

These solutions can roughly be categorized as follows:

Changing the presentation method: using interfaces such as maps, radars, or mini-games instead of pop-ups.
Changing the type of information: incorporating more images, videos, or illustrations to aid in understanding privacy settings.
Altering the presentation steps: no longer requiring users to focus on cookies upon opening a page but instead making the effects of settings visually apparent through default settings or timelines mapping.

Here are three selected ones:

Categorized cookies settings
- Introduce a detailed setting menu that categorizes cookies so that users can better understand the purpose of each cookie setting.
- Descriptions are provided under each cookie setting, differentiated by different colors, providing options for users who want to learn more.
- The traditional options of accepting all cookies or accepting necessary cookies only are still retained.
An interactive slider bar
- Develop an interactive slider that visually demonstrates the differences between accepting and rejecting cookies. Users can slide a bar to toggle between the two options and see real-time changes in website behavior.
- Include clear, concise explanations alongside the slide to help users understand the implications of their choices.
- Offer additional information or resources for users who want to learn more about cookies and privacy.
Real-Time Cookie Tracker
- Create a real-time cookie tracker feature that displays a list of active cookies being used by the website.
- Include information such as the name of the cookie, its purpose, and whether it’s from a first-party or third-party source.
- Allow users to easily manage their cookie preferences directly from the tracker, enabling them to accept, reject, or customize settings for individual cookies.

I suppose these few solutions are the most likely to develop. They are both interesting and practical, focusing on balancing the contradiction between protecting privacy settings and browsing webpages.

4. INITIAL PROTOTYPING

4.1 First Prototype

For the first prototype, there is still a pop-up window, but with much more detailed control of cookie settings. Cookies are divided into three types: essential, analysis, and marketing so that users know exactly how each one works and can obtain full control over them. I suppose this will bridge the gulf of execution.

Besides, to let users feel that they are not being forced to make these settings, I changed the title to “Choice is yours!”.

4.2 Second Prototype

For the second prototype, a pop-up window appears first, but instead of the settings of each cookie, a slider bar for privacy and functions is displayed. This serves as a privacy meter that users can drag freely. An explanation of what users will get from this setting will appear as they drag the slider bar. By changing the way they interact, it will make the decision intuitive and easy

4.3 Third Prototype

For the third prototype, I canceled the pop-up window, allowing users to access the page directly without any disruption. Instead, a small indicator of cookies appears beside the page elements that require cookies, such as the register button, images, and personalized advertisements. This approach makes the interface less intrusive, minimizing disruption for users while they complete their tasks. Additionally, it is easy and direct to manipulate, with each icon corresponding to an element that users can adjust at any time.

5. EVALUATION

5.1 Planning

This assessment will be conducted via an online survey. Participants will first watch a demonstration video of the prototype, then click on the prototype link to experience it themselves. Subsequently, users will be asked to answer some questions. Since our survey requires more time commitment, I aim to keep the questions simple and straightforward to ensure response rates.

The questions will include users’ preferences for each model and reasons, their favorite model, as well as additional suggestions. I expect to receive approximately 20 responses, including both qualitative and quantitative data (see Appendix 12.5: Evaluation Survey).

These likes and dislikes will be quantified as evaluations of each feature according to the following rules (see Table 1). For open-ended questions, I will categorize them into three types: strengths, weaknesses, and suggestions. This categorization will help identify the advantages, drawbacks, and areas for improvement of the prototypes.

Preference	Score
Strongly dislike	-2
Dislike	-1
Neutral	0
Like	1
Strongly Like	2

5.2 Reuslt

The evaluation of the three prototypes was conducted through asynchronous online surveys, involving a total of 19 participants. Half of the participants were family members and friends, while the other half were students from the HCI course. Efforts were made to include the same group of individuals who filled out the previous survey on this topic (see Appendix 12.7: Evaluation Results).

1 First Prototype

Quantitative Outcome:

11 out of 19 participants liked or strongly liked the design.
5 participants remained neutral.
3 participants chose ‘dislike’ or ‘strongly dislike’.
Average score: 0.57.

Pros:

Familiarity due to similarity to current interface.
Clear and customizable controls for different types of cookies.
Straightforward navigation and customization options.
Auto-allowance button for essential cookies adds convenience.

Cons:

Some participants found the pop-up annoying.
Not everyone appreciated the cookie categorization due to time constraints.

Second Prototype

Quantitative Outcome:

4 out of 19 participants liked or strongly liked the design.
7 participants remained neutral.
8 participants chose ‘dislike’ or ‘strongly dislike’.
Average score: -0.21.

Pros:

Easy-to-use interface.
Reduced cognitive load when interacting with the pop-up.
Slider feature offers a new approach to control.

Cons:

Slider feature is unstable and difficult to control.
Text visibility is lacking until the slider is moved.
Requires too much thought and time for effective use.
Confusing slider options make settings difficult to understand.

3 Third Prototype

Quantitative Outcome:

11 out of 19 participants liked or strongly liked the design.
1 participant remained neutral.
7 participants chose ‘dislike’.
Average score: 0.43.

Pros:

Directly shows the impact of cookies on the website, enhancing transparency.
Natural integration into the website improves user interaction.
Intuitive interface that reduces distraction.
Cookies are hidden unless explicitly chosen.

Cons:

Cookie icon may distract from webpage content.
Lacks a straightforward method to adjust cookie settings.
Requires decisions for each instance, increasing user workload.
Placement of the interface may interfere with content reading.
Initial confusion with many icons on the page.

4 Most Preferred Prototype

Among the prototypes:

9 out of 19 preferred the first prototype.
2 participants preferred the second prototype.
8 participants chose the third prototype.

Reasons for Preference:

First Prototype:

Simplicity with no distractions and familiar feel.
Easiest to discover and understand.
Fastest to navigate.
Most convenient with a single button press.

Third Prototype:

Cookie icon serves as a clear indicator of elements using cookies.
Design perceived as less bothersome and more natural.
Easy-to-use interface that does not obstruct content.
Simple and appealing design.

Additional Opinions:

Some participants argued against color coding in the first prototype due to potential biases from website owners.
Others wanted to know how websites would change with different cookie settings.
Some suggested that the first and third prototypes could be combined for better functionality.

Overall, the evaluation showed a preference for the first and third prototypes due to their simplicity, reduced intrusiveness, clear indicators, and controllability. Combining these two prototypes could effectively address their individual disadvantages and lead to a better solution. Additionally, I am considering using a notification window instead of a pop-up window to further reduce the impact on the website browsing experience.

6. ITERATION PLANNING

6.1 Re-Introduction

In the results, I have indicated users’ preferences for each prototype as well as their evaluations. It’s evident that Prototype 1 and Prototype 3 received higher scores and more user preference. The positive feedback highlights factors such as:

Simplicity
Convenience
Intuitiveness
Less interference
Better integration with web pages
Appealing design

Drawbacks

Prototype 1: Still functions as a pop-up.
Prototype 3: Requires users to control settings one by one, which might disrupt reading.

Both prototypes received favorable ratings, with users suggesting that these two could be combined. I believe this is a great idea. Prototype 1’s detailed simplicity can compensate for the shortcomings of Prototype 3, while Prototype 3’s seamless integration and adjustable settings can address the limitations of Prototype 1.

In the final prototype development, I plan to incorporate both approaches.

Considerations for Undiscovered User Needs

There are several points to consider regarding undiscovered user needs:

Window Presentation:
- Does it need to occupy a large portion of the page and obstruct content in a traditional manner?
- Are there less intrusive presentation methods available?
Cookie Icon Disruption:
- How can the disruption to content by cookie icons be minimized?
- How can users easily understand their usage status?
Integration of Prototypes:
- Is it possible to control the opening and closing of Prototype 3 through the window of Prototype 1?
- Can we link from Model 3 to the settings of Model 1?

A new round of brainstorming will be conducted to find suitable solutions for these issues, contributing to the development of the final prototype. Afterward, I will create more comprehensive middle-fidelity models using Figma and carry out another round of evaluation.

6.2 Re-Brainstorming

The core problem of the second round of brainstorming is: “Combine the two prototypes and attempt a less intrusive pop-up method with the possibility to change settings at any time.”

My plan is to start with independent brainstorming, noting down one-sentence bullet points. Then, I will explain the objectives and purpose to ChatGPT and ask for extended ideas. Finally, I will examine each viewpoint and define the final prototype. I have chosen a combination of the Gentle Fade-In Modal and the Corner Button Widget method (see Appendix 12.8: Re-Brainstorming Results).

1. Gentle Fade-In Modal

The gentle fade-in modal is a subtle and non-intrusive way to present cookie preferences to users. When a user visits a webpage, a modal window fades into view after a short delay, often accompanied by a soft animation or transition effect. This modal contains information about cookie usage on the site and provides options for managing preferences. The gentle fade-in ensures that the modal doesn’t abruptly disrupt the user’s browsing experience while still drawing attention to the importance of managing cookies. Once the user interacts with the modal or makes their preferences known, it fades out smoothly, allowing them to continue browsing without distraction.

2. Corner Button Widget

The corner button widget method involves placing a small, unobtrusive button or icon in one of the corners of the webpage, typically the bottom-right corner. This button serves as a discreet entry point for accessing cookie settings. When clicked or tapped, it triggers the display of a compact panel or menu containing options for managing cookie preferences. By positioning the button in a corner, it remains out of the way of the main content, minimizing visual clutter. Additionally, users can easily locate and interact with the button, as it consistently appears in the same location across different pages of the website. This approach ensures that users have convenient access to cookie controls without disrupting their browsing experience.

7. FINAL PROTOTYPE

The changes made to the final prototype are as follows:

Non-Intrusive Interface:
- There will be no intrusive pop-ups when the webpage is first opened.
- Instead, a small window will gently fade into the main interface at the bottom right corner, and small cookie icons will appear by the elements that use cookies.
- These windows and icons will not interfere with the user’s reading and browsing experience. If the user ignores or closes them, the window and icons will fade out, and a small widget will appear at the bottom right.
- The webpage will use default cookie settings, accepting all cookies, to ensure that all functions on the webpage work properly.
User Empowerment:
- When the user notices the pop-up and wishes to change settings, they will see the title “The choice is yours,” designed to empower the user.
- A brief description will explain why the website uses cookies, followed by the categorization of cookies into three types: necessary, analytics, and marketing. Each cookie will be classified into one of these categories and differentiated by color.
- Additionally, each cookie will have a separate switch for control.
Interactive Elements:
- When the user hovers over the switch of a particular cookie, the cookie icon next to the element on the page that utilizes this cookie will be enlarged, and a short description of the cookie’s function will be displayed.
- This enhances the visual correspondence between elements, improving the mapping between the pop-up settings and cookie icons.
- Moreover, when a user toggles a setting on or off, the corresponding icon will synchronize its state accordingly, and the associated page elements will also change. For instance, if the user disables Cookie 2, the previously visible photos will disappear, and the color of the cookie icon will turn gray.
Traditional Setting Options:
- The pop-up retains traditional setting options, allowing users to accept all cookies, including all necessary cookies.
- When users click on these options, settings will automatically adjust based on the type of cookies. For example, selecting to accept only necessary cookies will enable those marked as necessary while disabling others.
- Users can see synchronous changes in both the UI and page elements after clicking the buttons. Once the settings are finalized, the pop-up will revert to a small widget.

The final prototype’s interaction interface is designed based on several principles of human-computer interaction, mainly including:

Simplicity:
- The design should be easy to understand, regardless of the user’s experience, knowledge, language skills, or current concentration level.
- The new interface contains no complicated text or privacy policies. Cookies are categorized clearly, and areas utilizing cookies are represented by small cookie icons for easy identification.
Mapping:
- This principle refers to creating interfaces where the design clarifies the effects of user interactions.
- The pop-up retains detailed explanations for each cookie. When users hover over a switch, the corresponding cookie icon on the page enlarges as an indicator and displays a brief description. This helps users understand what is affected by the setting change, its location, and potential outcomes, effectively reducing the gulf of execution.
Perceptibility:
- This refers to the user’s ability to perceive the state of the system.
- Users can easily perceive the status of settings through each switch, the color of the corresponding cookie icon (colored or grayed out), and whether the associated page elements are displayed normally. They can also change these settings at any time by clicking on the widget in the bottom right corner.
Flexibility:
- Wherever possible, we should support the different interactions in which people engage naturally, rather than forcing them into one against their expertise or preferences.
- The new interface does not force users to make choices. For different types of users, such as those who are not concerned about settings and just want to browse the webpage, the interface does not obstruct access to content. Instead, it maintains default settings, allowing users to adjust and change settings later as desired. For those who wish to make detailed adjustments, the interface provides a simple and intuitive way to easily modify each cookie.

8. Final Evaluation

The evaluation of the final prototype was conducted through asynchronous online surveys, with a total of 21 participants. Half of the participants were my family members and friends, while the other half were students from the HCI course. I attempted to locate the same group of individuals who filled out the previous survey (see Appendix 12.11: Final Evaluation Survey).

Q1: The way of pop-up is less disturbing to web browsing.
The average score is 1.15, indicating that most participants believe that pop-up windows are not too disturbing for web browsing. The standard deviation is 0.895, suggesting some degree of disagreement among participants, but overall, there is a certain level of consistency.
Q2: The categorization of cookies was helpful to understand what they do.
The average score is 1.047, indicating that most participants believe that categorizing cookies helps understand their function. The standard deviation is 0.931, suggesting some degree of disagreement among participants, with some possibly believing that categorization does not help much.
Q3: The cookie icons provide a good indication as a signifier.
The average score is 1.1, indicating that most participants believe that cookie icons provide good indicators. The standard deviation is 0.727, indicating relatively consistent agreement among participants regarding the recognition of cookie icons.
Q4: It is easy and intuitive to change different settings.
The average score is 1.15, indicating that most participants believe that changing different settings is easy and intuitive. The standard deviation is 0.757, suggesting some degree of consistency among participants, but some may find changing settings not as straightforward.
Q5: I can easily tell what the current state of the setting is.
The average score is 1.15, indicating that most participants believe they can easily know the current state of settings. The standard deviation is 0.863, suggesting some differences in the perception of the current setting state among participants, but overall, there is a relatively consistent understanding.
Q6: Overall, I prefer the new interface over the cookie pop-ups.
The average score is 1.1, indicating that most participants prefer the new interface over cookie pop-up windows. The standard deviation is 0.612, suggesting a relatively consistent viewpoint among participants regarding overall preference.

8.1 Advantages and Disadvantages

Based on user comments, the advantages and disadvantages of the model are as follows. The majority of the comments express positive sentiments, highlighting the convenience, enjoyment, and effectiveness of the features provided. However, there is a minor concern regarding the granularity of displaying individual cookies.

Pros:

Convenience of having many options for choosing.
Enjoyment and ease of interaction with pop-ups.
Appreciation for the categorization of cookies and understanding their purposes.
Positive feedback on the UI’s ability to communicate effectively while remaining unobtrusive.

Cons:

Criticism of too much granularity in displaying individual cookies, suggesting a preference for a more streamlined approach.

8.2 Suggestions from Participants

The suggestions participants left can be summarized as follows. The comments reflect a variety of perspectives, including usability preferences, technical considerations, user habits, and suggestions for improvement in the prototype’s design and functionality.

Preference for Browser Extension: Some participants suggest that having the prototype as a browser extension would be more convenient or preferable for them.
Developer’s Perspective: One participant mentions the potential challenges from a developer standpoint, expressing concerns about the difficulty of tracing individual UI elements to cookie use.
Lack of Attention to Cookie Settings: Another participant admits to not paying much attention to cookie settings and usually choosing less risky options, making it difficult for them to fill out forms that require attention to cookie content.
Curiosity about Cookie Icon Location and Dark Mode: One participant expresses curiosity about the factors determining the location of the cookie icon and suggests implementing a dark mode for the settings pop-up for better user experience.

8.3 Conclusion

Overall, the final prototype integrates the strengths of the first and third models from the previous round and has received positive feedback. Participants appreciate its simplicity, intuitiveness, and ease of use. However, experienced experts have raised practical issues, such as the granularity of cookie options and challenges for developers. If there is another round of development, considering perspectives from website operators and developers may yield different insights.

Questions	Strongly Agree	Agree	Neutral	Disagree
3. Less disturbing	5	12	2	2
4. Categorization is helpful	7	9	4	1
5. Icons are a good indication	4	10	3	4
6. Intuitive to change settings	9	7	3	2
7. Easily tell what the current state	5	10	4	2
8. Prefer the new interface	6	12	2	1