Background
Challenge
Solution
My Role
Misinformation refers to false or inaccurate information that is spread often unintentionally, misleading individuals, and potentially causing harm [7]. Various methods can combat misinformation by helping users identify potential misleading or false content, such as misinformation warnings. Despite these efforts, current methods often rely heavily on visual cues. This makes them inaccessible to screen reader users.
Screen reader users often face unique challenges when determining a website’s validity due to the shortcomings of screen reader technologies. Users should be able to confidently distinguish false information for guaranteed access to reliable information.
My team and I researched factors that influence screen reader users' judgments about website reliability. We conducted scenario-based observational studies with three blind or low vision (BLV) participants by presenting one legitimate and one illegitimate website for comparison.
The factors found would inform design recommendations for a technology-based solution.
UX and Product Designer
1
Understanding Accessibility
The Accessibility Considerations in HCI course, began with an in-depth curriculum about the needs of diverse users. We focused on people with physical and/or cognitive disabilities and/or people who are elderly. I was tasked to investigate the challenges of designing technology for the needs of users with physical and cognitive disabilities. This included analyzing web pages for current standards of accessibility and concepts surrounding inclusive design, e.g., assistive technologies, universal design and usability, and ability-based design.
2
Research - First Steps
Our research aimed to address the gap in understanding how screen reader users assess the reliability of online content. We studied the key challenges faced by these users through scenario-based observations and follow-up interviews. By observing how participants navigate websites and make judgments about reliability, we identified the factors influencing their decision-making processes. The insights from these observations informed the design of new technologies and methodologies. These will better support screen reader users in distinguishing between factual and misleading information online.
3
Methods
Our team recruited three blind low vision (BLV) participants through a participant pool. The email contained an accessible consent form for participants to review before the meeting.
Participant 1
Age: 63
Gender: Male
Highest degree: Masters
Occupation: Previously an Accessibility Tester
Years using a screen reader: 30
Internet use: Daily
Participant 2
Age: 35
Gender: Male
Highest education: Masters
Work: Employed full-time as an attorney
Screen reader usage: Over 15 years
Internet usage: Daily
Participant 3
Age: 31
Gender: Female
Highest Education: Bachelors
Work: Independent contractor as Screen reader website tester
Internet use: Daily
I noted that the participants had a very similar background and are not the most representative sample of screen readers.
Observation Protocol
Our team conducted a scenario-based observation to understand participant’s reasonings about the validity of a website. We presented two websites, one legitimate website and one illegitimate website, verified by an external website. Our team chose two websites using the Media Bias Fact Checker website. Media Bias Fact Checker ranks websites on their bias, their factual reporting, and their credibility. We chose http://www.climate.news as our illegitimate website. Media Bias Fact Checker ranked Climate.news “very low” on factual reporting, categorizing it as a Conspiracy or Pseudo-Science webpage.
The second website should contain verified information supported by evidence and cannot be well-known or popular. Participants may have predetermined opinions on its validity due to prior knowledge or exposure. The website https://www.lawfaremedia.org/ met our criteria. Media Bias Fact Checker ranked it as ‘very high’ in factual reporting and categorized it as a ‘least biased’ website.
4
Findings
Our team organized our findings into four main categories for assessing website validity: Accessibility and Page Structure, Text Content, Website Ethos, and Participant Characterization.
Accessibility and Page Structure
Participants sought various accessibility features as an indicator for website legitimacy. These include a semantic heading structure, “Skip to Main Content” link, and ease of navigation when using a screen reader. However, participants glossed over some inaccessible elements, such as unlabeled images, due to their previous experiences with inaccessible yet legitimate websites. Participants also cross-referenced other website structures as a baseline for comparison. When discussing their experience with both websites, participants expressed navigating the valid website more easily than its counterpart.
Text Content
Participants associated more opinion-based titles and news articles as less legitimate. They found text-based content with grammatical errors, as apparent in the illegitimate website, less trustworthy. Participants also cross-referenced from other sources or their prior knowledge of current events when reading article titles.
Website Ethos
As one of the first indicators of legitimacy, participants trusted websites with a labeled logo more than ones without. Participants also felt safer when browsing websites with more common domain names (such as .com, .org, and .edu) as opposed to lesser-known ones (such as .news). Within each website, they sought links to recognizable social media platforms, such as Facebook and X (formerly Twitter). All participants searched for an About page to associate the company or organization with a sense of credibility. Participants assessed the textual content for any opinion-based or outlandish claims.
Participant Characterization
Participants spent more time on the non-trusted website than the trusted one. When assessing their confidence on identifying trustworthy websites, participants rated themselves between a 4 or 5 (very highly). All participants considered themselves accessibility experts and used the internet daily.
5
Evaluation
Our team organized our findings into four main categories for assessing website validity: Accessibility and Page Structure, Text Content, Website Ethos, and Participant Characterization.
Design and Navigation:
-
Users rely on semantic heading structure and easy-to-navigate pages
-
Website architecture with these features increases confidence
Attention to Detail:
-
Users carefully read content, noting misspelled words and biased headings as indicators for illegitimacy
-
Common domain names and labeled logos increased participant’s trust
Evaluation Consistency:
Order of website presentation did not affect assessment abilities
Confirmation from Previous Studies:
-
Findings are consistent with other research on blind users' website credibility evaluation.
-
Align with Heuer and Glassman's study on manipulable criteria for news website reliability: content, professional standards, sources, and reputation.
Call for Tailored Technology Solutions:
Further studies are needed to improve accessible tools for determining website legitimacy.
6
Conclusion
During my interview, I observed my participant’s process of analyzing web content. Viewing first-hand how screen reader users view content has afforded context to accessibility problems today. The experience also provided knowledge on interviewing people that are Blind or Low Vision. The use of accessible consent documents, sending information by email beforehand, and asking for verbal consent, have all been part of that lesson.
The findings reflected the need for semantic heading structure, carefully written content, and verifiable companies behind websites. These findings aid in my understanding of designing for screen-reader users and in creating websites that reflect trustworthiness.
Limitations
Interviewed only three screen reader users, therefore the results cannot be generalized.
-
Participants:
-
Native English speakers
-
Extensive screen reader usage
-
“Accessibility experts”
-
At least a bachelor’s or master’s degree
-
-
Reliance on self-reported data may introduce biases such as social desirability and recall errors.
​
Future Work
-
Include a more diverse sample.
-
Combining qualitative observations with quantitative measurements could provide a better understanding of users' preferences.
-
Focus on creating systems offering real-time feedback on the trustworthiness of online content.
-
Collaboration among researchers, designers, and individuals with disabilities is crucial for ensuring inclusivity in design.