Remote communication as a new norm
During the COVID-19 pandemic, work is not the same. As a Forbes article by Nancy Doyle pointed out, as more people have been required to work from home, it debunks the myth that it is ‘too hard’ for people to work remotely. In this way, the pandemic reinforces the concept of the social model of disability. That is – ability and access is often influenced by society and systems. These systems and policies, such as allowing remote work, or flexible timing, can create greater access.
There is diversity and intersectionality in disability. Some people may prefer working from an office, others may prefer or need to work remotely. Remote, or virtual usability testing is not new. What is new is that in the context of a pandemic, it has become the status quo. Remote communication is the new norm.
For World Usability Day 2018 we wrote about the importance of including people with disabilities in usability testing. The importance of this has not diminished, with the shift to virtual spaces. If anything, it has increased.
We both recruit and run usability sessions with people with disability through Intopia Connect. Based on our experiences, we want to share some quick tips.
Tips for virtual and in-person usability testing
1. Fix known accessibility issues first
As with face-to-face sessions, fix the already identified challenges before running usability testing. Known issues may come in the form of complaints that have been received by people who use the website, system or app. Identified issues may also be those that relate to the Web Content Accessibility Guidelines (WCAG). Examples include:
- a lack of headings
- poor link text (read more, read more, read more)
- a keyboard trap (where someone using a keyboard only can’t move beyond a particular component in your website) or general lack of keyboard focus
- missing alternative text (‘alt text’) on non-decorative images
- lack of colour contrast
Fixing known issues first can reduce frustration of having information that is presented in an inaccessible way in the feedback sessions.
2. Include people with disabilities across usability testing (UX)
Create a test plan and consider the participant diversity. Involve a representative set of users, and people of different ages, with diverse interests and people who use technology in different ways. When running usability testing with people with diverse backgrounds, consider intersectionality.
If you are running research sessions with women, include women with disabilities. When you are conducting UX feedback sessions with people who have cultural and linguistically diverse backgrounds (CALD), include people with disabilities with CALD backgrounds.
Around 18 per cent of Australians have disabilities. As a starting point, ideally include at least one in five participants in standard testing and research who:
- use screen reading technologies such as JAWS, NVDA, Talkback, VoiceOver
- have low vision and may use ZoomText, zoom in general or high contrast
- are colour blind
- are neurodiverse, have dyslexia, dyscalculia, autism or ADHD
- are hard-of-hearing or Deaf and use captions or sign languages such as Australian sign language (Auslan) or New Zealand Sign Language (NZSL)
- have limited dexterity, or use Dragon Naturally Speaking
- have psychosocial disability such as anxiety or PTSD
- are deafblind and use braille displays and other communication tools
Jakob Nielson recommends in a March 2000 article, three rounds with five people each to validate, and iterate depending on the purpose of the research. The Australian-based UNSW Disability Innovation Institute launched 2020 Guidelines for Co-Producing Research with People with Disability in August.
3. Be flexible with timing
When booking in usability testing times, be flexible. Recognise not just the availability of the facilitator(s), note taker(s) but also timing for participants.
Check if there are better times for people to be uninterrupted. This is especially important if family or housemates are sharing workspaces while people are working from home.
Ideally provide some flexibility for days and times:
- Some people may be working varied hours or shifts
- Some people may need longer in the morning to get ready
- In the case of face-to-face meetings, avoiding peak hour can be better to arrange an accessible taxi or if taking public transport with a guide dog
- Some people may also have less energy in the mornings or afternoons
Make sure that the app or website that is needing the usability testing doesn’t have any planned upgrades around the time of the usability testing sessions.
4. Choose accessible spaces
With face-to-face sessions, make sure that the usability session is held at an accessible location or venue. This includes having as a minimum:
- an accessible entrance
- stair-free access to meeting rooms, such as ground floor meeting rooms or access via an elevator
- an accessible bathroom
- ideally close to public transport and to an accessible parking space
5. Use accessible communications and technologies
Send consent forms or information about the usability testing sessions in an accessible format to participants beforehand.
For face-to-face sessions, provide information about getting to the venue, in an accessible format.
For remote sessions, choose accessible technologies. Some technologies have presented challenges or opportunities for accessibility for people with screen readers or needing captions. Smashing Magazine published an article in June 2020, titled “Which Video Conferencing Tools Are Most Accessible”. The article lists some of the different accessibility features of various video conferencing platforms, such as Zoom, Google Meet, Webex, Slack, Microsoft Teams.
For accessibility and functionality of the software for usability testing, it is important that diverse participants with disabilities can independently:
- Dial in or access the session
- Turn on and off mute
- Turn on and off video
- Screen share
- Access accurate captions
Consider and use screen-sharing techniques that support accessibility. See an article by Allison Ravenhall, accessibility sensei, on screen-sharing from mobile phones. For VoiceOver on iPhone we found participants can use a Bluetooth speaker or Bluetooth headphones to share announcements during virtual usability testing.
Ideally allow time to test the technologies in the lead-up to the sessions, rather than directly beforehand. This can:
- Provide time to fix any issues you may encounter
- Reduce the potential for extra stress if changes need to be made
6. Arrange a practice session or meeting in advance
Book in a practice session to test the technologies, including the web conferencing software. Provide guidance on the shortcuts and options for accessibility in the conferencing platform in advance.
An advance practice session will allow you to understand the assistive technologies, or adaptive strategies that individuals are using. It also means you can confirm that settings can be adapted if needed. For example, sometimes participants may need to be asked to reduce the speed of their screen-reader during the usability session so the facilitators can follow the announcements.
The advance meeting provides a chance for participants to practice using the selected web conferencing platform, or to test out various features. For example, being able to share-screen comfortably, or to take off screen-curtains (which blacks out a screen for people who are blind, using a screen-reader for greater privacy). This will also provide a chance to get to know the participant in advance.
7. Run accessible usability testing
Before the start of the session, make sure participants turn off notifications and hide or remove private or sensitive documents especially before sharing screen to protect privacy. Confirm consent for participation and recording.
Have the note-takers or observers turn off their video after the initial introductions to avoid distractions. Allocate time for observers to ask questions at the end.
Cognitive fatigue is more common in online discussions compared with face-to-face sessions. Ideally, sessions should only run for an hour. If sessions need to be longer, make sure there are breaks within the schedule. In addition, provide the option of taking a break or stopping at any time.
8. Wrap up
After the usability session ends:
- save recordings locally
- thank everyone and acknowledge time and patience.
- ask for feedback
Feedback is crucial as a standard principle, as usability testing and experience provides excellent learning opportunities.
These tips are provided based on some experiences in running usability testing and lessons learnt, which we hope will help. If you have suggestions of your own, please contact firstname.lastname@example.org.
If you would like to sign up for paid opportunities to give feedback on accessibility and usability, please register on our Intopia Connect site.
For World Usability Day we wrote an article on the importance of usability testing including people with disabilities. The next World Usability Day will be held on 21 November 2020.