Involving Users in the Development of a Web Accessibility Tool
The European Union (EU) is increasingly focused on design for all issues and ensuring that access to information and telecommunications meets the needs of all people in order to address the digital divide and create an information society for all. This includes the estimated 37 million people with disabilities in the EU, as well as other groups who could face barriers to e-inclusion such as older people and people with access limitations. This could include users of alternative devices (e.g. PDAs) or with limited bandwidth. Member States of the EU are required to take the needs of disabled people (including access to information) into account according to Declaration 22 of the Amsterdam Treaty [1]. Advice and guidance is available, such as Europe's Information Society Thematic Portal [2] which provides information to raise awareness of the potential barriers to access and how they can be easily avoided if Web site designers follow Web accessibility guidelines.
EU activities to support Web accessibility and design for all include:
- A recommendation for a European curriculum for designers and engineers through the eEurope initiative.
- The adoption of the W3C/WAI Web Content Accessibility Guidelines (WCAG) for public Web sites in the EU.
- The EU Research Framework Programme supports research and development efforts in this area. This includes the Information Society Technologies Programme Priority 2 (FP6/IST 2003-2006), which has funded the European Internet Accessibility Observatory (EIAO) Project [3] together with two other 'cluster' projects SupportEAM [4] and BenToWeb [5].
This article will focus on the European Internet Accessibility Observatory Project (EIAO) which began in September 2004 and is being undertaken in co-operation and partnership with industry and service providers and brings together the following institutions from 6 countries across Europe:
- Agder University College, Norway (Co-ordinating partner).
- Vista Utredning AS, Norway.
- FTB-Volmarstein, Germany.
- Manchester Metropolitan University, UK.
- Nettroken AS, Norway.
- University of Tromsø, Norway.
- FBL, Italy.
- Technical University of Warsaw, Poland.
- Aalborg University, Denmark.
- Intermediaum AS, Norway.
The goal of the EIAO Project is to contribute to better e-accessibility for all citizens and to increase the use of standards for online resources. The project will establish the technical basis for a possible European Internet Accessibility Observatory. A collection of Web accessibility metrics (WAMs), based on the checkpoints developed by World Wide Web Consortium for the Web Content Accessibility Guidelines (WCAG) version 1.0 [6] (with potential migration to 2.0), and the tools for automated data collection and dissemination will be developed and continuously improved upon throughout the project by feedback from users and user testing to sharpen the relevance of the automatically collected data. The work is carried out in collaboration with the cluster projects (BenToWeb and Support EAM) and in co-ordination with the World Wide Web Consortium Web Accessibility Initiative (W3C/WAI).
The European Internet Accessibility Observatory will consist of the following elements:
- A set of Web Accessibility Metrics (WAMs) based on the WCAG 1.0 checkpoints (with a view to conversion to version 2.0 when appropriate).
- An Internet robot (ROBACC - ROBot assessing web ACCessibility) for automatic and frequent collection of data on Web accessibility and deviations from Web standards (i.e. the WAI guidelines)
- A data warehouse (ROBACC DW) providing online access to collected accessibility data.
All software developed in the project will be released under an open source licence.
The combined elements of EIAO (i.e. WAMs, ROBACC, ROBACC DW) illustrated below in Figure 1, will be referred to as the Observatory throughout this article.
Figure 1: Elements of the European Internet Accessibility Observatory
(The relations among the elements indicate how user testing is planned to improve the automatic evaluation.)
The overall aim is that all kinds of Web sites should be evaluated for accessibility, but the focus initially is on EU public services, governmental and local authorities, and private service providers such as banks and public transport. By providing frequently updated data on Web accessibility metrics, as well as deviations from standards, the Observatory aims to:
- Provide quantitative background for policy-making and targeted actions to improve Web accessibility. This may be of special importance for the discussion going on about legislation (anti-discrimination laws) for equal rights related to Web accessibility.
- Promote e-accessibility by raising awareness and encouraging competition through benchmarking of accessibility metrics and deviations from standards.
- Provide background for the discussion of standards and the development of new standards.
- Contribute to the Lisbon strategy of making Europe the World's most competitive knowledge-based economy by 2010.
EIAO is funded by the EU Sixth Framework Programme, Priority 2: Information Society Technologies. The project is carried out as part of a Web Accessibility Benchmarking cluster together with projects Support-EAM and BenToWeb. The three projects are working together to develop an EU-harmonised Web evaluation methodology based on the W3C/WAI Guidelines.
User Requirements Data Gathering
By involving users throughout the EIAO Project, it will be possible to use the findings from each user group to influence the technical development of the project in terms of interface and schema design. Also, by identifying statements provided by the users, a clearer picture will be provided of what users actually want to get from the Observatory. Figure 2 illustrates the involvement of users (through requirements gathering and user testing) throughout the project development cycle:
Figure 2: EIAO Development Cycle
At present there is little evidence of user involvement in the development of standards and guidelines for Web accessibility, or for the development of automated tools for testing. However it is becoming increasingly evident that to be able to create Web sites that are accessible to a greater number of people, with a greater number of needs and requirements, it is essential to include users in the process from the start. The work that has been undertaken tends to focus on the testing of specific Web sites or features provided on a Web page (such as the search function, or login process) rather than on the gathering of user requirements, perceptions and attitudes to the Web.
Studies which consider more generally the needs of users provide interesting results. For example, the Disability Rights Commission study of the accessibility of Web sites [7] revealed that all categories of disabled user find Web site designs take insufficient account of their specific needs. Another example is an evaluation of an online database conducted by House et al [8], where the team made initial assumptions relating to what user requirements would be, but the assessments revealed a variety of needs which differed between groups of users (library staff, information officers, lawyers etc) thus indicating that requirements can and must be defined more specifically, and that they vary from group to group.
The EIAO Project has set out to identify more generally what users want from an accessible Web site and the problems they have encountered. Having established general requirements and problems experienced, the project has compared and identified relevant checkpoints based on the WCAG, which are being used to develop the WAMs for the Observatory.
To identify user requirements, two main groups of users were identified: end-users and stakeholder groups:
- End-users were identified as users who could benefit from accessible content and for whom accessibility may be an issue. This could include people with disabilities, people who use specific devices such as a PDA or mobile phone and people who have other access limitations such as limited bandwidth.
- Stakeholders were identified as groups of users such as Web developers, policy makers, and Web site owners.
The methods chosen to identify the requirements of end-users and the stakeholder groups were a combination of questionnaires and follow-up interviews. Drawing on experiences from other surveys where significant response rates have been difficult to achieve, the project focused on obtaining richer, more qualitative data rather than large-scale quantitative results.
This approach will provide the Project Team with 'statements' on accessibility which will help with the development of the first iteration (and possibly further iterations) of the ROBACC WAMs.
End-users
Around 500 questionnaires were distributed to potential end-users. Responses were received from 109 people and follow-up interviews conducted with 30 people. Although it was not possible to ensure a truly representative spread across age range, gender, and disability group, genders were well represented and a reasonable spread across the range of ages achieved with the exception of people over the age of 65. The disability groups identified for the study were all represented (e.g visual, hearing, physical, cognitive/learning, access), but a greater number of responses were received from people with visual and/or physical disabilities. This may be due to several factors, such as the design of the questionnaire, method of returning responses or simply that people with visual or physical disabilities felt more strongly about Web accessibility.
Stakeholders
Distribution took place to an initial sample of 50 organisations identified as potential stakeholders. Respondents were asked at the end of the questionnaire if they would be willing to take part in a follow-up interview.
The initial number of responses was low, therefore a decision to undertake further distribution was made. This was done via relevant email lists (e.g. EdeAN, CETIS, TechDis) and via the Europa Information Society News Thematic Portal. As a result of this extra distribution, the response rate increased to a satisfactory level for the purposes of the project. It was particularly pleasing that many respondents provided comments to back up their quantitative responses, which then provided the Project Team with a better insight into their requirements.
The data gathered from the questionnaires was explored further through follow-up interviews. From the 43 questionnaire responses received, 23 agreed to take part in the interviews. This work is ongoing at the time of writing.
End-User Requirements
The first phase of gathering feedback on what users want from an accessible Web enabled the Project Team to identify the main accessibility problems experienced by users, which could then be compared with the Web accessibility metrics identified for inclusion (based on WCAG 1.0). At present 20 WCAG checkpoints (priority levels 1, 2 and 3) have been identified for inclusion (although 3 may be withdrawn because they are likely to be deprecated in WCAG version 2.0).
Main Accessibility Problems
The results revealed keyboard access (shortcut keys, tab navigation and/or keyboard navigation) to be the most frequently mentioned accessibility problem, followed by problems either with lack of ALT text or poor use of ALT text. Users also mentioned problems relating to the organisation of the page, an inability to navigate the site, poor use of titles for Web pages, and confusing use of language such as acronyms and abbreviations that are not fully explained.
Users indicated they felt excluded from a number of elements found in Web pages such as images, multimedia and forms. This was illustrated further during the interviews where users talked about problems accessing Flash, PDF, and JavaScript, and how complex forms could be tiring to complete with added accessibility problems particularly when using a screen reader.
When asked to suggest ways in which to improve accessibility, the most popular suggestion (from a list of possibilities) related to the organisation of the page or site - 'a clear design with menus'. Appropriate markup (e.g.Titles and Headings) to help inform the user about the content of the page were mentioned quite frequently, as were the design of forms that are easy to navigate and to complete. Other accessibility issues mentioned by the users related to slow download times when not on broadband, and having to register on some Web sites before being allowed to access the information on the page.
Comparison with WCAG
The user requirements identified in this study have been compared with the WAMs and the corresponding Web Content Accessibility Guidelines (1.0) and each problem identified by the respondents has been linked to a potentially relevant checkpoint. Overall, nine Priority 1, twenty-one Priority 2, and eleven Priority 3 checkpoints were identified as being of relevance to accessibility problems identified by the respondents either from questionnaire responses or via the follow-up interviews.
The fact that more accessibility issues can be related to Priority 2 of the WCAG (version 1.0) suggests that there are differences between some users' needs and their perceptions of accessibility, and the formal recommendations such as those produced by the WCAG. It also suggests that some aspects of Priority 1 checkpoints are either not as relevant to real users or are being handled well by Web developers. Further investigation of this aspect is needed.
Some of the problems indicated by users could not be identified in the WAMs proposed for the first iteration. This situation will be monitored over each iteration and during the user testing stage of the project (to be undertaken in late 2005).
Stakeholder Requirements
Accessibility Awareness
The findings reported in Section 3 suggest stakeholders are in a position that would be open to the services offered by the Observatory because they have an understanding of the importance of accessibility issues and of methods available to help and guide them towards creating better Web sites. However, it cannot be assumed that all stakeholders will have this level of awareness as some respondents commented that people they have to liaise with (e.g. managers, policy makers, externally appointed Web designers) are often less aware and therefore need to have the information they receive tailored to their needs.
Despite the fact that the majority of stakeholders surveyed showed awareness of accessibility issues and were in support of 'design for all' principles, not all were actively involved in creating accessible Web sites. This was often dependent on the nature of their organisation and work (i.e. customer-driven, resource-driven, outsourcing etc). Stakeholders who were involved in addressing accessibility (through design, liaison, advice, etc) cited a number of tools used to help them design accessible content, the most popular methods being the use of guidelines and standards. Respondents who said they regularly checked their sites for accessibility cited a combination of tools (automated, manual, external audits) rather than any one particular tool.
Accessibility Checking
Respondents were asked what accessibility data they might find useful for their work, and were given some options to comment on. Comparisons and benchmarking prompted mixed responses. While generally in favour of being able to compare the accessibility of their Web site with other similar organisations, and to be able to have benchmarks of best practice for comparison with their Web sites, respondents had concerns about the methods used to benchmark - in particular the sole use of automated tools, or the accessibility criteria used.
Respondents were very positive about the usefulness of Web accessibility indicators, of monitoring of accessibility (e.g. of a Web site) over time, and of examples of good practice. They had more reservations about automatically collected data on accessibility - mainly because they associated this with automated testing and were unsure of the capability of automated tools.
They were very positive about receiving extra information such as suggestions on how to repair faults, also a ranked list of improvements needed to make their site more accessible. They were also positive about being able to integrate manual and automated tests (if this could be done), and of receiving a regular accessibility report about their site.
When asked about accessibility checks and accessibility reports, respondents in favour of this type of data said they would like to be able to configure the depth or level themselves, according to circumstances and to their target audience. Respondents also said it would be useful to have the answers to more technical questions, as well as being able to assess the compatibility of their Web sites with different assistive technologies.
Respondents also mentioned the need to raise awareness about accessibility, so that people know why it is important, not just what needs to be done. Many talked about the importance of user testing and user feedback, in particular because of the limitations that were identified in respect of automated testing.
Issues for Consideration
Overall, respondents were very positive about the Observatory. The main issues raised by respondents which require further consideration related to the limitations of automated testing and the difficulty of including users in this type of data collection. This is something the Project Team will need to address, based on the user testing sections of the Unified Web Evaluation Methodology (UWEM). The work of the UWEM should also help to address concerns over the accuracy and reliability of automated testing.
In relation to legal issues that the Project Team may need to consider, the main concern related to making claims about accessibility levels that might be used in a legal case, or with which organisations do not agree. One useful suggestion was for the Project Team to include some kind of disclaimer on the site, clearly stating the scope of the Observatory.
Conclusion
These findings show the current situation and defines the work needed for the project to progress. Responses from the end-users have provided an initial picture of end-user requirements and their perceptions of accessibility, which can be fed into development of the WAMs, the Cluster work and other relevant EIAO project work.
The questionnaires were important to establish general levels of awareness. However they also identified the main difficulties experienced by developers, owners, and decision makers when dealing with accessibility in their context which would lead to the development of a tool that will meet their requirements.
The follow-up interviews were important to identify any concerns users may have with such an Observatory. The Project Team will therefore take great care to give all opinions close consideration.
The next stage of user involvement will take place after the first release of the Observatory software. This will comprise the validation of the software through a series of user testing and validation exercises. This stage of the project is scheduled to take place towards the end of 2005 and will run over several iterations according to each release of the software. Further iterations of user requirements gathering are also planned after each release of the software, providing invaluable feedback at each of the development stages.
The EIAO Project is co-funded by the European Commission under the IST contract 2003-004526-STREP. Further details are available on the Project Web site [3].
References
- The Amsterdam Treaty http://europa.eu.int/abc/obj/amst/en/
- Europe's Information Society Thematic Portal http://europa.eu.int/information_society/index_en.htm
- The European Internet Accessibility Observatory http://www.eiao.net/
- SupportEAM Project http://www.support-eam.org
- BenToWeb Project http://www.bentoweb.org
- World Wide Web Consortium Web Content Accessibility Guidelines 1.0 http://www.w3.org/TR/WAI-WEBCONTENT/
- Disability Rights Commission (DRC). The Web: Access and Inclusion for Disabled people. A formal Investigation conducted by the Disability Rights Commission. London: DRC, 2004.
- House Van, Nancy A. et al. User-Centered Iterative Design for Digital Libraries: The Cypress Experience, February 1996, Dlib Magazine http://www.dlib.org/dlib/february96/02vanhouse.html