Main content (Site structure)
Web accessibility assessment Tool – WaaT
Introduction & Information details for the WaaT Tool
The Web Application Assessment Tool is a tool (Web and Standalone version) for the accessibility verification of Web applications. WaaT supports developers and designers and accessibility analyzer in test existing, or under development, web sites and web applications. It provides users with the ability to define some input parameters, rules or constraints regarding the accessibility evaluation (e.g. evaluate a web page according to a specific disability or a particular existing guideline). WaaT helps developers and designers to find and correct errors or bad configuration which prevent impaired users to be able to correctly consume the web application. The product has been entirely developed within the research activities held during the ACCESSIBLE project.
The assessment tool, in both version standalone or web module on ACCESSIBLE portal, has been designed and developed to take as input the URL (or local) path of the web page or directly the html source code and to produce as output the evaluation results of the accessibility assessment process.
Every aspect of a web application is taken into account during the evaluation of the accessibility assessment process:
- Complete site parts are crawled automatically by the tool (maximum number of pages can be specified by the user as a parameter of the process).
- The markup code, both HTML and XHTML, is validated using the W3C Markup Validator which is embedded in the WaaT.
- HTML is parsed to obtain detailed information on elements and attributes which are used to compose the pages.
- ARIA UI components are validated in order to make sure that they are well described and that their role and value labels are correctly identified. This is very important as most of the new web pages include JavaScript UI components and to expose the accessibility information these components should include the ARIA markup.
- The CSS are validated by the W3C CSS Validator embedded in the WaaT.
- All the information is collected and used to evaluate the accessibility of the examined pages using the accessibility requirements and constraints specified by the user.
The WaaT consists of the following six main components which are described with more details in the public deliverables D3.3 and D5.2 and are being presented in the figure 1 below:
- 1. A Graphical User Interface (GUI):The GUI of the WaaT tool consists of a set of forms that can be accessed by users, in order to help them with the accessibility assessment of preferable web applications. The GUI of the assessment tool is also responsible for the representation of the assessment results after the completion of the evaluation process. Via the GUI, the user is able to select the preferable tests to be executed using knowledge concerning the HAM methodology, which is included the ACCESSIBLE ontologies. Moreover, using the GUI, the user can load a Virtual User Model, on which the evaluation process will be based. After the completion of the evaluation process, the Web Accessibility Evaluator returns the results to the GUI, so as to be presented to the user.
Figure 1: WaaT architecture overview
- 2. The Rules Inference Engine: The Rules Inference Engine is responsible for the communication between the WaaT and the ACCESSIBLE ontologies where knowledge regarding the HAM methodology is stored. The Rules Inference Engine is able to run SWRL rules as well as execute SPARQL queries, in order to extract specific knowledge from the ontology.
- 3. The XML Storing/Loading Module: As the execution of SWRL rules defined in the ACCESSIBLE ontology is generally a time-consuming process, an XML storing/loading module is introduced. This module is able to automatically generate an XML file containing all the necessary knowledge of the ontology that is required for the assessment process. After the generation of the XML document the XML Storing/Loading module is responsible for the “virtual connection” between the WaaT tool and the ontology.
-
4. The Core Web Applications Assessment Module: The Core Web Applications Assessment Module is the core component of the WaaT tool and includes all the required algorithms and methodologies for the execution of preferable user’s selections. This module consists of the following six sub-components:
- i. The Web Crawler, which returns the single pages of a web site, when an entire web site is being selected to be evaluated.
- ii. The W3C Markup Validator , which evaluates the conformance of a web page against the technical specifications of HTML and XHTML.
- iii. The HTML Parser that parses the web page source code and returns the necessary information concerning the desired elements/attributes of the HTML/XHTML.
- iv. The W3C CSS Validator, which evaluates the conformance of a CSS file against the CSS 2.1 Specifications.
- v. The CSS Parser that is responsible for the parsing of all the CSS files that are connected with the examined web page.
- vi. The Web Accessibility Evaluator, which is the core component of the Core Web Applications Assessment Module performing the accessibility evaluation of a web application, according to the user’s preferences.
-
5. The Reporting Module: The Reporting Module of WaaT is responsible for the generation of various types of reports including the results of the evaluation process. The supported report types include the following:
- A PDF report containing all the errors and warnings of the evaluated web page(s). For each error/warning, a description, a tip for its fixation and the corresponding HTML source code (where the violation appears) are provided. Moreover, the success criterion and the technique of WCAG 2.0 (where the error/warning refers to) is also provided.
- An EARL-based report of the detected errors and warnings. EARL is a standard format for support of the evaluation of Web applications. It contains a vocabulary to describe the results of a test’s execution, in order to facilitate its processing and interpretation by different tools. EARL is expressed in RDF, which can be extended easily and be adapted to any domain, as in this case accessibility.
- A PDF version of the EARL-based report, containing all the information of the EARL-based report in human readable format.
- A report containing all the HTTP content transferred between the WaaT and the server that hosts the evaluated web application, using the HTTP Vocabulary in RDF. This report can be used as input to other web accessibility evaluators, in order to perform a similar use case scenario and compare their results with those of WaaT.
Owner/Main Developer (including also key partners contribution)
CERTH/ITI, FORTH (implemented the User Interface of the online tool)
Intelectual property Rights (IPR issues, Licences)
MIT licence
Nature of the Code
Open source
Published Papers
- Lopes, R., Votis, K., Carriço, L., & Likothanassis, S. A Service Oriented Ontological Framework for the Semantic Validation of Web Accessibility. In M. Cruz-Cunha, E. Oliveira, A. Tavares, & L. Ferreira (Eds.), Handbook of Research on Social Dimensions of Semantic Technologies and Web Services, IGI Global, Hershey, 2009, 49-67, DOI=10.4018/978-1-60566-650-1.ch003
- Lopes, R., Votis, K., Carriço, L., Tzovaras, D., and Likothanassis, S. Towards the universal semantic assessment of accessibility. In Proceedings of the 2009 ACM symposium on Applied Computing, (Honolulu, USA, 2009), ACM, 147-151, DOI=10.1145/1529282.1529311
- Lopes, R., Votis, K., Carriço, L., Tzovaras, D., and Likothanassis, S. The semantics of personalised web accessibility assessment. In Proceedings of the 2010 ACM Symposium on Applied Computing, (Sierre, Switzerland, 2010), ACM, 1440-1441, DOI=10.1145/1774088.1774394
- Votis, K., Lopes, R., Tzovaras, D., Carriço, L., and Likothanassis, S. A Semantic Accessibility Assessment Environment for Design and Development for the Web. In Proceedings of the 5th International Conference on Universal Access in Human-Computer Interaction. Part III: Applications and Services, (San Diego, USA, 2009), Springer-Verlag, 803-813. DOI=10.1007/978-3-642-02713-0_86
- Partarakis, N., Doulgeraki, C., Antona, M., Oikonomou, T., Kaklanis, N., Votis, K., Kastori, G.E., and Tzovaras, D. A unified environment for accessing a suite of accessibility evaluation facilities. In Proceedings of the 6th international conference on Universal access in human-computer interaction: design for all and eInclusion - Volume Part I, (Orlando, USA, 2011), Springer-Verlag, 267-275.
- Fernandes, N., Lopes, R., and Carriço, L. An architecture for multiple web accessibility evaluation environments. In Proceedings of the 6th international conference on Universal access in human-computer interaction: design for all and eInclusion - Volume Part I, (Orlando, USA, 2011), Springer-Verlag, 206-214.
| Strengths | Weaknesses |
|---|---|
|
|
Contribution to the state of the art
There is a large number of software tools performing accessibility evaluation of web sites based on the guidelines of popular accessibility standards, such as WCAG 1.0, WCAG 2.0 and Section 508. Recently some tools partially supporting the WAI-ARIA guidelines have been also appeared. The most common technologies that are checked include Cascading Style Sheets (CSS), XHTML, PDF, images, Synchronized Multimedia Integration Language (SMIL), and Scalable Vector Graphics (SVG). The automated checking on a single web page is the most common feature supported. However, some tools support evaluation of groups of pages or entire web sites. The report of the evaluation results may include step-by-step evaluation guidance, displaying information within web pages or more formal report types, such as EARL-based reports [1]. Some accessibility evaluators provide also repair functionality by changing the source code of the web pages, helping with captioning audio or video content, or converting document into accessible mark-up.
Many tools, such as the Foxability, WAVE, HERA and Hera-FFX have been developed based on the WCAG 1.0 and 2.0 guidelines. Also AChecker is an open source web accessibility evaluation tool developed by the Adaptive Technology Resource Centre at the University of Toronto. It supports a variety of international accessibility guidelines like Section 508, Ley Stanca (Italy), WCAG 1.0 (levels A, AA and AAA) and 2.0 (levels A, AA, and AAA), and BITV 1.0 (Germany). AChecker presents results in three categories: known problems, likely problems and potential problems. Worldspace FireEyes is a free web accessibility evaluation tool introduced by Deque Systems, Inc evaluating the compliance of a web site according to standards such as WCAG 1 (Priorities 1, 2 and 3), WCAG 2 (levels A and AA), Section 508 and contains some dynamic rules that test for WAI-ARIA compliance. The FireEyse also includes features such as: color contrast analyzer, dynamic report filtering, interactive issue remediation and transcripts of all pages visited in a session. Worldspace FireEyes is fully JavaScript aware and handles event-based page content. It works as a complement of the Firebug Firefox extension.
Total Validator is another accessibility validator supporting WCAG 1.0, WCAG 2.0 and Section 508 standards. It includes a HTML validator, an accessibility validator, a spelling validator, a broken links validator. There is a web version, a Firefox extension, and a desktop version of the tool available. TAW3 is an accessibility validator developed by the Spanish Fundación CTIC. It is available in two versions: a plug-in for Mozilla Firefox and in a standalone version. TAW3 analyses websites according to WCAG 1.0 and WCAG 2.0 guidelines by providing fixes and recommendations. TAW results are presented with different representation of violations (problems, warnings, and not reviewed).
Although there are many existing tools performing accessibility evaluation of web sites, there is a missing point: the personalized accessibility evaluation according to the specific needs/preferences of a user and/or a disabilitiy. This is extremely valuable considering that people with disabilities have often special needs varying from person to person, even if they are having the same disability. The current WaaT tool performs personalized web accessibility evaluation of web application using personas and virtual user models of people with disabilities and elderly.
Technical Testing Results from the developer and beneficiary validation
WaaT tool has been tested in pilot tests with developers and beneficiaries and several important usability issues occurred. In the following table we outline the most important issues that were identified during the testing, and how they were addressed for the improvement of the tool.
| Issue description | Priority | Solved | Solution / Explanation |
|---|---|---|---|
| For Colour-blindness, the colours of the images must be checked as well. | 1 | YES | Fixed. |
| For VERITAS website, there were 6 elements without title attribute, and only 4 were recognised. | 1 | YES | Fixed. |
| It is unclear when a long description for an image is needed or not, as the tool seems to randomly identify it as errors. | 1 | YES | All the images are examined for the existence of "longdesc" attribute. In the latest version of WaaT, an image without a long description is considered as a warning (not as error as it was previously considered). |
| Keyboard accessibility issues were raised, however keyboard access keys are not considered a good practice. | 1 | YES | The developed approaches regarding the keyboard-triggered event handlers have been based on the WCAG 2.0 technique G90 (http://www.w3.org/TR/2012/NOTE-WCAG20-TECHS-20120103/G90.html). The objective of this technique is to permit individuals who rely on a keyboard or keyboard interface to access the functionality of the content. According to this technique, the WaaT tool examines if all event handlers triggered by non-keyboard UI events are also associated with a keyboard-based event, or provide redundant keyboard-based mechanisms to accomplish the functionality provided by other device-specific functions. |
| No ARIA assessment | 1 | YES | WAI-ARIA assessment is included. |
| The analysis of images and the forced presence of links with images. This is wrong. | 1 | YES | As you can see in the following references, the "longdesc" attribute is poorly supported and instead of using "longdesc", it is suggested the D-Link technique to be used. According to this technique, when the developer wants to give a long description of an image, he must put a link with the letter "D" in brackets ("[D]") before or after the image. http://www.w3.org/TR/WCAG-TECHS/G73.html http://www.htmlcodetutorial.com/images/_IMG_LONGDESC.html http://www.w3schools.com/tags/att_img_longdesc.asp http://reference.sitepoint.com/html/img/longdesc#longdesc__fig-ff-longdesc http://en.wikipedia.org/wiki/HTML_element#longdesc So, we've changed a bit the specific approach, according to the following: 1) The specific output result is now considered as a warning (not as an error as it was considered previously) 2) Search for "[D]", not for hyperlink in general. |
| Unsure if the alt="" for background images is detected correctly; unsure if longdesc attribute missing should be considered an error; it is not always clear how | 1 | YES | The "alt" attribute is checked for all the images of the web page. The lack of "longdesc" attribute is now considered as a warning, not an error. |
| For VERITAS website, is provided, but the error states “3.1.1 Provide an "xml:lang" attribute to the Web page, in order to identify the default language of the document”. This seems to be an error. | 1 | YES | Fixed in the latest version of WaaT. |
| I become a NAN result by testing the site with accessible approach as option. | 1 | YES | Fixed in the latest version of WaaT. |
| line numbers seem to be set -1 always, which makes it difficult sometimes to find the error. In the portal version the line numbers seem correct. | 1 | YES | Fixed in the latest version of WaaT. |
| Not always clear how an error is corrected. | 1 | YES | The descriptions of the errors as well as the hints have been corrected/improved. |
| Number of onclick event handlers without redundant onkeypress event handlers is indicated as an error, while it is not an error at all, and an alternative approach is possible, while also keyboard can be used. So this should not be an error (VERITAS website success criterion 2.1.1 line 238). | 1 | YES | The developed approaches regarding the keyboard-triggered event handlers have been based on the WCAG 2.0 technique G90 (http://www.w3.org/TR/2012/NOTE-WCAG20-TECHS-20120103/G90.html). The objective of this technique is to permit individuals who rely on a keyboard or keyboard interface to access the functionality of the content. According to this technique, the WaaT tool examines if all event handlers triggered by non-keyboard UI events are also associated with a keyboard-based event, or provide redundant keyboard-based mechanisms to accomplish the functionality provided by other device-specific functions. |
| Sometimes not so clear description of the error. | 1 | YES | The descriptions of the errors as well as the hints have been corrected/improved. |
| The line of problematic elements is hardly ever correct. So difficult to find the error. | 1 | YES | Fixed in the latest version of WaaT. |
| While success criterion 4.1.1 is listed as consisting of errors, it is enlisted as errors in the earl reporting. (VERITAS) | 1 | YES | Regarding success criterion 4.1.1, the assessment of the VERITAS web site returns 68 CSS warnings, which are also listed as warnings in the EARL report. |
| E.g. for VIPI-Project.eu, it indicates under Guideline 1.1 - Level A Number of |
1 | YES | As you can see in the following references, the "longdesc" attribute is poorly supported and instead of using "longdesc", it is suggested the D-Link technique to be used. According to this technique, when the developer wants to give a long description of an image, he must put a link with the letter "D" in brackets ("[D]") before or after the image. http://www.w3.org/TR/WCAG-TECHS/G73.html http://www.htmlcodetutorial.com/images/_IMG_LONGDESC.html http://www.w3schools.com/tags/att_img_longdesc.asp http://reference.sitepoint.com/html/img/longdesc#longdesc__fig-ff-longdesc http://en.wikipedia.org/wiki/HTML_element#longdesc So, we've changed a bit the specific approach, according to the following: 1) The specific output result is now considered as a warning (not as an error as it was considered previously) 2) Search for "[D]", not for hyperlink in general. |
| Errors about associated with images is often wrong. Seems an image always needs to be associated with . This is not correct. | 1 | YES | As you can see in the following references, the "longdesc" attribute is poorly supported and instead of using "longdesc", it is suggested the D-Link technique to be used. According to this technique, when the developer wants to give a long description of an image, he must put a link with the letter "D" in brackets ("[D]") before or after the image. http://www.w3.org/TR/WCAG-TECHS/G73.html http://www.htmlcodetutorial.com/images/_IMG_LONGDESC.html http://www.w3schools.com/tags/att_img_longdesc.asp http://reference.sitepoint.com/html/img/longdesc#longdesc__fig-ff-longdesc http://en.wikipedia.org/wiki/HTML_element#longdesc So, we've changed a bit the specific approach, according to the following: 1) The specific output result is now considered as a warning (not as an error as it was considered previously) 2) Search for "[D]", not for hyperlink in general. |
| For VERITAS website, is provided, but the error states 3.1.1 Provide an "xml:lang" attribute to the Web page, in order to identify the default language of the document. This seems to be an error. | 1 | YES | Fixed in the latest version of WaaT. |
| It seems Guideline 4.1 - Level A makes no sense as it indicated A[attributes={name=header_page}; value=[]] which I could not even find back in the source code. (VERITAS) Equally, 4 elements were identified as having no title attribute. None could be found. | 1 | YES | Fixed in the latest version of WaaT. |
| Legends, as asked for by the | 1 | YES | This approach is based on the H71 technique of WCAG 2.0 (http://www.w3.org/TR/2012/NOTE-WCAG20-TECHS-20120103/H71). As it is mentioned in the specific technique: "Check that each fieldset has a legend element that includes a description for that group." |
| Export As Pdf Failed One Time. App Crashed. | 1 | YES | During lots of tests, never managed to reproduce such an error. |
| I was not able to select more than one disabled person for evaluation assessment. | 1 | Partially | It is not supposed to support the combination of personas. The sequential assessment of two or more personas is supported instead. |
| It seems to work properly but when testing the same URL more than one time under the same conditions every result (percentage of accessibility) is different!? | 1 | YES | Fixed in the latest version of WaaT. |
| The standalone application had a deadlock during the initialization. | 1 | YES | It was fixed in the latest version. |
| No evaluation for people who cannot move any of their body parts. | 2 | YES | Evaluation for upper limb impaired users can be performed. |
| Blue letters on blue background are difficult to read | 2 | YES | The accessibility assessment of web applications will be mainly performed via the portal. The standalone version of WaaT is mainly for debug purposes and can be used by developers as a secondary tool for web assessment. The GUI of the standalone version is not supposed to be fully accessible and attractive. |
| In some cases more details are needed | 2 | YES | The descriptions of the errors as well as the hints have been corrected/improved. |
| Large amount of warnings not expanding correctly. | 2 | YES | The portal provides better presentation mechanisms. |
| Windows size don´t adjust to the computer screen. You can´t maximize the window. | 2 | YES | Fixed. The GUI of WaaT was made resizable. |
| Number of onclick event handlers without redundant onkeypress event handlers is indicated as an error, while it is not an error at all, and an alternative approach is possible, while also keyboard can be used. So this should not be an error (VERITAS website success criterion 2.1.1 line 238), rather a warning. | 2 | YES | As shown in http://www.w3.org/TR/2012/NOTE-WCAG20-TECHS-20120103/G90.html, this is actually an error. |
| The information is not always sufficient. | 2 | YES | The descriptions of the errors as well as the hints have been corrected/improved. |
| By clicking on the red/black/blue numbers, you get more detail per error. However this should be better indicated, as otherwise people will in general not use it. Same with possibility to do manual checks and check checkboxes etc. in the options provided when selecting success criteria; but waiting time was sometimes almost 10 minutes for 1 page. What then for a larger website? | 2 | YES | The portal provides a better GUI. The duration of the assessment process depends on the content of the examined web page/site. However, the WaaT offers two different types of evaluation: 1) a light one, which is faster and 2) a full version that includes all the supported tests (even these that are time-consuming). |
| "Choosing approaches manually" does not clearly explain what you do then. Also, the HAM methodology should be explained via a help, etc. | 2 | YES | It is better explained through the portal. |
| EARL reporting needs to be improved. Current version is difficult to use due to lack of overview. | 2 | YES | Overview tables are included in the PDF version of the report. |
| Scrolling by mouse wheel is too slow in the standalone-application. If the program would tell me the line in which the warnings occur, it would be really easy to fix the code. | 2 | YES | The mechanism that calculates the line in the source code where the error/warning occurs has been improved in the latest version of WaaT. Regarding the slow scrolling and other GUI problems, they are addressed in the portal. |
| The gui is crowded | 2 | YES | The accessibility assessment of web applications will be mainly performed via the portal. The standalone version of WaaT is mainly for debug purposes and can be used by developers as a secondary tool for web assessment. The GUI of the standalone version is not supposed to be fully accessible and attractive. |
| The gui needs redesign. | 2 | YES | The accessibility assessment of web applications will be mainly performed via the portal. The standalone version of WaaT is mainly for debug purposes and can be used by developers as a secondary tool for web assessment. The GUI of the standalone version is not supposed to be fully accessible and attractive. |
| The reporting should indicate on what basis the assessment was carried out. E.g. if it is based on a persona, that should be shown in the EARL reporting. As is now, the reports created do not make any mention of the assessment procedure. | 2 | YES | Fixed. Now the PDF report contains information about the persona or impairment that has been selected. |
| The results should be displayed in a more understandable way, for example, grouped. (Portal) | 2 | YES | The accessibility assessment of web applications will be mainly performed via the portal. The standalone version of WaaT is mainly for debug purposes and can be used by developers as a secondary tool for web assessment. The GUI of the standalone version is not supposed to be fully accessible and attractive. |
| Why is WCAG 1.0 not supported? This is still used a lot by organisations and public websites. Also, when doing it manually, you are not able to only check on one single level only, unless you start unchecking the different principles in each level. | 2 | YES | WCAG 1.0 was initially supported and according to the reviewers' comments during the 1st review, WCAG 1.0 was removed. Checkboxes enabling the check/uncheck of all the approaches belonging to the same priority level have been added. |
| You need to type "http://" otherwise an error occurs - but there is no feedback regarding this - it's just a test and trial error | 2 | YES | Fixed. Now, the "http://" can be omitted. |
| Red textlinks that are not underlined are not recognized. | 3 | YES | The portal, which is the main tool for web accessibility assessment, offers a much better GUI as well as better presentation mechanisms. |
| Some of the error messages have spelling mistakes. This should be corrected. E.g. success criterion 2.1.1 | 3 | YES | The descriptions of the errors as well as the hints have been corrected/improved. |
| Window is getting closed after choosing an URL - that is confusing. After checking the URL which is displayed in a small window the Main-Window opens again. Why can the Mail-Window not stay open in the background? | 3 | YES | A much better interface is available through the portal. |
| it is not clear if selected impairments in one category are remembered when the category is changed (missing united list); this works much better in the portal. | 3 | YES | The accessibility assessment of web applications will be mainly performed via the portal. The standalone version of WaaT is mainly for debug purposes and can be used by developers as a secondary tool for web assessment. The GUI of the standalone version is not supposed to be fully accessible and attractive. |
| The "Use personas"-Button should be placed at the top-left corner because of overseeing it when intuitively focusing the top-left corner similar to reading a book. | 3 | YES | The "Use personas" button has been placed at the top-left corner. |
| The main problem is that the reporting afterwards does not indicate what disability or persona had been selected. This makes analysis afterwards very difficult, unless you make notes separately on what you are assessing and how. | 3 | YES | Fixed. Now the PDF report contains information about the persona or impairment that has been selected. |
| There is no way to customize the reporting results. | 3 | YES | The portal offers better reporting mechanisms. |
| The main problem is filtering and selecting different result sets from the report. If I can do that I will be able to effectively focus and fix major problems. | 3 | YES | The portal offers better reporting mechanisms and categorization of the results. |
| Green codes are appearing as black codes in the reporting, confusing. Also difficult to find errors as there is no table of content or headings to ease navigation. | 3 | YES | The portal includes various presentation mechanisms supporting different categorisation types. A table of content is included in the PDF report. |
| Hierarchical presentation of the results according to the seriousness of the guidelines | 3 | YES | The results are presented by priority level, which refers to the seriousness of the guidelines. Moreover, the portal offers better reporting mechanisms and categorization of the results. |
| I could not export results to pdf. | 3 | YES | The results can be successfully presented in a PDF report. |
| It´s necessary for the application to allow you to check the accessibility problems within the tool. | 3 | YES | The accessibility assessment of web applications will be mainly performed via the portal. The standalone version of WaaT is mainly for debug purposes and can be used by developers as a secondary tool for web assessment. |
| My suggestion is to offer some sort of highlighting option for problems of the specific page as all most common W3C validators do. Highlighting the error directly in the source and a suggested solution would be very helpful. | 3 | YES | The accessibility assessment of web applications will be mainly performed via the portal. The standalone version of WaaT is mainly for debug purposes and can be used by developers as a secondary tool for web assessment. |
| Non attractive. | 3 | YES | The accessibility assessment of web applications will be mainly performed via the portal. The standalone version of WaaT is mainly for debug purposes and can be used by developers as a secondary tool for web assessment. The GUI of the standalone version is not supposed to be fully accessible and attractive. |
| Spelling errors in the reporting, e.g. 4.1.2 Number of elemenets without "title" attribute. It also seems Guideline 4.1 - Level A makes no sense as it indicated A[attributes={name=header_page}; value=[]] which I could not even find back in the source code. | 3 | YES | The descriptions of the errors as well as the hints have been corrected/improved. The specific result of Success Criterion 4.1.2 includes a hyperlink ( element) without "title" attribute. |
| The problematic element as reported is often not traceable in the source code. Seems it is "restructured" or stripped. | 3 | YES | Improvements have been made in the latest version of WaaT. |
| The reporting should link directly to the source files where the errors and warning points. For example if there is an error in file a.html line 135, when I click on the error the file a.html should be open with the cursor in the line 135 | 3 | Partially | As the WaaT tool does not support direct editing of the problematic elements in the source code (because when evaluating a URL the specific web page/site is stored on a server), it can only report the file and the line where the error appeared. |
| Too much information provided regarding the warnings (I would prefer a more briefer content). | 3 | YES | The amount of information regarding the warnings (as well as the errors) depends on the content of the examined web page. However, the portal includes better and more attractive presentation mechanisms. |
| More details about how to fix errors | 3 | YES | The descriptions of the errors as well as the hints have been corrected/improved. |
| More information and examples are needed | 3 | YES | The descriptions of the errors as well as the hints have been corrected/improved. |
| A progress bar showing, while the site(s) are assessed with the estimated time till completion based on the number of tags to analyse and how time-consuming they are (if the task should take longer than a few minutes) | 3 | Partially | The accessibility assessment of web applications will be mainly performed via the portal. The standalone version of WaaT is mainly for debug purposes and can be used by developers as a secondary tool for web assessment. The GUI of the standalone version is not supposed to be fully accessible and attractive. |
| No "any" checkbox | 3 | YES | Three new checkboxes have been added, allowing to select/deselect all the approaches of priority A, AA and AAA, respectively. |
| It needs more help tags; multi selection possibilities. | 3 | YES | The descriptions of the errors as well as the hints have been corrected/improved. |
| Needs refinement in interaction. | 3 | YES | The accessibility assessment of web applications will be mainly performed via the portal. The standalone version of WaaT is mainly for debug purposes and can be used by developers as a secondary tool for web assessment. The GUI of the standalone version is not supposed to be fully accessible and attractive. |
| Portal version has a higher setup time through project creation; the urls in a project can't be modified later, there is no link to the project overview site, sometimes got redirected to start page for no reason. Standalone version seems to have trouble interacting with AdobeReader, and the copy/paste functionality of the URL line is not good. No control of which 10 pages are tested; manual copy/paste of source code overrides this. In the portal I don't see where to specify the number of pages to assess | 3 | YES | The PDF report has been improved in the latest version of WaaT. |
| Some refinements needed. | 3 | YES | The accessibility assessment of web applications will be mainly performed via the portal. The standalone version of WaaT is mainly for debug purposes and can be used by developers as a secondary tool for web assessment. The GUI of the standalone version is not supposed to be fully accessible and attractive. |
| Sometimes one of the results does not expand, even when clicking on the "expand all" button, all will expand except one. | 3 | YES | Fixed in the latest version of WaaT. |
| The actual reporting analysis is quite difficult and tedious to use as one has no concrete overview, and the pdf has no headings included for easy navigation. | 3 | YES | The PDF report has been improved in the latest version of WaaT. It includes overview table with hyperlinks to the errors/warnings for each guideline. |
| The explanation of the full version is not entirely visible. The icons for the reports also jump depending on being in A, AA or AAA. | 3 | YES | The accessibility assessment of web applications will be mainly performed via the portal. The standalone version of WaaT is mainly for debug purposes and can be used by developers as a secondary tool for web assessment. The GUI of the standalone version is not supposed to be fully accessible and attractive. |
| The portal is more user friendly than the standalone version. There should be a "select all" and "deselect all" to facilitate to choose the guidelines. | 3 | YES | The accessibility assessment of web applications will be mainly performed via the portal. The standalone version of WaaT is mainly for debug purposes and can be used by developers as a secondary tool for web assessment. The GUI of the standalone version is not supposed to be fully accessible and attractive. |
| There is no "any" checkbox when you try to select the WCAG2 Evaluation approaches. | 3 | YES | Fixed. Three new checkboxes have been added, allowing to select/deselect all the approaches of priority A, AA and AAA, respectively. |
Position in the theoretical development process
The WaaT evaluation tests have been implemented according to the defined ACCESSIBLE Harmonized methodology (HAM) which is described in deliverable D3.1 "ACCESSIBLE harmonizes methodology".
The Web Accessibility Assessment Tool (Waat) supports the accessibility assessment of Web applications according to WCAG 2.0 and WAI-ARIA implemented tests. The following tables present all the trivial tests that have been implemented in the WaaT, according to the WCAG 2.0 guidelines and the WAI-ARIA.
Implemented tests in WaaT following accurately the test procedure defined by the guidelines of WCAG 2.0
| WCAG2 Guideline | Success Criterion | Technique(s) |
|---|---|---|
| 1.1 | 1.1.1 (Level A) | H24, H45, H46, H35, H37, H36, H44, H2, H67 |
| 1.2.8 (Level AAA) | H46 | |
| 1.3 | 1.3.1 (Level A) | H71, H63, H44, H39, H43, H51, H71, H44, H73, H65 |
| 1.4.2 (Level A) | G170, G18 | |
| 1.4.4 (Level AA) | C12, C13, C14, C28 | |
| 1.4.6 (Level AAA) | G17, G18 | |
| 1.4.8 (Level AAA) | C21, C24, C12, C13, C14 | |
| 2.1 | 2.1.1 (Level A) | H91, G90, SCR2, SCR20, G202, H71 |
| 2.2.2 (Level A) | G11 | |
| 2.2.4 (Level AAA) | G75, G76 | |
| 2.3 | 2.3.1 (Level A) | G15, G19, G176 |
| 2.3.2 (Level AAA) | G19 | |
| 2.4 | 2.4.1 (Level A) | H64, H50 |
| 2.4.2 (Level A) | G88, H25, H24 | |
| 2.4.5 (Level AA) | G126, H59, G185, G126 | |
| 2.4.6 (Level AA) | G130 | |
| 2.4.8 (Level AAA) | H59 | |
| 2.4.9 (Level AAA) | H30, H24 | |
| 3.1 | 3.1.1 (Level A) | H57, H54 |
| 3.1.4 (Level AAA) | H28 | |
| 3.2 | 3.2.2 (Level A) | G80 |
| 3.2.5 (Level AAA) | G76, H76, SVR1, H83, G110, SCR24 | |
| 3.3.2 (Level A) | G162, H65, H44, H71, H44, G162 | |
| 3.3.5 (Level AAA) | H89 | |
| 4.1 | 4.1.1 (Level A) | H93, G134, G192, H94, G134, G192, H74, H75, H88 |
| 4.1.2 (Level A) | H91, H65, H64, H88, H44 |
Tests implemented in the WaaT according to the WAI-ARIA specifications
| WAI-ARIA Test | Implementation details |
|---|---|
| Check for role attribute presence | Identify elements with "role" attribute and value equal to one of the following: "status", "tree", "row", "alertdialog", "navigation", "option", "menuitem", "application", "definition", "list", "tabpanel", "treeitem", "contentinfo", "combobox", "separator", "note", "region", "tablist", "tootltip", "log", "search", "link", "checkbox", "math", "dialog", "heading", "document", "radiogroup", "rowheader", "banner", "img", "group", "progressbar", "main", "marquee", "menubar", "listbox", "presentation", "radio", "treegrid", "directory", "complementary", "button", "menu", "rowgroup", "alert", "toolbar", "grid", "menuitemcheckbox", "menuitemradio", "spinbutton", "tab", "textbox", "scrollbar", "timer", "slider", "gridcell", "form", "columnheader", "listitem", "article" |
| Check for elements with missing required WAI-ARIA properties |
Examine the following (based on WAI ARIA roles:
|
| Check for problems in aria properties |
Examine the following:
|
| Check for elements with invalid role attribute | Elements with invalid role, which is not one of the following: "alert", "alertdialog", "application", "article", "banner", "button", "checkbox", "columnheader", "combobox", "complementary", "contentinfo", "definition", "dialog", "directory", "document", "form", "grid", "gridcell", "group", "heading", "img", "link", "list", "listbox", "listitem", "log", "main", "marquee", "math", "menu", "menubar", "menuitem", "menuitemcheckbox", "menuitemradio", "navigation", "note", "option", "presentation", "progresbar", "radio", "radiogroup", "region", "row", "rowgroup", "rowheader", "search", "separator", "scrollbar", "slider", "spinbutton", "status", "tab", "tablist", "tabpanel", "textbox", "timer", "toolbar", "tooltip", "tree", "treegrid", "treeitem" |
A path to obtain the required quality to integrate the tools in mainstream development environments (e.g. Netbeans, JDeveloper)
The Waat tool has been developed with the usage of open source technologies and open standards and for that reason there is the possibility of integrating the WaaT in existing IDE environments (e.g. NetBeans, JDeveloper). Currently, the AEGIS FP7 EU project has implemented an appropriate WaaT plug-in for the support of the NetBeans IDE environment. This implemented plug-in can be used by interested developers and designers in order to evaluate their Web applications during their software development process.
