Main content (Site structure)
Web Service accessibility assessment Tool – WebSaaT
Introduction & Information details for the WebSaaT Tool
The Web Service Assessment Tool is tool for the accessibility verification of Web services. WebSaaT supports SOA developers and architects in engineering and developing accessible SOAP Web-Services. Considering it as a black box, the tool can take as input the relevant WSDL-WADL file which describes the service, applies on it the Web Service accessibility guidelines defined in the ACCESSIBLE harmonized methodology and produces as output the evaluation result of the WS accessibility assessment process. The report produced as output contains warnings and errors and the relative guidelines which triggered the alert, together with the possible actions that could be taken to solve the problem (e.g. add something missing).
The product has been entirely developed within the research activities held during the ACCESSIBLE project and is based on the open source web-based service alignment tool provided by CERTH/ITI from ASK-IT and OASIS integrated project. The tool will guide the user in the assessment of the accessibility of web-services, taking a WSDL or a WADL description file as input and producing a report on accessibility conformance of the web-service. The user can fine tune the results of accessibility assessment using the “alignment” facility which can permit to align the service under exam with one of the “Ideal Operation” identified by the OASIS FP7 project and then apply the related ACCESSIBLE Guidelines as defined in the harmonized methodology. The OASIS semi-automatic Service alignment tool is in fact integrated in WebSaaT and substitute the initial ASK-IT Service Alignment tool which provides a similar alignment but with a complete manual process.
The final evaluation is based by these three different "phases":
- General information obtained parsing the WSDL/WADL file
- Ad-hoc information obtained from the alignment of the Service’s operations to the concepts defined as "Ideal" in OASIS FP7 project
- Information from the application of ACCESSIBLE harmonized methodology
Figure 1: Main view of the Web Services Assessment standalone tool
Owner/Main Developer (including also key partners contribution)
CERTH/ITI
Intelectual property Rights (IPR issues, Licences)
MIT licence
Nature of the Code
Open source
Published Papers
- Giakoumis, D., Votis, K., Tzovaras, D., Likothanassis, S. and Hassapis, G., Introducing accessibility in the Web services domain. in 3rd IEEE International Conference on Computer Science and Information Technology, ICCSIT, (Chengdu, China, 2010), 18-22.
- Giakoumis, D., Tzovaras, D., and Hassapis, G., Enabling accessibility characteristics in the Web Services domain, in Stephanidis, C. ed. Universal Access in Human-Computer Interaction. Design for All and eInclusion, Springer Berlin / Heidelberg, 2011, 177-186.
Strengths and Weaknesses
| Strengths | Weaknesses |
|---|---|
|
|
Contribution to the state of the art
A lot of attention has been paid the recent years, towards the definition and standardization of accessibility guidelines for content delivered through the Internet. A large number of research works have been conducted in this direction, whereas web content accessibility guidelines have also become a World Wide Web Consortium (W3C) standard (WCAG - Web Con- tent Accessibility Guidelines). This fact comes to no surprise, since people with disabilities hold an important amount of worldwide population. Their proper inclusion in everyday life activities commands for special attention to be taken during the design and development of web applications. Thus, Web services play a major role in our everyday life. In the modern web, continuously growing service clouds combine various services into distributed applications [15]. Content and functionality derived from these services can be subsequently delivered to end users, on the basis of service selection, composition and orchestration mechanisms. Following this line, the innovative WsaaT tool tries for the first time to fill in missing pieces regarding the notion of accessibility in the web service (WS) domain. It builds upon the notion of accessible WSs and the WS accessibility assessment framework introduced in ACCESSIBLE project.
So, the proposed Web Service accessibility assessment Tool (WSaaT) has been developed, towards further facilitating developers and designers to build either SOAP or REST -based accessible products.
Technical Testing Results from the developer and beneficiary validation
WebSaaT tool has been tested in pilot tests with developers and beneficiaries and several important usability issues occurred. In the following table we outline the most important issues that were identified during the testing, and how they were addressed for the improvement of the tool.
| Issue description | Priority | Solved | Solution / Explanation |
|---|---|---|---|
| More details on guidelines should be provided to developers. | 1 | YES | More details are now provided at the presentation of results regarding the guidelines that should be followed from the WS under assessment. |
| Reporting the results does not provide solutions for the problems. | 1 | YES | Appropriate tips are now provided within the WebSaaT reports, in respect of each guideline failed, providing more direct information regarding the solutions for the problems identified. |
| In some cases the description of the error can be more detailed so developer can easily understand it even if he lacks accessibility knowledge | 1 | YES | More details are now provided at the presentation of results regarding the guidelines that should be followed from the WS under assessment. Moreover, appropriate tips are provided within the reports, further facilitating the developer to understand an error more easily. |
| Sometimes the problem is not completely described. | 1 | YES | More details are now provided at the presentation of results regarding the guidelines that should be followed from the WS under assessment. Moreover, appropriate tips are provided within the reports, further facilitating the developer to easily understand an error. |
| When viewing the results, the main window will resize, but it will not scale the elements according to a percentage grid in the height direction. This is important to oversee the results somehow. | 1 | YES | The elements are now scaled according to a percentage in the height direction, thus allowing the user to better oversee the results. |
| The status of single Guidelines is easy to understand, but more information about "accessibility level" should be indicated. | 1 | YES | More information about the accessibility level is now provided in the reports. |
| The user of the tool might get lost among so many steps. After some tests the interface becomes familiar and the assessment is fast. | 1 | YES | The tutorial provided with the application helps users to get familiarized with the procedure through the standalone Tool. Once they are familiarized, the WS accessibility assessment process becomes straightforward and easy. |
| It is only possible to select 1 ideal operation but you can choose "all" checkboxes. | 1 | YES | The user is now not allowed to simultaneously choose more than one radio buttons. |
| Some of the tips are NULL or not very detailed. Some tips have "null" values in PDF reports. Sometimes the tips should be more detailed. The rules to follow to remove the error are not always clear or present. | 1 | YES | More appropriate tips are now provided within the WebSaaT reports, in respect of all guidelines failed, providing more comprehensive, direct information regarding the solutions for the problems identified. |
| If a wrong url is inserted and the parse button is pressed the application is freezing and can only be closed by the task manager. | 1 | YES | If a wrong URL is inserted and the parse button is pressed, the standalone version of the Tool now does not freeze. |
| In the "ideal operation" mapping some problems were found while testing two webservices in sequence. The ideal operation of the first is shown even if another one is selected. A logout/login was required | 1 | YES | While testing two WSs in sequence, the standalone Tool now correctly displays the inputs/outputs of the second Ideal Operation selected. |
| The ideal operation selection provides multiple choice interface (checkbox), which should be a radio button. | 2 | YES | Checkboxes have now been replaced with radio buttons |
| Reports should be more detailed on accessibility limitations identified | 2 | YES | More details are now provided within the assessment result reports, regarding the accessibility guidelines that are not followed from the WS under assessment. |
| The reports clearly identify the errors, but some more details about the limitations and the guideline can help in better understanding the issue | 2 | YES | The guidelines and the limitations posed are now better explained through the provision of more detailed info in the reports, i.e. through a better description about the accessibility level each guideline belongs to and appropriate tips. |
| Too much information reported. A better structure is needed. | 2 | YES | Reporting of the WebSaaT assessment results follows an EARL-based structure. The specific standard has been followed so as to provide users of the Tool a comprehensive presentation of the accessibility issues identified in a WS. However, the information provided within the reports has been further examined, and appropriate adjustments have been made in the content delivered, leading to even more comprehensive reports in PDF format. |
| After the WS definition is parsed, you have to select a "node of an operation" but it is not clear which is a node. After trial and error you find out where to click just to be able to go on with the evaluation. | 2 | YES | The names of the possible nodes that the user should select from are now provided within the relative message shown to the user, thus making clearer where s/he should click. |
| In some cases the guideline didn't provide a solution. | 2 | YES | Appropriate tips are now provided within the WebSaaT reports, in respect of each guideline failed, providing more direct information regarding the solutions for the problems identified. |
| It is not very clear how to define the ideal operation of your webservice. It should be better explained to the developers. | 2 | YES | The tutorial that accompanies the standalone tool provides a clear description of the ideal operations. |
| Not all the guidelines provided are simple to understand or apply. | 2 | YES | Accessibility assessment is a novel concept in the WS domain. It is expected that during their first interaction with the WSaaT, WS developers may be unfamiliar with WS accessibility concepts and guidelines. However, the description of the guidelines provided in the assessment reports has been further elaborated, so as to make them simpler to be understood and applied on WSs. |
| The UI is usable, but a progress bar could improve the usability of the tool. | 2 | NO | The standalone version of WebSaaT is mainly for debug purposes. Nevertheless, it can also be noted that in the standalone version of the Tool, the processing made from the PC in each step has a very short duration, thus a progress bar would eventually be pointless. |
| In the standalone tool there is no possibility to select guidelines or disabilities. The ACCESSIBLE portal offers a more complete, but also more complex assessment | 2 | NO | The standalone version of WebSaaT is mainly for debug purposes. It selects automatically the guidelines to use, on the basis of the type of the WS operation being assessed. The capability to select guidelines and disabilities manually is a feature of the tool's Portal version. |
| No way to complete the tests without a guide | 2 | YES | WS Assessment is a novel procedure, thus users are expected to be initially unfamiliar with it. For this reason, an appropriate tutorial accompanies the standalone version of the Tool. |
| Help links about guidelines when the test is made and there are errors (examples, tutorials,…) | 2 | YES | Appropriate tips are now provided within the WebSaaT reports, in respect of each failed guideline. They provide more direct information regarding the solutions for the problems identified, as well as in some cases, examples further facilitating the developer. |
| Test of WebService is quite easy. The "ideal operation" can provide more detailed results but needs a strong knowledge of the web service being tested. | 2 | YES | Typically, the Tool is expected to be used by a developer of a WS, thus, it is expected that the developer will indeed have a strong knowledge of the service being tested. |
| The assessment takes too many steps. It's difficult to use. | 2 | YES | The tutorial that accompanies the standalone version of the Tool provides a clear description of the steps that need to be followed for WS accessibility assessment. Moreover, the user of the Tool can thereafter find the Step X labels on the tool's buttons helpful, so as to better familiarize with the procedure in practice. |
| The assessment procedure was quite straight forward. More information about "ideal operation" procedure can speed up the procedures | 2 | YES | The tutorial that accompanies the standalone version of the Tool provides a comprehensive description of the Accessible Ideal Operations. At the UI of Ideal Operation selection, the user is now instructed that if needed, more information regarding the Ideal Operations can be found in the tutorial. |
| The steps are very complex and not clear. The usability should be improved | 2 | YES | Accessibility assessment is a novel concept in the WS domain. It is expected that during their first interaction with the WSaaT, WS developers may be unfamiliar with WS accessibility concepts and guidelines. The tutorial that accompanies the standalone version of the Tool provides a clear description of the steps that need to be followed for WS accessibility assessment. The user of the Tool may thereafter find the "Step X" labels on the buttons helpful, so as to better familiarize with the procedure in practice. |
| When viewing the results, scrollbar was not visible in the default window size. Scrolling with the mouse wheel was not possible. Focus problem | 3 | YES | Scrollbar is now always visible when viewing the results. Scrolling with the mouse is possible, over the grey area of the results pane. |
| Would an EARL reporting in pdf be possible here? | 3 | YES | The user of the tool has the option to obtain a PDF file describing the content of the EARL report that is generated for the assessment. The PDF is the human-readable version of the EARL report, and is similar to it. |
| After some tests the interface became familiar and the assessment is fast. The user interface of the two tools are quite different, maybe a more similar UI can help the users. | 3 | NO | The standalone version of WebSaaT is mainly for debug purposes. Moreover, it should be noted that the standalone and the portal versions of the tool were designed so as to offer similar functionality, in a way that is optimal for each version's operational context. |
| It is not clear that there are drop-down-menus at the 3rd step. Only by clicking in a white field you find out that there are options to choose | 3 | YES | These are the typical Java-based drop-down menus, used to further facilitate users when providing as input the name of an operation's input/output. At this point, it is not important for the user to be informed that s/he should provide input in a textbox or by using a drop-down menu. However, the fact that drop-down menus are used, is clearly shown at the Tool's tutorial. |
| The position of labels "Step 1" and "Step 2" are unfortunately not intuitively telling the user where to go. As to say: This is a kind of hidden information, where and how it is placed. | 3 | YES | The tutorial that accompanies the standalone version of the Tool provides a clear distinction among the steps that need to be followed for WS accessibility assessment. The user of the Tool may thereafter find the Step X labels on the buttons helpful, so as to better familiarize with the procedure in practice. |
Position in the theoretical development process
A WS accessibility assessment theoretical Framework (WSaaF) has been deployed consisting of a large number of WS accessibility guidelines. The inclusion of the proposed guidelines in the (both SOAP and REST -based) WS development process, can allow future service selection and provision systems to have in their disposal WSs, that foster the special needs of people with disabilities. Moreover, theWS accessibility assessment Tool (WSaaT), that was built on top of the WSaaF, is capable to significantly facilitate future developers towards the goal in hand. Our developed tool provides the means for semi-automatic assessment of a service against guidelines of the WSaaF. Through the accessibility assessment process, WSs can be categorized in three accessibility classes (AAA, AA and A), on the basis of the accessibility guidelines they conform to (more details for the theoretical framework could be extracted by the deliverable D3.1 ).
Regarding the alignment operation procedure that has included within the WebSaaT tool, some of the following Ideal Operations (Web services returning images, audios, videos, info mobility content) have been defined:
1. Image Provider Operation
Description: An operation that provides one or more images as output
- Input
- NONE
- Output
- 1. Accessible Image
- i. Image Object
- ii. Image Object URL
- iii. Alternative Text
- 1. Accessible Image
2. Audio Provider Operation
Description: An operation that provides one or more audio files as output
- Input
- NONE
- Output
- 1. Audio Object
- 2. Audio Object URL
- 3. Title
- 4. Short Description
- 5. Full Transcript
3. Video Provider Operation
Description: An operation that provides one or more video files as output
- Input
- NONE
- Output
- 1. Accessible Video
- i. Video Object
- ii. Video Object URL
- iii. Title
- iv. Short Description
- v. Full Transcript
- vi. Audio Description
- a. Audio Description Object
- b. Audio Description Object URL
- c. Audio Description Title
- d. Audio Description Short Description
- e. Audio Description Full Transcript
- f. Audio Description Sign Language Interpretation
- 1. Accessible Video
