软件测试中的非功能性,非功能性属性的测试

6. 可用性测试

Usability is defined in ISO 9241-11 as "the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use." A pre-requisite to usability testing is a pre-specified context of use, set of users and the tasks they would accomplish with the system. Testing for usability will cover aspects like

在ISO9241-11中，可用性的定义是“某个产品能够在被特定用户使用时，有效力，有效率的到达特定目的，并且在具体使用过程中令人满意”。可用性测试的前提条件是需要有预先指定的使用环境，弄清用户和他们想要用系统完成的任务。可用性测试将包含以下几方面：

– whether the users are able to complete their task,

– 用户是否可以完成他们的任务。

– efficiency with which they are able to do the task,

– 用户能否有效率的完成任务。

– is there a big variation in productivity between an expert user and an average user

– 在专家用户和一般拥护间是否存在巨大的效率差异。

– mental stress or cognitive workload on the user, especially for safety critical applications

– 考虑用户的精神压力和工作负担，尤其是对

– Some qu

– 对用户的满意程度和理解程度进行一些定量的测量。

The MUSiC project has developed the following tools and methods for usability testing[2].

MUSiC项目已经为可用性测试

– Performance Measurement Method

– 性能测量方法

– Analysis of what is measured through video recording is an effective way of measuring usability. The users can be observed during the typical tasks they perform to calculate effectiveness and efficiency of the system under test.

– 利用视频来分析测量结果是可用性测试的有效手段。

– Usability context analysis guide

– 可用性环境分析指南

– All usability measurements are subject to context of test. This guide provides a systematic method for describing the context of use, so that testing may be done under similar context.

– 所有可用性测试都依附于

– Video-assisted usability analysis

– 视频辅助可用性分析

– DRUM (Diagnostic recorder for Usability Measurement) is a software tool that supports video-assisted usability analysis. Video clips can be used to highlight usability problems to the designers. This tool supports analysis of the video log to derive usability metrics as well.

– DRUM(可用性测量诊断录象)是一款支持视频辅助可用性分析的软件工具。视频剪辑可以使可用性问题更加明显地呈现给设计者。该工具还能够对视频日志的分析来制定可用性

– User Satisfaction- SUMI

– 用户满意度问卷-SUMI

It is a questionnaire based method for evaluating usability. It focuses on the user perceptions. It gives an overall score and sub-scores on efficiency, affect, helpfulness, control and learnability. These scores are generated using a knowledge base created through a survey.

这是一种基于评价可用性的方法的问卷。它的重点在于用户的理解。该问卷对系统的效率、影响力，友好度，可控性和可学习性给于总分和小分。这些分数是由通过广泛调查而建立的知识体系产生出来的。

Efficiency score relates to the users feeling that the software is helping them to do the tasks faster and in an efficient manner. Affect score reflects the psychological state of the user after using the tool, i.e. do they feel good or frustrated. Helpfulness score reflects the users feelings on whether the software communicates in a helpful way and assists in the resolution of operational problems (errors, warnings, etc.). Control score gives a measure of the users feeling that they can use the software effectively. Learnability score reflects the users perception on how easy it is to learn and start using the software with the help of manuals and tutorial materials.

效率得分与用户感觉到这个软件能帮助他们以一种有效的方式更快地完成任务有关。而影响力得分则反映了用户在使用软件后的心理状态有关，比如说，他们是感觉到轻松还是沮丧。友好度得分是指用户是否觉得这个软件以一种友好的方式跟他们交流并用可选项的方式来协助他们处理问题(错误，警告等等)。可控性得分则是对用户感觉到他们在有效使用软件的度量。最后，可学习性得分则反映了用户感觉到他们通过帮助手册和指南材料学习软件的难以程度。

In MIEL, we have piloted the SUMI method on IQMEn [5]an internally developed process automation tool. It has helped in capturing user perception formally & quantitatively.

在MIEL，我们已经对对IQMEn尝试了SUMI方法，那是一个内部开发过程的自动化工具。它在形式上和数量上对用户理解程度的捕捉是有用的。

– Guide to cognitive workload measurement

– 对工作负担测量的指导

This method was developed at Delft university. Such evaluation is required as per the health and safety regulations for certain applications where mental stress of the operator can cause disasters.

这是由Delft大学开发的一种方法，该评价方法就如同某些健康和安全性规章一样，在精神压力可能导致事故的一些应用中应该采用。

7. Conclusion

7. 结论

A functionally correct product can be rated low by the users, if it doesn't meet their non-functional expectations like performance, usability etc. Non-functional testing brings out these potential problems with the product, which could cause major customer dissatisfaction. NFT can be done in the framework of existing test process, and we have outlined activities that can be performed as part of NFT in the test phases. We have also discussed issues relating to planning and scheduling of NFT.

一个功能正确的软件，如果它没有满足用户的诸如在性能和可用性等方面的非功能性

We have covered major aspects of performance testing and usability testing. As a next step, we are planning to add process assets like a template for test planning for non-functional attributes, guidelines for performance and usability testing, based on the concepts we have discussed here. NFT will be a focus in our testing of the products in our operations.

本文已经涉及了

References

[1] Software System Testing and quality assurance – Boris Beizer - Van Nostrand Reinhold

[2] "Scenario-Based Analysis of software Architecture", Kazman, R., Abowd, G., Bass, L., and Clements, P., IEEE Software, 13(6):47-56, 1996.

[3] http://www.usability.serco.com

[4] "A framework for product quality improvement", Krishnan Rangarajan, Vishu Hegde, Jacob Jacob, N.Swaminathan , APSES 99 (submitted)

[5] " Usability Analysis with the SUMI method", Krishnan Rangarajan, P. Rajashekar, Jacob Jacob, S.C Nirmala , APSES 99 (submitted)

[6] “Testing Server Applications” http://www.icon-stl.net/~djmosley/perform.htm

22/2<12