Looking at Software Product Line Testing Tools

Software testing is a practice often used to determine and sometimes improve software quality. It is also a very labor and resource intensive process that often accounts for more than 50% of the total cost of software development[1]. According to Geppert[2] Software Testing is an area with high potential for reuse and reducing the current testing effort is a high priority topic.

Finding an effective and efficient software testing tool could be a life-saver for a project or a company. Yet there is no single test tool suitable for all possible systems and industry sectors. Deciding what criteria to apply when selecting a specific tool for a project is quite tricky [3].

In 2000, Harrold[4] said that: "We must develop methods and tools that implement the techniques and that can be used to demonstrate their effectiveness. To accomplish this, an important criterion is that these methods and tools be scalable to large systems. An efficient approach for development of methods and tools is to provide ways to automatically create them."

In our study of Software Product Lines Testing Tools we can see a tendency directed to Automatic Test Case Generation Tools and according to Bertolino [5] "The dream would be a powerful integrated test environment which by itself, as a piece of software is completed and deployed, can automatically take care of possibly instrumenting it and generating or recovering the needed scaffolding code (drivers, stubs, simulators), generating the most suitable test cases, executing them and finally issuing a test report".

Product lines exploit the commonalities and variability of reusable assets across different products. Nowadays, the most evident and perhaps most urging question is how to handle and represent variability[6]. Due to the large scale and huge complexity of today’s software-intensive systems, tool support is a key success factor in Software Product Line Engineering[7] and Software Product Line Testing.

[1]Myers, Glenford J., The art of software testing, New York: Wiley, c1979.

[2]Geppert, B., Robler, L. F., Weiss D. M., Towards Generating Acceptance Tests for Product Lines,Software Reuse: Methods, Techniques and Tools: 8th International Conference, ICSR 2004, Lecture Notes in Computer Science, 2004.

[3]Yang, Q., Li J. J., Weiss D., A Survey of Coverage Based Testing Tools, Proceedings of the 2006 International Workshop on Automation of Software Test.

[4]Harrold, M. J., Testing: A Roadmap, International Conference on Software Engineering, Proceedings of the Conference on The Future of Software Engineering, 2000.

[5]Bertolino A., Software Testing Research: Achievements, Challenges, Dreams, International Conference on Software Engineering, Future of Software Engineering, 2007.

[6]Jaring, M., and Bosch, J. Representing Variability in Software Product Lines: A Case Study. In Chastek G. J. (Ed.): Proc. Software Product Lines, 2nd Int. Conf, SPLC 2, San Diego, CA, USA, August 19-22, 2002, LNCS 2379, p.15-36.

[7]Clements, P. and Northrop, L., Software Product Lines: Practices and Patterns: SEI Series in Software Engineering, Addison-Wesley, 2001.