A brief dive into Variability Testing

With the growth of applications complexity, the amount of variation points and the number of possible combinations among them can be considered as a problem to be concerned with, specially surrounding the creation of test assets that handle many variations. This context depicts a tradeoff between testability and the number of variabilities a product line contains.
In this context, we have just finished a systematic review on Software Product Line Testing (SPLT) which covers, among many other issues regarding this interesting research topic, variability testing concerns.

It is worthwhile to mention this aspect has been gained special attention among the SPLT researchers since this is not a very mature topic with many "open questions" to be addressed.
Following we illustrate the summary of important studies we have collected on this topic, presenting the effort devoted to solve this problem.

One proposes a solution, the cumulative variability coverage which accumulates coverage information across a series of product line instance development activities, to be further exploited in a target testing activities for product line instances. In another solution, constraints are placed into the product line architecture. Instead of having components with large amount of variability, for testability improvement commonalities and variabilities must be separated and the variabilities must be encapsulated into subcomponents. The objective is to reduce the retest of components and products when modifications are made so that independence of feature and components as well as the reduction of side effects are important. Other proposes to establish a coherent traceability from requirements to implementation and test assets. There are some ways to achieve this traceability between test assets and implementation, where the mechanism used in the product to implement the variation can be appropriate for implementing the test software for that portion of the software.

Furthermore, our intention regards to figure out how the SPLT approaches tackle variability along the software lifecycle, even though we have not gained answer to this question yet by reading and analyzing these and other studies. This can indeed be an extra research question to be addressed in future work!

A case study on technologies to implement variability in service-oriented product lines

In the course I.N.1.1.7.4 - Advanced Seminars in Software Reuse at CIN/UFPE was presented a case study definition on technologies to implement variability in service-oriented technologies. This post briefly describes the case study definition and some issues founded during the study.

Software Product Line (SPL) and Service-Oriented Architecture(SOA) emerge as two powerful concepts and their combination is a new research area. The combination of these two concepts is expected to become a new development paradigm maximizing reuse and business integration. In order to include services in a product line, developers could include variation points in the architecture implemented as a component or as a service as mentioned in the SOAPL 2007.

Thus, the goal of this case study is to identify the most suitable technology to implement variability in the context of service-oriented product lines. From this perspective, this case study will evaluate service-oriented technologies (OSGi and Web Services) and non-service-oriented technologies (XVCL) through a comparison between the technologies. In conjunction with the technologies, some variability mechanisms are applied to handle the variations in the source code.

Our contribution is that with this evaluation, we can select a technology that will help software engineers or researchers to integrate the technology with guidelines and criteria support in their organization or academic research.

However, some issues remain interesting for discussion:

1. OSGi can be considered as a service-oriented technology?

- The current release of OSGI stated that this technology incorporate interoperability of applications and services based on its component integration platform. However, this the concept of interoperability is not clear, because the OSGi is specific to Java.

2. It is possible combine OSGi and Web Services to solve the first issue?

3. Is it interesting to analyze these two technologies separately or the combination between them?

Feature Interaction in Software Product Lines

Software product lines (SPL) is considered a key approach for companies interested in improving software development increasing their productivity, quality and reducing costs. In SPL, features define commonalities and variabilities of the products, but they are not independent from each other, since there are interactions among them. These interactions among features can occur in unexpected ways causing impact in a SPL, affecting, for example, the reusable assets.

In this way, conflicts in most of the cases can be dealed during domain analysis phase through a management of feature dependencies. A feature dependency model represents not only static dependencies, but it also represents dynamic relationships and behavior characteristics among features. Additionally, it helps to trace the dependencies and how to configure the products member in a SPL.

Thus, due to the importance to solve this problem in a SPL environment, a systematic review was made to investigate approaches that proposed a solution for this. The main goal of the review was to identify how to classify and represent these dependencies and the guidelines used to identify the interactions to better understand a solution to avoid this problem.

However, some questions remain interesting for discursion: How identify feature interaction in a domain analysis? What activities are necessary? Is it interesting to have standardization on classification of feature interaction to support the dependencies identification among features?

Service-Oriented Product Line

This post describes important issues related to service-oriented analysis and design that were identified with a systematic review on the area and also present our future work related to combining service-oriented architecture and software product line. This research is being performed at Cin/UFPE.

The aim of the systematic review was to analyze the existing service-oriented analysis and design approaches with the objective of understand and summarize evidences about analysis and design discipline, identify activities, key points, drawbacks and gaps.

After analyze the existing approaches, we could identify some gaps such as a lack of architecture documentation and concerns with reusability and other quality attributes. We also detected common activities such as service identification from business process and legacy applications, create a service inventory blueprint, service and components specification and realization strategy selection such as wrapper or develop a new service from scratch.

With the systematic review on service-oriented analysis and design approaches concluded and the problems identified on the existing service-oriented approach, together with a systematic review on domain design approaches, a software product line design process, David Parnas’ ideas and based on Attribute Driven Development, our mission is to develop an approach for service-oriented product line.

Service-oriented architecture and software product line share a common goal, both concepts focus on reusability which brings productivity gains, decreased development costs, reduce time to market, higher reliability, and competitive advantage [Soa and Spl conference].

SOA and SPL also need a well defined process to be adopted and reduce its risks as stated by Thomas Erl and Klaus Pohl. This is the main motivation for our research that intends to develop a unified process for service-oriented architecture and software product line.

However, some questions remain interesting for discursion: How to combine service-oriented architecture and software product line? How can components be substituted by services in software product line context? How to consider different organizations with its own business processes to develop a service-oriented architecture as a product regarding variability?

Scoping on Software Product Lines

In the course I.N.1.1.7.4 - Advanced Seminars in Software Reuse at CIN was achieved a research about current status on Software Product Lines (SPL) Scope, phase of SPL planning, related with the identification of the costs and viability of the product line. For one that is not familiar with the theme this text can be interesting.

The research consisted of a systematic review whose objective was to review the software product lines approaches to identify, compare and summarize evidence about the Scope Definition Techniques. The research questions investigated in this review were related with the identification of the activities, scope types, stakeholders, strengths, drawbacks, and the use of metrics for scoping definition.

With the analysis we identify that metrics definition is the problematic aspect of the approaches, where only the papers of Klaus Schmid, Isabel Jhon and Shin Young Park cite metrics for scoping in its studies. Schmid utilize techniques of business objectives operationalization based in GQM, Jhon define characterization metrics and Park uses metrics of cost to analyze economical value for the core assets.

After of the analysis a idea was questioned, what the viability of relate scope techniques for SPL with agiles development methodologies? and what scope techniques are most appropriate with agiles methodologies?

2nd RiSS - RiSE Summer School

In the last year, we created RiSS - RiSE Summer School on Software Reuse. The main goal of RiSS is to discuss the main software reuse issues with the main experts in the field from industry and university. In this year, we had the second edition and I believe that the summer school was very nice. In the program, a full discussion about software product lines, the main topic on the software reuse area. As lecturers, we had experts from industry and university from several countries around the world. This mix is part of the RiSS successful.

In this year, we had five lecturers, one Workshop on Software Reuse Efforts and an awesome panel with the lecturers answering questions from the attendants.

During three days, we were in front of the sea as you can see and with a full room composed of 100 people interested on the topic.
In this year, we had the award again for the best lecture and Paul Clements joined to Wayne Lim with the best presentation about product line architecture.
















In name of the organization, I would like to say thank you for the participants, lecturers, and authors.

See you next time!!

One more Unforgettable RiSE Day

On November 26th, we had at C.E.S.A.R one more RiSE Day. This special workshop was composed of many discussions involving too many different issues in software reuse. In this edition, we had invited participants:Klaus Schmid, Kyo Kang, Rob Ommering, and Michalis Anastasopoulos.

The great agenda started with an overview about RiSE Labs and continued with the presentations about product lines, service-oriented product lines, bug triage and rise tools. Every year, I believe that this workshop is getting better.

Thanks for all the students and the invited participants for valuable feedback and patience.

Asset Retrieval Tools, Question Answering and Future

Nowadays, we are living in a world widely connected to the Internet where all users from the computer science area or not, are using search engines to find something. Different types of advances are facing these systems including song recognition and face detection. On the other hand, others relevant directions are being explored also. On July 2005, AskJeevs (ask.com) was acquired by InterActiveCorp for roughly $2.3 billions.

As you can see by the name, Ask.com is a question answer system on the web. In a paper from Communications from the ACM (CACM), Roussinov et al. discuss these systems with experimental data and interesting insights. Five years ago, I wrote a “paper” (in Portuguese) called: Common Component Market (CCM): Even your mother will want one! [you can see a draft in English here].

Nowadays, looking again from that paper, the Ask.com scenario, and the current search engines for source code, how far we are from doing questions such as the ones described in my “paper” and others based on requirements, design, and combining other information such as specific metrics.

The time will say it. Source code engines such as B.A.R.T, Merobase, Koders, etc., maybe can show it soon or not.

Software Reuse Business

In 1996, Wayne Lim wrote a paper Evolution of the Software Reuse Business published in the 4th International Conference on Software Reuse (ICSR). In his paper, Wayne discussed the roots of software reuse regarding the business point of view.

Wayne presented the case of the Raytheon and its efforts at the Raytheon Missiles Systems Division dating back to the mid-1970’s. In that time, the company developed a system which included logic structures, an index system, a library, some design specifications and coding standards. Some years after (1981), Raytheon through the Raytheon Computer Services created its reusable system ReadyCode. Next, with this experience they created a company, MasterSoftware, called the world’s first fully reusable software company, as described by Wayne.

Nowadays, we do not have any news about this company even tough Raytheon is still on business. In Wayne’s analysis, the main issue was that their system was success within Raytheon and the market outside the company boundaries is sometimes very different. Wayne highlights the importance of marketing since reuse within a company differs from reuse as an external business. The second point is the life cycle stage of the technology. In that time, reuse was a new way for the market and the need for education and culture was/is too important too.

From that time, we can see many companies on the road and some of them created by the main researchers on the field working in different areas. Examples of companies created from academic researchers include Bayfront Technologies Inc, a company located in California, founded in 1992 by James Neighbors, one of the pioneers in domain analysis. Semantic Designs, Inc, located in Austin, Texas, USA, was founded in 1995 by Dr. Ira Baxter and Dr. Christopher Pidgeon. BigLever Software, Inc is also located in Austin, Texas, being founded in 1999 by Charles Krueger, an important expert in software reuse. Other important researchers also have their companies, such as Bill Frakes, with Software Engineering Guild, Ruben Prieto-Diaz, with Reuse, Inc, Ted Biggerstaff, with www.softwaregenerators.com, and Wayne C. Lim, with the Lombard Hill Group, all of them working with consulting and services related to software reuse.
Not directly associated to academic researchers, the Flashline, Inc. company was a large metadata repository vendor, located in Cleveland, OH, USA. It was purchased in 2006 by BEA Systems, Inc, which incorporated Flashline's repository into its product family. Examples of reuse-dedicated companies from other countries include The Reuse Company, located in Madrid, Spain, with tools for knowledge reuse, and Pure-Systems, founded in 2001 in Germany.

In Brazil, RiSE is also a company offering software reuse solutions based on the C.E.S.A.R and RiSE expertise on the topic.

All of these efforts show that the market is looking for ways to increase the productivity, quality and reduce costs and software reuse is an effective way to achieve it. However, in order to be successful it involves a mix of different and important ingredients.

An Integrated Cost Model for Product Line Engineering

Today, we will publish in this blog one more M.Sc. dissertation defended in our group.

Jarley Nobrega's dissertation presents a relevant contribution for the field defining an economic model for software product lines.

Here is the abstract of the work:

In the software development community, the process of using existing artifacts rather than building them from scratch – generally known as software reuse – has been advanced as a way in which the problems associated with cost and schedule overruns can be avoided. Despite the potential rewards from an effective reuse program, it appears that its large-scale adoption is not particularly prevalent. Among the factors that inhibit reuse adoption there are the economic obstacles faced by organizations, which are concerned with the cost related to develop software for reuse and with reuse. Currently, thedecisions concerning large-scale reuse are often related with an economic viewpoint, since the development of software to be reusable can be considered as an investment. Moreover, the adoption of a software product line in a reuse context comes up with some inhibitors, such as the application of cost models in a restricted way, the lack of an investment analysis strategy, and the fact that a few cost models have a reuse scenario-based approach.

In this context, this work presents an integrated cost model for product line engineering in order to help the decisions concerning reuse investment. The foundations of the model were based on an extensive survey on cost models for software reuse and its extension to the product line approach. The model presents the definition of a set of cost and benefits functions, the description of reuse scenarios for product line engineering, and an investment analysis strategy. In addition, a simulation model based on the Monte Carlo method was proposed for simulating the reuse scenarios.

Finally, this work discusses the results of a case study in the context of a real software development environment where the model was applied.

See the full document here.