Diretrizes para avaliação de qualidade de quase-experimentos e experimentos controlados em linha de produto de software

Abstract

Experimentation in the area of Software Engineering (SE) has become essential to evaluate and improve the software and its development process, in addition to reducing maintenance efforts based on precisely evaluated and higher quality artifacts. One way to achieve higher quality is throughout the planning and conducting of experiments in a systematic and rigorous way. In order for SE experiments to be more rigorous, reproducible and auditable, there is a need to assess their quality in order to increase the reliability of the obtained results and evidences. Although the SE community superficially addresses how to evaluate the quality of controlled experiments, it is noted there is no effort to do so in specific contexts, such as Software Product Line (SPL). Such evaluation in this context is essential for an emerging research area such as SPL, which has been consolidated in recent decades in the scientific and industrial communities. In particular, in the scientific community there is a lack of experimentation produces a body of quality knowledge and experiments can be repeated in search of generalization of results. On the other hand, the industrial community can benefit from quality experiments seeking to acquire technologies and processes based on evidenced results. Thus, the objective of this work is to elaborate guidelines for quality evaluation of controlled experiments and quasi-experiments, based on the context of SPL. Therefore, it will be necessary to know the state of the art about experiments and quasi-experiments have been developed in SPL. In addition, it was necessary to study and compare approaches for quality evaluation of experiments and quasi-experiments in SE by means of a controlled experiment. The results of this experiment served as a basis for the elaboration of the guidelines. These guidelines were evaluated through a qualitative study, adopting Grounded Theory procedures, from the point of view of experts in Experimental SE and SPL. The results obtained provided evidence regarding the proposed guidelines application feasibility