Now showing 1 - 9 of 9
  • Publication
    Reporting experiments to satisfy professionals' information needs
    ( 2014) ;
    Juristo, Natalia
    ;
    Rombach, H. Dieter
    Although the aim of empirical software engineering is to provide evidence for selecting the appropriate technology, it appears that there is a lack of recognition of this work in industry. Results from empirical research only rarely seem to find their way to company decision makers. If information relevant for software managers is provided in reports on experiments, such reports can be considered as a source of information for them when they are faced with making decisions about the selection of software engineering technologies. To bridge this communication gap between researchers and professionals, we propose characterizing the information needs of software managers in order to show empirical software engineering researchers which information is relevant for decision-making and thus enable them to make this information available. We empirically investigated decision makers' information needs to identify which information they need to judge the appropriateness and impact of a software technology. We empirically developed a model that characterizes these needs. To ensure that researchers provide relevant information when reporting results from experiments, we extended existing reporting guidelines accordingly. We performed an experiment to evaluate our model with regard to its effectiveness. Software managers who read an experiment report according to the proposed model judged the technology's appropriateness significantly better than those reading a report about the same experiment that did not explicitly address their information needs. Our research shows that information regarding a technology, the context in which it is supposed to work, and most importantly, the impact of this technology on development costs and schedule as well as on product quality is crucial for decision makers.
  • Publication
    A controlled experiment on component fault trees
    ( 2013)
    Jung, Jessica
    ;
    ;
    Höfig, Kai
    ;
    Domis, Dominik
    ;
    Hiller, Martin
    In safety analysis for safety-critical embedded systems, methods such as FMEA and fault trees (FT) are strongly established in practice. However, the current shift towards model-based development has resulted in various new safety analysis methods, such as Component Integrated Fault Trees (CFT). Industry demands to know the benefits of these new methods. To compare CFT to FT, we conducted a controlled experiment in which 18 participants from industry and academia had to apply each method to safety modeling tasks from the avionics domain. Although the analysis of the solutions showed that the use of CFT did not yield a significantly different number of correct or incorrect solutions, the participants subjectively rated the modeling capacities of CFT significantly higher in terms of model consistency, clarity, and maintainability. The results are promising for the potential of CFT as a model-based approach.
  • Publication
    Experimental comparison of two safety analysis methods and its replication
    ( 2013)
    Jung, Jessica
    ;
    Höfig, Kai
    ;
    Domis, Dominik
    ;
    ;
    Hiller, Martin
    (Background) Empirical Software Engineering (SE) strives to provide empirical evidence about the pros and cons of SE approaches. This kind of knowledge becomes relevant when the issue is whether to change from a currently employed approach to a new one or not. An informed decision is required and is particularly important in the development of safety-critical systems. For example, for the safety analysis of safety-critical embedded systems, methods such as Failure Mode and Effect Analysis (FMEA) and Fault Tree Analysis (FTA) are used. With the advent of model-based systems and software development, the question arises whether safety engineering methods should also be adopted. New technologies such as Component Integrated Fault Trees (CFT) come into play. Industry demands to know the benefits of these new methods over established ones such as Fault Trees (FT). (Methods) For the purpose of comparing CFT and FT with regard to the capabilities of the safety analysis methods (such as quality of the results) and to the participants' rating of the consistency, clarity, and maintainability of the methods, we designed a comparative study as a controlled experiment using a within-subject design. The experiment was run with seven academic staff members working towards their PhD. The study was replicated with eleven domain experts from industry. (Results) Although the analysis of the tasks' solutions showed that the use of CFT did not yield a significantly different number of correct or incorrect solutions, the participants rated the modeling capacities of CFT higher in terms of model consistency, clarity, and maintainability. (Conclusion) From this first evidence, we conclude that CFT have the potential of being beneficial for companies looking for a safety analysis approach for projects using model-based development.
  • Publication
    A controlled experiment on component fault trees
    ( 2013)
    Jung, Jessica
    ;
    ;
    Höfig, Kai
    ;
    Domis, Dominik
    ;
    Hiller, Martin
    In safety analysis for safety-critical embedded systems, methods such as FMEA and fault trees (FT) are strongly established in practice. However, the current shift towards model-based development has resulted in various new safety analysis methods, such as Component Integrated Fault Trees (CFT). Industry demands to know the benefits of these new methods. To compare CFT to FT, we conducted a controlled experiment in which 18 participants from industry and academia had to apply each method to safety modeling tasks from the avionics domain. Although the analysis of the solutions showed that the use of CFT did not yield a significantly different number of correct or incorrect solutions, the participants subjectively rated the modeling capacities of CFT significantly higher in terms of model consistency, clarity, and maintainability. The results are promising for the potential of CFT as a model-based approach.
  • Publication
    Experimental comparison of two safety analysis methods and its replication
    ( 2013)
    Jung, Jessica
    ;
    Höfig, Kai
    ;
    Domis, Dominik
    ;
    ;
    Hiller, Martin
    Empirical Software Engineering (SE) strives to provide empirical evidence about the pros and cons of SE approaches. This kind of knowledge becomes relevant when the issue is whether to change from a currently employed approach to a new one or not. An informed decision is required and is particularly important in the development of safety-critical systems. For example, for the safety analysis of safety-critical embedded systems, methods such as Failure Mode and Effect Analysis (FMEA) and Fault Tree Analysis (FTA) are used. With the advent of model-based systems and software development, the question arises whether safety engineering methods should also be adopted. New technologies such as Component Integrated Fault Trees (CFT) come into play. Industry demands to know the benefits of these new methods over established ones such as Fault Trees (FT). (Methods) For the purpose of comparing CFT and FT with regard to the capabilities of the safety analysis methods (such as quality of the results) and to the participants' rating of the consistency, clarity, and maintainability of the methods, we designed a comparative study as a controlled experiment using a within-subject design. The experiment was run with seven academic staff members working towards their PhD. The study was replicated with eleven domain experts from industry. (Results) Although the analysis of the tasks' solutions showed that the use of CFT did not yield a significantly different number of correct or incorrect solutions, the participants rated the modeling capacities of CFT higher in terms of model consistency, clarity, and maintainability. (Conclusion) From this first evidence, we conclude that CFT have the potential of being beneficial for companies looking for a safety analysis approach for projects using model-based development.
  • Publication
    Evaluating a model of software managers' information needs - an experiment
    Background: The identification of alternative technologies - which is one step of the formal decision-making process - results in a judgment on whether a technology is considered as a candidate for the further decision-making process. A model has been proposed that aims at improving the delivery of relevant information from ESE research to software managers. Objectives: Evaluate the effectiveness of the model of software managers information needs. Method: Experiment with software managers from industry who read two versions of a report on a controlled experiment, one of which contained the information as required by the information needs model. Results: Participants reading the version of the report based on the model perceived that they could judge better whether the technology is appropriate than those reading the original version of the paper. Conclusion: The information needs model provides a means for supporting the identification of alternative solutions and thus has the potential to solve the problem of making decisions.
  • Publication
    An approach to and evaluations of assisted living systems using ambient intelligence for emergency monitoring and prevention
    ( 2009) ; ;
    Storf, Holger
    ;
    Steinbach-Nordmann, Silke
    ;
    Prückner, Stephan
    Ambient Assisted Living (AAL) is currently one of the important research and development areas, where software engineering aspects play a significant role. The goal of AAL solutions is to apply ambient intelligence technologies to enable people with specific needs to continue to live in their preferred environments. This paper presents an approach and several evaluations for emergency monitoring applications. Experiments in a laboratory setting were performed to evaluate the accuracy of recognizing Activities of Daily Living (ADL). The results show that it is possible to detect ADLs with an accuracy of 92% on average. Hence, we conclude that it is possible to support elderly people in staying longer in their homes by autonomously detecting emergencies on the basis of ADL recognition.
  • Publication
    An approach to and evaluations of assisted living systems using ambient intelligence for emergency monitoring and prevention
    ( 2009) ; ;
    Storf, Holger
    ;
    Steinbach-Nordmann, Silke
    ;
    Prückner, Stephan
    Ambient Assisted Living (AAL) is currently one of the important research and development areas, where software engineering aspects play a significant role. The goal of AAL solutions is to apply ambient intelligence technologies to enable people with specific needs to continue to live in their preferred environments. This paper presents an approach and several evaluations for emergency monitoring applications. Experiments in a laboratory setting were performed to evaluate the accuracy of recognizing Activities of Daily Living (ADL). The results show that it is possible to detect ADLs with an accuracy of 92% on average. Hence, we conclude that it is possible to support elderly people in staying longer in their homes by autonomously detecting emergencies on the basis of ADL recognition.
  • Publication
    Reporting experiments in software engineering
    ( 2008) ;
    Ciolkowski, Marcus
    ;
    Pfahl, Dietmar
    Background: One major problem for integrating study results into a common body of knowledge is the heterogeneity of reporting styles: (1) It is difficult to locate relevant information and (2) important information is often missing. Objective: A guideline for reporting results from controlled experiments is expected to support a systematic, standardized presentation of empirical research, thus improving reporting in order to support readers in (1) finding the information they are looking for, (2) understanding how an experiment is conducted, and (3) assessing the validity of its results. Method: The guideline for reporting is based on (1) a survey of the most prominent published proposals for reporting guidelines in software engineering and (2) an iterative development incorporating feedback from members of the research community. Result: This chapter presents the unification of a set of guidelines for reporting experiments in software engineering. Limitation: The guideline has not been evaluated broadly yet. Conclusion: The resulting guideline provides detailed guidance on the expected content of the sections and subsections for reporting a specific type of empirical study, i.e., experiments (controlled experiments and quasi-experiments).