Fraunhofer-Gesellschaft

Publica

Hier finden Sie wissenschaftliche Publikationen aus den Fraunhofer-Instituten.

Toward an automated benchmark management system

 
: Do, L.N.Q.; Eichberg, M.; Bodden, E.

:

Association for Computing Machinery -ACM-; Association for Computing Machinery -ACM-, Special Interest Group on Software Engineering -SIGSOFT-; Association for Computing Machinery -ACM-, Special Interest Group on Programming Languages -SIGPLAN-:
SOAP 2016, 5th ACM SIGPLAN International Workshop on State Of the Art in Program Analysis. Proceedings : Santa Barbara, CA, USA, June 14, 2016
New York: ACM, 2016
ISBN: 978-1-4503-4385-5
S.13-17
International Workshop on State Of the Art in Program Analysis (SOAP) <5, 2016, Santa Barbara/Calif.>
Englisch
Konferenzbeitrag
Fraunhofer IEM ()

Abstract
The systematic evaluation of program analyses as well as software-engineering tools requires benchmark suites that are representative of real-world projects in the domains for which the tools or analyses are designed. Such benchmarks currently only exist for a few research areas and even where they exist, they are often not effectively maintained, due to the required manual effort. This makes evaluating new analyses and tools on software that relies on current technologies often impossible. We describe ABM, a methodology to semi-automatically mine software repositories to extract up-to-date and representative sets of applications belonging to specific domains. The proposed methodology facilitates the creation of such collections and makes it easier to release updated versions of a benchmark suite. Resulting from an instantiation of the methodology, we present a collection of current real-world Java business web applications. The collection and methodology serve as a starting point for creating current, targeted benchmark suites, and thus helps to better evaluate current program-analysis and software-engineering tools.

: http://publica.fraunhofer.de/dokumente/N-422286.html