The Challenge of Supporting Repository-Based Continuous Learning with Systematic Evaluation and Maintenance

Experience-based continuous learning is an essential for improving methods, techniques, and tools in a discipline such as software engineering. This can be facilitated by capturing and managing the experience in a repository. Since we have to handle a "continuous stream of experience," such a repository requires maintenance. This maintenance is a challenge: It should not simply happen ad-hoc, but in a systematic manner: To make "well-founded" decisions about maintenance, the repository and its usage have to be analyzed (i.e., evaluated). To improve maintenance itself, it is necessary to learn about maintenance itself. This paper describes the challenge problem of maintaining such a repository and a solution to this problem in the form of a maintenance and evaluation framework. We show how other approaches from Artificial Intelligence, Knowledge Engineering, and Software Engineering fit into our framework and illustrate this with examples. Since other approaches at different levels of detail can be plugged in, the framework forms a good basis for both experimentation and use in practice.