Learning and adaptivity for connectionist models and neural networks

This report contains the talks accepted for the meeting of the working group "connectionism" of the German Society of Computer Science (GI) on September 29th, 1999 in Magdeburg. It takes place in conjunction with the GI-Workshop-Days "Learning, Knowledge Discovery, and Adaptivity". The meeting is devoted to the discussion of new trends and ongoing research projects in the areas of connectionism and neural networks with specific emphasis to knowledge discovery and adaptivity. In the first paper A. Albrecht and C.K. Wong of the Chinese University of Hong Kong propose a "Modified Perceptron Algorithm Using Logarithmic Cooling Functions". They consider the problem of separating n-dimensional vectors into two classes by linear threshold functions. They present preliminary theoretical research on simulated annealing-based algorithms applied to this separation problem. The aim is to show how the convergence rate to optimum solution depends on specific parameters of the underlying energy landscape. They provide a probabilistic exponential bound on the run-time of the minimization problem. A. Hirabayashi, H. Ogawa and A. Nakashima of the Tokyo Institute of Technology consider memorization learning, which reduces only the training error. It does not guarantee good generalization capability in principle, and sometimes causes over-fitting. However, the goal in supervised learning is to obtain good generalization capability. The authors analyze the relation between good generalization and memorization learning. In this context the concept of "admissibility" becomes important. They investigate conditions for admissibility, and devise a method for memorization learning to be admitted by projection learning, which takes directly into account generalization capability. Effectiveness of the proposed method is illustrated by computer simulation. O. Mihatsch and R. Neuneier from Siemens AG, München, modify the criterion of reinforcement learning. They argue that optimizing the expected return is not always the most suitable because many applications require robust control strategies which also take into account the variance of the return. Therefore they use a new risk-sensitive control and learning theory along with the corresponding risk sensitive Q-learning algorithm. They apply their approach to the task of allocating funds to the German stock index DAX. Traditional risk-sensitive control techniques are computationally infeasible for such a large scale problem. They present the first systematic design of a multi-period risk averse asset allocation scheme which does not rely on an analytical market model. T. Villmann, S. Schünemann and B. Michaelis from the University of Leipzig discuss methods to choose the size of the neigborhood in topology representing networks, which are a generalization of Kohonen maps. They discuss two different approches to this problem, the latter using the Mahalanobis distance. For both approaches analytical results are derived. The practical properties are demonstrated by numerical examples. L. Wiskott from Wissenschaftskolleg Berlin introduces a hierarchical model for the visual system. It is trained with an unsupervised algorithm maximizing invariance properties. In this specific setup patterns uniformly move through the input area. The algorithm simultaneously detects the type of the pattern (what-information) as well as its location (where-information). The performance of the different approaches is demonstrated by numerical examples. J. Kindermann and G. Paaß consider a variant of "soft" decision trees for classification. Instead of using a greedy search they use an ensemble of plausible trees to describe the classification uncertainty. They use the Bayesian framework, which defines a joint probability distribution for all parameters. To switch between trees of different structure they use the reversible jump algorithm Markov Chain Monte Carlo procedure. They apply their method to a real credit-scoring problem. In the final paper G. Paaß and J. Kindermann discuss the query learning problem for neural classification networks. In a Bayesian decision theoretic framework they develop a query selection criterion which explicitly takes into account the overall "utility" of classification errors. The procedure rests on the MCMC-generation of posterior distributions. They determine the derivative of the overall utility with respect to changes of the queries and obtain an optimal query by stochastic gradient search.