Multitask Learning via Task Embeddings for Glass Property Prediction with Improved Sample Efficiency

Glassy materials are prevalent in many modern-day applications, which place ever new demands on the materials' properties. However, because of their amorphous nature, the design of glasses with specific properties is inherently difficult and necessarily data-driven. Due to the large size of the compositional space and the cost of the manufacturing process, glass data is only sparsely available. It is thus important to design and train models which maximize the benefit of each individual training sample on the models' resulting prediction quality. In this paper, we study a multitask learning approach to meet this challenge. We analyze its effect on the sample efficiency, that is, the dependence of the test error on the training set size, of a neural network model for predicting properties of oxide glasses. In contrast to existing encoder-based multitask learning models in computational glass science, we propose a decoder-based model, consisting of task embeddings in the input layer and a subsequent shared network architecture. This allows us to apply well-established single-task models in a multitask learning setting. We demonstrate in a series of numerical experiments, predicting the Young's modulus and the log-viscosity, respectively, that this approach notably improves the model's sample efficiency.
To deal with parametric target properties, we complement the task embeddings by a parameter embedding and show that this preserves the advantageous multitask learning effect.
In all experiments, we observe an algebraic scaling law for the decay of the test error with respect to the number of training samples.