Neural Scaling Laws for Learning-based Identification of Nonlinear Systems

The use of machine learning models in system identification has increased due to their ability to approximate complex nonlinear dynamics with high accuracy. However, often it is not clear how the performance of trained models scales with given resources such as data, compute, and model size. A recent development in the machine learning community is the use of so-called neural scaling laws (NSLs) between the amount of training resources and performance of a model. To allow for a better understanding of the scalability of the performance of machine learning models, we empirically verify NSLs in the context of system identification from input-state-output and input-output data using different evaluation metrics for accuracy, different systems and different neural network-based system identification architectures. Our results demonstrate predictable scaling behavior, providing a framework to forecast performance improvements and guide model design and data acquisition strategies.