Approximating multi-purpose AC optimal power flow with reinforcement trained Artificial Neural Network

Solving AC-Optimal Power Flow (OPF) problems is an essential task for grid operators to keep the power system safe for the use cases such as minimization of total generation cost or minimization of infeed curtailment from renewable DERs (Distributed Energy Resource). Mathematical solvers are often able to solve the AC-OPF problem but need significant computation time. Artificial neural networks (ANN) have a good application in function approximation with outstanding computational performance. In this paper, we employ ANN to approximate the solution of AC-OPF for multiple purposes. The novelty of our work is a new training method based on the reinforcement learning concept. A high-performance batched power flow solver is used as the physical environment for training, which evaluates an augmented loss function and the numerical action gradient. The augmented loss function consists of the objective term for each use case and the penalty term for constraints violation. This training method enables training without a reference OPF and the integration of discrete decision variable such as discrete transformer tap changer position in the constrained optimization. To improve the optimality of the approximation, we further combine the reinforcement training approach with supervised training labeled by reference OPF. Various benchmark results show the high approximation quality of our proposed approach while achieving high computational efficiency on multiple use cases.