Inverse Design of Metasurfaces Using Reinforcement Learning Combined with Physics-informed Neural Networks

Designing high-performance optical metasurfaces traditionally relies on computationally expensive iterative simulations and is often limited by the scalability of gradient-based neural networks that struggle with high-dimensional inverse design tasks. To address these challenges, we present a data-free machine learning framework that combines a physics-informed neural network (PINN) with reinforcement learning (RL) for the inverse design of transmission-type all-dielectric metasurfaces. The PINN learns to model light diffraction directly from Maxwell's equations and boundary conditions, serving as a fast and accurate surrogate model that replaces computationally expensive full-wave 3D numerical electromagnetic simulations within the RL loop. The RL agent optimizes metasurface geometries by trial and error, receiving rewards based on the mean absolute error between target optical spectra and spectra predicted by the PINN for each proposed design. In our experiments, the RL agent outperforms conventional gradient-free optimizers in both accuracy and final design quality. This approach paves the way for data-efficient, physically reliable inverse design of next-generation optical devices.