Integrated planning and scheduling for customized production using digital twins and reinforcement learning

For customized production in small lot-sizes, traditional production plants have to be reconfigured manually multiple times to be adapted to variable order changes, what significantly increases the production costs. One of the goals of Industry 4.0 is to enable flexible production, allowing for customer-specific production or even production with lot size 1 in order to react dynamically to changes in production orders. All of this with increased quality parameters such as optimized use of machines, conveyor belts and raw materials, which ultimately leads to optimized resource utilization and cost-efficiency. To address this challenge, in this paper, we present a digital twin based self-learning process planning approach using Deep-Q-Network that is capable of identifying optimized process plans and workflows for the simultaneous production of personalized products. We have evaluated our approach on a virtual aluminum cold milling factory from the SMS Group, in the context of the BaSys 4 project. The goal of the evaluation was to provide evidence that the proposed approach is able to handle large problem space effectively. Our approach ensures the efficiency of the personalized production and the adaptivity of the production system.