With the advent of (deep) neural networks, computers nowadays excel at tasks such as image or speech recognition, previously unthought to be solved by machines. At the same time, deep learning is becoming increasingly important for industry, engineering, natural sciences but also society. Therefore, security and equity concerns but also external constraints such as natural laws represent fundamental obstacles for the general breakthrough of conventional machine learning (ML). Both, ML and computer simulations, share the goal of predicting the behaviour of a complex system using data analysis techniques and mathematical modelling approaches. Thereby, astrophysical phenomena, such as modelling the formation of our Milky Way galaxy, are inherently an interdisciplinary, massively multi-scale, multi-physics problem, commonly addressed with numerical models requiring high-performance computing facilities and millions of CPU hours. Nevertheless, scientific knowledge gain is limited by the amount of computing resources required to calculate all the relevant physics. Thus, there is a pressing need for a paradigm shift in the way we build and employ our numerical models. In this talk I will present some of the ideas we pursue to explore how modern ML techniques can be incorporated to obtain new insights into the physical processes of the formation and evolution of our Milky Way galaxy. In order to fully exploit those innovative methods in the natural sciences we need to develop ML methods that are inherently interpretable and respect the laws of physics. Therefore, physics-informed neural networks are one promising way to achieve this goal and with the example of chemical reaction networks I will present how those types of neural networks will help us increase the physical fidelity of our models.