Model order reduction

Model order reduction (MOR) is a technique for reducing the computational complexity of mathematical models in numerical simulations. As such it is closely related to the concept of metamodeling, with applications in all areas of mathematical modelling.

Overview

Many modern mathematical models of real-life processes pose challenges when used in numerical simulations, due to complexity and large size (dimension). Model order reduction aims to lower the computational complexity of such problems, for example, in simulations of large-scale dynamical systems and control systems. By a reduction of the model's associated state space dimension or degrees of freedom, an approximation to the original model is computed which is commonly referred to as a reduced order model.

Reduced order models are useful in settings where it is often unfeasible to perform numerical simulations using the complete full order model. This can be due to limitations in computational resources or the requirements of the simulations setting, for instance real-time simulation settings or many-query settings in which a large number of simulations needs to be performed. ^{[1] [2]} Examples of Real-time simulation settings include control systems in electronics and visualization of model results while examples for a many-query setting can include optimization problems and design exploration. In order to be applicable to real-world problems, often the requirements of a reduced order model are: ^{[3] [4]}

• A small approximation error compared to the full order model.
• Conservation of the properties and characteristics of the full order model (E.g. stability and passivity in electronics).
• Computationally efficient and robust reduced order modelling techniques.

It is interesting to note that in some cases (e.g. constrained lumping of polynomial differential equations) it is possible to have a null approximation error, resulting in an exact model order reduction. ^[5]

Methods

Contemporary model order reduction techniques can be broadly classified into 5 classes: ^{[1] [6]}

• Proper orthogonal decomposition methods. ^{[7] [8]}
• Reduced basis methods. ^[9]
• Balancing methods
• Simplified physics ^[10] or operational based reduction methods. ^[3]
• Nonlinear and manifold model reduction methods. ^[11]

The simplified physics approach can be described to be analogous to the traditional mathematical modelling approach, in which a less complex description of a system is constructed based on assumptions and simplifications using physical insight or otherwise derived information. However, this approach is not often the topic of discussion in the context of model order reduction as it is a general method in science, engineering, and mathematics.

Proper orthogonal decomposition, reduced basis, and balancing methods fall into the category of projection-based reduction. Projection-based reduction relies on the projection of either the model equations or the solution onto a basis of reduced dimensionality compared to the original solution space. Methods that also fall into this class but are perhaps less common are:

• Proper generalized decomposition ^[12]
• Matrix interpolation ^[13]
• Transfer function interpolation
• Piecewise tangential interpolation
• Loewner framework
• (Empirical) cross Gramian
• Krylov subspace methods ^[14]

Nonlinear and manifold model reduction methods derive nonlinear approximations on manifolds and so can achieve higher accuracy with the same number of degrees of freedom than traditional methods that obtain linear approximations in subspaces. ^[11] Building on nonlinear approximations is essential for efficiently reducing certain problem classes such as wave problems and advection-dominated problems in computational fluid dynamics. The nature and principles underlying nonlinear model reduction methods are broad and include template-based methods, ^{[15] [16] [17]} the use of neural networks ^{[18] [19] [20]} and online adaptive spaces. ^{[21] [22]}

There are also nonintrusive model reduction methods that learn reduced models from data without requiring knowledge about the governing equations and internals of the full, high-fidelity model. Nonintrusive methods learn a low-dimensional approximation space or manifold and the reduced operators that represent the reduced dynamics from data. Methods that are non-intrusive include:

• Dynamic mode decomposition ^{[23] [24] [25]}
• Operator inference ^{[26] [27]}
• Loewner framework ^{[28] [29]}

Implementations

• RBmatlab: A MATLAB library containing all reduced simulation approaches for linear and nonlinear, affine or arbitrarily parameter dependent evolution problems with finite element, finite volume or local discontinuous Galerkin discretizations.

• Model Reduction inside ANSYS: implements a Krylov-based model order reduction for multiphysical finite element models in Ansys. Model simplification via Model Reduction inside Ansys is suitable for optimization strategies in component development as well as for integrating compact models into an overall system simulation in the fields of electronics, automotive or microsystems. Despite reduction, the examination parameters are retained, which means fast results can be achieved with regards to designs and system simulations.

• pyMOR: pyMOR is a software library for building model order reduction applications with the Python programming language. Its main focus lies on the application of reduced basis methods to parameterized partial differential equations. All algorithms in pyMOR are formulated in terms of abstract interfaces for seamless integration with external high-dimensional PDE solvers. Moreover, pure Python implementations of finite element and finite volume discretizations using the NumPy/SciPy scientific computing stack are provided for getting started quickly.
• emgr: Empirical Gramian Framework. Empirical gramians can be computed for linear and nonlinear control systems for purposes of model order reduction, uncertainty quantification or system identification. The emgr framework is a compact open source toolbox for gramian-based model reduction and compatible with OCTAVE and MATLAB.
• KerMor: An object-oriented MATLAB© library providing routines for model order reduction of nonlinear dynamical systems. Reduction can be achieved via subspace projection and approximation of nonlinearities via kernels methods or DEIM. Standard procedures like the POD-Greedy method are readily implemented as well as advanced a-posteriori error estimators for various system configurations. KerMor also includes several working examples and some demo files to quickly get familiarized with the provided functionality.
• JaRMoS: JaRMoS stands for "Java Reduced Model Simulations" and aims to enable import and simulation of various reduced models from multiple sources on any java-capable platform. So far support for RBmatlab, KerMor and rbMIT reduced models is present, where we can only import the rbMIT models that have previously been published with the rbAppMIT Android application. Extensions so far are a desktop-version to run reduced models and initial support for KerMor kernel-based reduced models is on the way.
• MORLAB: Model Order Reduction Laboratory. This toolbox is a collection of MATLAB/OCTAVE routines for model order reduction of linear dynamical systems based on the solution of matrix equations. The implementation is based on spectral projection methods, e.g., methods based on the matrix sign function and the matrix disk function.
• Dune-RB: A module for the Dune library, which realizes C++ template classes for use in snapshot generation and RB offline phases for various discretizations. Apart from single-core algorithms, the package also aims at using parallelization techniques for efficient snapshot generation.
• libROM: Collection of C++ classes that compute model order reduction and hyper-reduction for systems of partial and ordinary differential equations. libROM includes scalable and parallel, adaptive methods for proper orthogonal decomposition, parallel, non-adaptive methods for hyper-reduction, and randomized singular value decomposition. libROM also includes the dynamic mode decomposition capability. libROM has physics-informed greedy sampling capability.
• Pressio: Pressio is an open-source project aimed at alleviating the intrusive nature of projection-based reduced-order models for large-scale codes. The core of the project is a header-only C++ library that leverages generic programming to interface with shared or distributed memory applications using arbitrary data-types. Pressio provides numerous functionalities and solvers for performing model reduction, such as Galerkin and least-squares Petrov–Galerkin projections. The Pressio ecosystem also offers: (1) pressio4py, a Python binding library for ease of prototyping, (2) pressio-tutorials, a library also offering end-to-end demos that one can easily play with, which can be found at https://pressio.github.io/pressio-tutorials/, (3) pressio-tools, a library for large-scale SVD, QR and sample mesh, and (4) pressio-demoapps, a suite of 1d, 2d and 3d demo applications for testing ROMs and hyper-reduction.
• PyDMD: PyDMD is a Python package that implements data-driven model order reduction based on Dynamic Mode Decomposition (DMD), an algorithm developed by Schmid ^[24] . DMD is used to analyze the dynamics of nonlinear systems and relies solely on high-fidelity measurements, making it an equation-free algorithm.

Applications

Model order reduction finds application within all fields involving mathematical modelling and many reviews ^{[10] [12]} exist for the topics of electronics, ^[30] fluid mechanics, ^[31] hydrodynamics, ^[32] structural mechanics, ^[7] MEMS, ^[33] Boltzmann equation, ^[8] and design optimization. ^{[13] [34]}

Fluid mechanics

Current problems in fluid mechanics involve large dynamical systems representing many effects on many different scales. Computational fluid dynamics studies often involve models solving the Navier–Stokes equations with a number of degrees of freedom in the order of magnitude upwards of ${\displaystyle 10^{6}}$. The first usage of model order reduction techniques dates back to the work of Lumley in 1967, ^[35] where it was used to gain insight into the mechanisms and intensity of turbulence and large coherent structures present in fluid flow problems. Model order reduction also finds modern applications in aeronautics to model the flow over the body of aircraft. ^[36] An example can be found in Lieu et al ^[37] in which the full order model of an F16 fighter-aircraft with over 2.1 million degrees of freedom, was reduced to a model of just 90 degrees of freedom. Additionally reduced order modeling has been applied to study rheology in hemodynamics and the fluid–structure interaction between the blood flowing through the vascular system and the vascular walls. ^{[38] [39]}

See also

• Dimension reduction
• Metamodeling
• Principal component analysis
• Singular value decomposition
• Nonlinear dimensionality reduction
• System identification
• Iterative rational Krylov algorithm (IRKA)

References

1. Lassila, Toni; Manzoni, Andrea; Quarteroni, Alfio; Rozza, Gianluigi (2014). "Model Order Reduction in Fluid Dynamics: Challenges and Perspectives". Reduced Order Methods for Modeling and Computational Reduction (PDF). pp. 235–273. doi:10.1007/978-3-319-02090-7_9. ISBN   978-3-319-02089-1.
2. Rozza, G.; Huynh, D. B. P.; Patera, A. T. (2008-05-21). "Reduced Basis Approximation and a Posteriori Error Estimation for Affinely Parametrized Elliptic Coercive Partial Differential Equations". Archives of Computational Methods in Engineering. 15 (3): 229–275. doi:10.1007/s11831-008-9019-9. ISSN   1134-3060. S2CID   13511413.
3. Schilders, Wilhelmus; van der Vorst, Henk; Rommes, Joost (2008). Model Order Reduction: Theory, Research Aspects and Applications. Springer-Verlag. ISBN   978-3-540-78841-6.
4. Antoulas, A.C. (July 2004). "Approximation of Large-Scale Dynamical Systems: An Overview". IFAC Proceedings Volumes. 37 (11): 19–28. CiteSeerX   10.1.1.29.3565 . doi:10.1016/S1474-6670(17)31584-7.
5. Ovchinnikov, Alexey; Pérez Verona, Isabel; Pogudin, Gleb; Tribastone, Mirco (2021-07-19). Valencia, Alfonso (ed.). "CLUE: exact maximal reduction of kinetic models by constrained lumping of differential equations". Bioinformatics. 37 (12): 1732–1738. arXiv: 2004.11961 . doi: 10.1093/bioinformatics/btab010 . ISSN   1367-4803. PMID   33532849.
6. Silva, João M. S.; Villena, Jorge Fernández; Flores, Paulo; Silveira, L. Miguel (2007), "Outstanding Issues in Model Order Reduction", Scientific Computing in Electrical Engineering, Springer Berlin Heidelberg, pp. 139–152, doi:10.1007/978-3-540-71980-9_13, ISBN   978-3-540-71979-3
7. Kerschen, Gaetan; Golinval, Jean-claude; VAKAKIS, ALEXANDER F.; BERGMAN, LAWRENCE A. (2005). "The Method of Proper Orthogonal Decomposition for Dynamical Characterization and Order Reduction of Mechanical Systems: An Overview". Nonlinear Dynamics. 41 (1–3): 147–169. CiteSeerX   10.1.1.530.8349 . doi:10.1007/s11071-005-2803-2. ISSN   0924-090X. S2CID   17625377.
8. Choi, Youngsoo; Brown, Peter; Arrighi, William; Anderson, Robert; Huynh, Kevin (2021). "Space--time reduced order model for large-scale linear dynamical systems with application to Boltzmann transport problems". Journal of Computational Physics. 424: 109845. arXiv: 1910.01260 . Bibcode:2021JCoPh.42409845C. doi:10.1016/j.jcp.2020.109845. ISSN   0021-9991. S2CID   203641768.
9. Boyaval, S.; Le Bris, C.; Lelièvre, T.; Maday, Y.; Nguyen, N. C.; Patera, A. T. (16 October 2010). "Reduced Basis Techniques for Stochastic Problems". Archives of Computational Methods in Engineering. 17 (4): 435–454. arXiv: 1004.0357 . doi:10.1007/s11831-010-9056-z. hdl:1721.1/63915. S2CID   446613.
10. Benner, Peter; Gugercin, Serkan; Willcox, Karen (2015). "A Survey of Projection-Based Model Reduction Methods for Parametric Dynamical Systems" (PDF). SIAM Review. 57 (4): 483–531. doi:10.1137/130932715. hdl: 1721.1/100939 . ISSN   0036-1445. S2CID   16186635.
11. Peherstorfer, B. (2022). "Breaking the Kolmogorov Barrier with Nonlinear Model Reduction". Notices of the American Mathematical Society. 69 (5): 725–733. doi:10.1090/noti2475.
12. Chinesta, Francisco; Ladeveze, Pierre; Cueto, Elías (11 October 2011). "A Short Review on Model Order Reduction Based on Proper Generalized Decomposition" (PDF). Archives of Computational Methods in Engineering. 18 (4): 395–404. doi:10.1007/s11831-011-9064-7. S2CID   54512292.
13. Choi, Youngsoo; Boncoraglio, Gabriele; Spenser, Anderson; Amsallem, David; Farhat, Charbel (2020). "Gradient-based constrained optimization using a database of linear reduced-order models". Journal of Computational Physics. 423: 109787. arXiv: 1506.07849 . Bibcode:2020JCoPh.42309787C. doi:10.1016/j.jcp.2020.109787. S2CID   60788542.
14. Bai, Zhaojun (2002). "Krylov subspace techniques for reduced-order modeling of large-scale dynamical systems". Applied Numerical Mathematics. 43 (1–2): 9–44. CiteSeerX   10.1.1.131.8251 . doi:10.1016/S0168-9274(02)00116-2.
15. Rowley, C.W.; Marsden, J.E. (2000). "Reconstruction equations and the Karhunen–Loève expansion for systems with symmetry". Physica D: Nonlinear Phenomena. 142 (1–2): 1–19. doi:10.1016/S0167-2789(00)00042-7.
16. Ohlberger, M.; Rave, S. (2013). "Nonlinear reduced basis approximation of parameterized evolution equations via the method of freezing". Comptes Rendus Mathematique. 351 (23–24): 901–906. doi:10.1016/j.crma.2013.10.028.
17. Reiss, J.; Schulze, P.; Sesterhenn, J.; Mehrmann, V. (2018). "The Shifted Proper Orthogonal Decomposition: A Mode Decomposition for Multiple Transport Phenomena". SIAM Journal on Scientific Computing. 40 (3): A1322 –A1344. doi:10.1137/17M1140571.
18. Kim, Youngkyu; Choi, Youngsoo; Widemann, David; Zohdi, Tarek (2021). "A fast and accurate physics-informed neural network reduced order model with shallow masked autoencoder". Journal of Computational Physics. 451: 110841. arXiv: 2009.11990 . doi:10.1016/j.jcp.2021.110841. S2CID   221949087.
19. Mojgani, Rambod; Balajewicz, Maciej (2021). "Low-Rank Registration Based Manifolds for Convection-Dominated PDEs". Proceedings of the AAAI Conference on Artificial Intelligence. 35: 399-407. arXiv: 2006.15655 . doi: 10.1609/aaai.v35i1.16116 . S2CID   220249659.
20. Berman, J.; Peherstorfer, B. (2024). "CoLoRA: Continuous low-rank adaptation for reduced implicit neural modeling of parameterized partial differential equations". PMLR. Proceedings of the 41st International Conference on Machine Learning. Vol. 235. Vienna. pp. 3565–3583.
21. Peherstorfer, B. (2020). "Model Reduction for Transport-Dominated Problems via Online Adaptive Bases and Adaptive Sampling". SIAM Journal on Scientific Computing. 42 (5): A2803 –A2836. doi:10.1137/19M1257275.
22. Hesthaven, J.S.; Pagliantini, C.; Rozza, G. (2022). "Reduced basis methods for time-dependent problems". Acta Numerica. 31: 265–345. doi:10.1017/S0962492922000058.
23. Rowley, C.W.; Mezic, I.; Bagheri, S.; Schlatter, P.; Henningson, D.S. (2009). "Spectral analysis of nonlinear flows". Journal of Fluid Mechanics. 641: 115–127. doi:10.1017/S0022112009992059.
24. Schmidt, P.J. (2010). "Dynamic mode decomposition of numerical and experimental data". Journal of Fluid Mechanics. 656: 5–28. doi:10.1017/S0022112010001217.
25. Tu, Jonathan H.; Rowley, Clarence W.; Luchtenburg, Dirk M.; Brunton, Steven L.; Kutz, J. Nathan (2014). "On dynamic mode decomposition: Theory and applications". Journal of Computational Dynamics. 1 (2): 391–421. arXiv: 1312.0041 . doi:10.3934/jcd.2014.1.391.
26. Peherstorfer, B.; Willcox, K. (2016). "Data-driven operator inference for nonintrusive projection-based model reduction". Computer Methods in Applied Mechanics and Engineering. 306: 196–215. doi:10.1016/j.cma.2016.03.025.
27. Kramer, B.; Peherstorfer, B.; Willcox, K. (2024). "Learning Nonlinear Reduced Models from Data with Operator Inference". Annual Review in Fluid Mechanics. 56: 521–548. doi:10.1146/annurev-fluid-121021-025220.
28. Mayo, A.J.; Antoulas, A.C. (2007). "A framework for the solution of the generalized realization problem". Linear Algebra and Its Applications. 425 (2–3): 634–662. doi:10.1016/j.laa.2007.03.008.
29. Antoulas, A.C.; Beattie, C.A.; Gugercin, S. (2021). Interpolatory Methods for Model Reduction. Society for Industrial and Applied Mathematics. ISBN   9781611976076.
30. Umunnakwe, Chisom Bernhard; Zawra, Ibrahim; Niessner, Martin; Rudnyi, Evgenii; Hohlfeld, Dennis; Bechtold, Tamara (2023). "Compact modelling of a thermo-mechanical finite element model of a microelectronic package". Microelectronics Reliability. 151 (115238). doi:10.1016/j.microrel.2023.115238.
31. Holmes, Philip; Lumley, John L.; Berkooz, Gal (1996). Turbulence, Coherent Structures, Dynamical Systems and Symmetry. Cambridge: Cambridge University Press. doi:10.1017/cbo9780511622700. ISBN   978-0-511-62270-0.
32. Copeland, Dylan; Cheung, Siu Wun; Huynh, Kevin; Choi, Youngsoo (2021). "Reduced order models for Lagrangian hydrodynamics". Computer Methods in Applied Mechanics and Engineering. 388: 114259. arXiv: 2104.11404 . doi:10.1016/j.cma.2021.114259. ISSN   0045-7825. S2CID   233388014.
33. Bechtold, Tamara; Schrag, Gabriela; Feng, Lihong (2013). System-Level Modeling of MEMS. Wiley-VCH Verlag GmbH & Co. KGaA. ISBN   978-3-527-31903-9.
34. McBane, Sean; Choi, Youngsoo (1 August 2021). "Component-wise reduced order model lattice-type structure design". Computer Methods in Applied Mechanics and Engineering. 381 (113813): 113813. arXiv: 2010.10770 . Bibcode:2021CMAME.381k3813M. doi:10.1016/j.cma.2021.113813. S2CID   224818337.
35. Lumley, J.L. (1967). The Structure of Inhomogeneous Turbulence," In: A. M. Yaglom and V. I. Tatarski, Eds., Atmospheric Turbulence and Wave Propagation. Moscow: Nauka.
36. Walton, S.; Hassan, O.; Morgan, K. (2013). "Reduced order modelling for unsteady fluid flow using proper orthogonal decomposition and radial basis functions". Applied Mathematical Modelling. 37 (20–21): 8930–8945. doi: 10.1016/j.apm.2013.04.025 . ISSN   0307-904X.
37. Lieu, T.; Farhat, C.; Lesoinne, M. (2006). "Reduced-order fluid/structure modeling of a complete aircraft configuration". Computer Methods in Applied Mechanics and Engineering. 195 (41–43): 5730–5742. Bibcode:2006CMAME.195.5730L. doi:10.1016/j.cma.2005.08.026. ISSN   0045-7825.
38. Xiao, D.; Yang, P.; Fang, F.; Xiang, J.; Pain, C.C.; Navon, I.M. (2016). "Non-intrusive reduced order modelling of fluid–structure interactions" (PDF). Computer Methods in Applied Mechanics and Engineering. 303: 35–54. Bibcode:2016CMAME.303...35X. doi: 10.1016/j.cma.2015.12.029 . ISSN   0045-7825.
39. Colciago, C.M.; Deparis, S.; Quarteroni, A. (2014). "Comparisons between reduced order models and full 3D models for fluid–structure interaction problems in haemodynamics". Journal of Computational and Applied Mathematics. 265: 120–138. doi: 10.1016/j.cam.2013.09.049 . ISSN   0377-0427.

Further reading

• Antoulas, Athanasios C. (2005). Approximation of Large-Scale Dynamical Systems. SIAM. doi:10.1137/1.9780898718713. ISBN   978-0-89871-529-3. S2CID   117896525.

• Benner, Peter; Fassbender, Heike (2014), "Model Order Reduction: Techniques and Tools" (PDF), Encyclopedia of Systems and Control, Springer, doi:10.1007/978-1-4471-5102-9_142-1, ISBN   978-1-4471-5102-9, S2CID   11873649

• Antoulas, A. C.; Sorensen, D. C.; Gugercin, S. (2001), "A survey of model reduction methods for large-scale systems" (PDF), Structured matrices in mathematics, computer science, and engineering, I (Boulder, CO, 1999), Contemporary Mathematics, vol. 280, Providence, RI: American Mathematical Society, pp. 193–219, CiteSeerX   10.1.1.210.9685 , doi:10.1090/conm/280/04630, ISBN   978-0-8218-1921-0, MR   1850408

• Benner, Peter; Gugercin, Serkan; Willcox, Karen (2013), A survey of model reduction methods for parametric systems (PDF)

• Baur, Ulrike; Benner, Peter; Feng, Lihong (2014), "Model order reduction for linear and nonlinear systems: a system-theoretic perspective" (PDF), Archives of Computational Methods in Engineering, 21 (4): 331–358, doi:10.1007/s11831-014-9111-2, S2CID   39068644

• Benner, Peter; Cohen, Albert; Ohlberger, Mario; Willcox, Karen (2017). Model Reduction and Approximation: Theory and Algorithms. SIAM Publications. doi:10.1137/1.9781611974829. ISBN   978-1-611974-81-2.

• Antoulas, Athanasios C.; Beattie, Christopher A.; Gugercin, Serkan (2020). Interpolatory Methods for Model Reduction. SIAM. doi:10.1137/1.9781611976083. ISBN   978-1-61197-607-6.

External links

• Model Order Reduction Wiki
• Model Reduction for Parametrized Systems
• European Model Reduction Network