Architecture / Building blocks

Latent space

Compress millions of points into a few hundred learned slots that hold the physics.

Industrial meshes have millions of cells. Modern surrogates encode them into a small, fixed set of latent tokens, do the heavy computation there, and decode back to any point.

• Encode: N mesh points → M latent tokens, with M ≪ N (for example 8 million cells → 512 tokens).
• Process: a stack of transformer blocks works only on the M tokens, so cost no longer depends on mesh size.
• Decode: query any point in space and read the field back from the latent tokens. The output resolution is free.
• Time-dependent problems can be rolled out entirely in latent space, which is much faster than stepping the full mesh.

Industrial example: The one-page CFD report

A CFD run produces millions of cells, but an engineer summarises it in a handful of numbers: lift, drag, suction peak, where the flow stagnates, how strong the wake is. A good summary lets an expert sketch the whole flow back. A latent space is that summary, learned automatically.

References

• Universal Physics Transformers (Alkin et al., NeurIPS 2024)
• Latent Neural Operator for Forward and Inverse PDE Problems (Wang et al., NeurIPS 2024)
• AROMA: Latent PDE Modeling with Local Neural Fields (Serrano et al., NeurIPS 2024)
• Geometry-Informed Neural Operator (GINO) (Li et al., NeurIPS 2023)

Related

• Tokens: Each mesh point becomes a vector of numbers describing where it is and what it carries.
• Cross-attention: A small set of queries reads from a large set of points, so cost stays linear.
• JEPA: Predict the latent of the flow from geometry and condition; decode a field only when you need one.