Architecture / Building blocks

Tokens

Each mesh point becomes a vector of numbers describing where it is and what it carries.

A network cannot read a mesh. Each point (or patch, or cell) is turned into a token: a vector of numbers that describes where it is and what it carries.

• Inputs per point: position, surface normal, distance to the wall, flow condition, design parameters.
• Fourier features map coordinates to sines and cosines at many frequencies, so the network can resolve sharp gradients such as a leading-edge suction peak.
• A small MLP lifts these raw features into a d-dimensional embedding, typically 64 to 512 numbers per token.

Industrial example: Pressure taps on a car in the wind tunnel

An aerodynamicist drills pressure taps along a car body. Each tap reports where it is and what it measures. Write each reading as one row of numbers, and you have exactly what a network calls a token.

References

• Fourier Features Let Networks Learn High Frequency Functions (Tancik et al., NeurIPS 2020)
• Transolver: A Fast Transformer Solver for PDEs on General Geometries (Wu et al., ICML 2024)

Related

• Latent space: Compress millions of points into a few hundred learned slots that hold the physics.
• Self-attention: Every token looks at every other token and decides what matters to it.
• Representations: Voxel grids, graphs or point clouds: how geometry and fields become numbers.