|
From Cheap Geometry to Expensive Physics: Elevating Neural Operators via Latent Shape Pretraining |
Soundness: 2: fair
Presentation: 3: good
Contribution: 3: good
Rating: 4: marginally below the acceptance threshold
Confidence: 5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully. |
This paper introduces a two-stage operator learning framework that first learns a latent geometric representation using an occupancy prediction model, followed by conditional inference for query locations through a transformer-based neural operator. The method is validated on four 2D and 3D examples by comparing relative L2 errors across three major operator learning models. The results consistently demonstrate performance improvements over the baseline models. The paper also discusses how increased dataset variability enhances performance and analyzes the effect of different KL coefficients.
1. The idea presented in this paper is clear and well-motivated. The use of cross-attention for learning latent geometric representations is interesting and may have broader applicability to PDE-based surrogate modeling approaches.
2. Table 2 provides a comprehensive comparison between the proposed method and three other approaches, enhancing the credibility of the results.
3. The ablation studies are clearly presented and address two important aspects: the effect of KL weights and the impact of geometric data variability.
1. The paper mainly evaluates model performance in terms of relative L2 error. However, introducing Stage 1 may introduce additional computational costs, which are not measured. Moreover, the advantages of transforming the original point cloud into a latent geometric representation are not fully explored. For instance, could this help reduce inference time for temporal propagation since the propagation occurs in latent space?
2. Table 6 shows that an AE alone suffices for the AirfRANS dataset, making VAE unnecessary. A similar analysis on the other datasets would help clarify this observation.
3. Finally, there is no comparison with other geometry-agnostic methods, such as “Li, Zongyi, et al. Geometry-informed neural operator for large-scale 3D PDEs. Advances in Neural Information Processing Systems 36 (2023): 35836–35854.” Without such comparisons, it is difficult to assess the effectiveness of the proposed method fully.
1. A clear computational cost study is needed to show a balance of the performance win and the speed reduction since the proposed approach introduces an additional stage compared to other models.
2. How to address problems with sharp boundaries and more chaotic dynamics, such as turbulence, with the proposed approach? |
Lightly AI-edited |
|
From Cheap Geometry to Expensive Physics: Elevating Neural Operators via Latent Shape Pretraining |
Soundness: 3: good
Presentation: 2: fair
Contribution: 1: poor
Rating: 2: reject
Confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. |
The authors of this work proposed a novel deep learning framework for the prediction of quantities of interest in computational mechanics such as stress, pressure, and velocity, where the solution of PDEs depend on the geometry of the computational domain. The tested their network by choosing dataset from fluid, solid, and electrical engineering and showed the performance by reporting relative errors in L2 norm. The study includes both 2D and 3D examples. At high level, the network has two stages, in the first stage, there is an encoder that only learns the geometric feature of computational domain from a set of point clouds. In stage 2, there is operator learning to learn PDE variables. Combing the features learned from stage 1 and stage 2, the network predicts variables in query coordinates.
The article is about an important subject in computational mechanics where we would like to accelerate geometric design, depending on various parameters, and we use deep learning frameworks for this goal. So, the topic is important. The authors investigated the performance using different datasets both from fluid and solid mechanics as well as electrical engineering (rather than only one dataset or one specific application).
As I mentioned in the Strengths section, the problem has been well defined. But I strongly believe that to overcome the challenge of this problem, there have been a class of point-cloud based neural networks (e.g., PointNet, PointNet++, etc.) that have been used for computational mechanics problems. PointNet has been introduced in 2017 and the usage of PointNet in CFD was introduced in 2021 for the first time and later on, there have been so many usage of PointNet in this area. Here is a list of some papers:
PointNet and PointNet ++ in computer graphics:
https://openaccess.thecvf.com/content_cvpr_2017/html/Qi_PointNet_Deep_Learning_CVPR_2017_paper.html
https://proceedings.neurips.cc/paper/2017/hash/d8bf84be3800d12f74d8b05e9b89836f-Abstract.html
PointNet for CFD:
https://doi.org/10.1063/5.0033376
PointNet for PINNs:
https://doi.org/10.1016/j.jcp.2022.111510
PointNet for Operator Learning:
https://doi.org/10.1016/j.cma.2024.117130
PointNet for KANs:
https://doi.org/10.1016/j.cma.2025.117888
None of these papers have been cited by the authors. So, I imagine that they have been not aware of these methodology. Because these networks handle the same issue mentioned by the authors in more efficient ways. There is only one network that learns both geometric features and also PDE variables (e.g., velocity, pressure, temperature, etc.) simultaneously. The proposed method in this article, while works, but it is very complicated in comparison. It has two stages and requires pre and post processing as explained by the authors. In simple words, the authors tried to learn geometry and variables in two different networks, while both can be learned simultaneously in one network. I like the idea; however, we need to introduce something more efficient compared to what have been developed so far; otherwise, what is the motivation for using this new method.
A few other comments:
I think Figure 1 is redundant, Figure 2 can be also presented better, in the main body of the article, we could have more visual results and comparison between different methods. There are some bold words in abstract, I do not think it is good to make them bold. Introduction is relatively long for ICLR and the methodology has been explained very briefly. Moreover, in the results, again there are some details about the dataset which could be moved to the supplementary sections. Numbers in tables are written using a large, unusual font sizes.
Have you ever tried PointNet, PointNet++, and other point-cloud based neural networks for solving these problems? Please see the details of my explanation in the Weaknesses section. |
Fully human-written |
|
From Cheap Geometry to Expensive Physics: Elevating Neural Operators via Latent Shape Pretraining |
Soundness: 4: excellent
Presentation: 4: excellent
Contribution: 4: excellent
Rating: 8: accept, good paper
Confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. |
The paper proposes a two-stage training framework for neural operator learning that leverages geometry only data (like cheap, unlabeled, physics-agnostic data) to improve performance on PDE solution predictions (via expensive, label-scarce physics data). This approach combines representation learning and operator learning, allowing physics-informed models to benefit from geometry-only datasets. This is typically ignored in standard PDE modeling.
* well-motivated framework bridging geometric pretraining and operator learning. PDE-based simulations (like CFD, elasticity, electromagnetics and so on) are computationally expensive, while 3D geometries are normally cheap to obtain.
* The occupancy field proxy is neat. It’s simple, generalizable and physics-agnostic. It seems to effectively force the encoder to learn spatial and structural priors that are transferrable to PDE learning tasks.
* The authors test some alternative proxy tasks (like signed distance fields, shortest vectors etc) and show comparable or improved results, especially for CFD datasets.
* There is nice integration with Transformer-based Neural Operators. The latent embeddings are seamlessly fed into three state-of-the-art transformer-based neural operators. This makes for broad applicability and makes the contribution compatible with ongoing work in neural operator transformers.
* Demonstrated generality across problems and architectures
* Good experiments
* The authors are honest about limitations
* Lack of joint optimization between pretraining and PDE stages.
* Limited exploration of theoretical connections between learned latent and PDE operator manifolds.
* Real-world benchmarks (e.g., turbulence, multiphysics coupling and stuff like that) could further validate scalability. The latter is important yet not discussed enough.
* The authors themselves allude to some limitations / weaknesses. Complex geometries with multiple interacting components may require multi-class or more expressive proxies. The lack of joint fine-tuning may limit performance.
On page 1 you comment on fixed-grid structures. Don’t you mean fixed topology?
Pg2 input function space is assumed to contain only geometry info. How much of a limitation do you see this as, and how might this be extended?
Pg4 you define \epsilon \sim N(0,1). Then on pg 5 you have \epsilon as the scale param in another N(0,\epsilon I). The latter use implies \epsilon > 0, at odds with the first usage definition.
Pg6 you exclude hyperparam tuning. I get your focus on the 2-stage training – but surely one of the selling points of the work is the superior performance, so it’s important to understand better the context of the reported results. Can you comment more?
Pg 7 – re. comment about unnorm data sets and MSE. Can’t you present NMSE, or R^2?
References – you should protect capitals – like {N}avier-{S}tokes etc. |
Fully human-written |
|
From Cheap Geometry to Expensive Physics: Elevating Neural Operators via Latent Shape Pretraining |
Soundness: 3: good
Presentation: 3: good
Contribution: 2: fair
Rating: 4: marginally below the acceptance threshold
Confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. |
Pre-training of neural operators using geometry information.
- Well written. The author motivated the geometry-based pre-training strategy well.
- Considered various benchmark problems.
- Considered various recently proposed backbone architectures.
- Error bars.
- line 53: This method is also limited to varying geometry problems. To say something about applicability to general PDE problems, it is necessary to show effectiveness on a non-varying geometry problem.
- Relative $L^2$ errors are too high—for engineering applications, two to three digits of accuracy is usually recommended. However, more than 10% of error makes it difficult to see whether the improvement comes from the proposed method or optimization error.
- The paper tackled data scarcity. However, the reader may not agree that more than 800 training data points are scarce. Especially, the AirfRans dataset provides data data-scarce scenario. Reporting the result in such a scenario may boost the credibility of this work.
- Asserting the encoder as a learned chart for the moduli space is good, but there was no quantitative validation of it. Reporting the distance between two latent codes from two point clouds representing the same geometry would improve the claim.
- Perhaps, further truncation of the AirfRans domain was due to failure in the original domain. (In fact, the original domain of the AirfRans dataset had been truncated already.)
- Training time ratio for 1st and 2nd stages?
- LNO is the most recent, but performed the worst. Could you comment on that? |
Fully human-written |