|
Generative Model via Quantile Assignment |
Soundness: 2: fair
Presentation: 4: excellent
Contribution: 2: fair
Rating: 6: marginally above the acceptance threshold
Confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. |
This paper proposes a novel deep generative model (DGM) called NeuroSQL, which departs from the common framework of encoder+decoder (as in VAEs) or adversarial training (as in GANs). Instead it directly learns latent codes for each training datum via a quantile-assignment (linear assignment / Hungarian) to a pre-specified lattice of latent quantiles, and trains a generator network to map those codes to data.
The authors formulate a minimisation over both generator parameters θ and a permutation π that maps quantile codes to data
For multivariate (latent > 1D) codes they leverage optimal-transport/multivariate quantile theory and solve assignment between training data and a fixed uniform grid in latent space.
OpenReview
They propose an alternating algorithm: (i) keep π fixed, update θ via generator training; (ii) fix θ, update π via Hungarian algorithm assignment; optionally use momentum smoothing of assignments.
OpenReview
Empirically they evaluate on 4 domains (MNIST, CelebA, AFHQ animal faces, and OASIS brain images) under a “small‐budget / low-data” regime: e.g., training data capped at ~2 k images, resolution up to 128×128, single Google Colab budget.
OpenReview
The main claims: (1) NeuroSQL is more stable (no adversarial or encoder collapse issues), (2) it yields better or competitive image quality (measured via proxy FID, LPIPS, SSIM) under matched generator/backbone conditions, (3) it is more resource-friendly in low-data/high-dimension settings.
Interesting idea / novel paradigm — Replacing the encoder or discriminator with an explicit assignment of latent codes (quantile grid) is novel, and links generative modelling with statistical quantile/transport theory.
Pragmatic focus on low-data regimes — The paper addresses an important setting: generating synthetic data when the training dataset is small relative to high ambient dimension (e.g., neuroimaging) which is under-studied.
Theoretical underpinning — The use of quantile assignment and the convergence argument in the univariate / multivariate case gives formal support to the latent-code approximation strategy.
1. Resolution / dataset scale limited — Their experiments are constrained to small image resolutions (64×64-128×128) and relatively small sample sizes (~2 k images) under a limited compute budget. While this is aligned with their motivation (low-budget), it raises the question of how the method performs at larger, contemporary scales (e.g., 256×256, ImageNet scale). The authors acknowledge this in future work.
2. Interpretability of latent codes — Since the quantile grid is fixed and codes are assigned via permutation, the learned latent space may lack the structure/meaningfulness of e.g., disentangled VAE latents or hierarchical GAN latents. The paper doesn’t deeply analyze the semantics of the latent codes—are they smooth, do they support interpolation, manipulation, etc.
The paper assumes a fixed quantile lattice in latent space. How sensitive is the method to the choice of quantile grid (e.g., Sobol vs uniform vs Gaussian quantiles)?
The assignment problem is discrete, yet the generator is trained with continuous gradients. How do you ensure smooth convergence given the alternating discrete–continuous optimization? |
Fully AI-generated |
|
Generative Model via Quantile Assignment |
Soundness: 2: fair
Presentation: 2: fair
Contribution: 2: fair
Rating: 4: marginally below the acceptance threshold
Confidence: 5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully. |
This paper build a new structure for generative models, with less dimension.
1. this is a new structure
2. test with many different dataset and genearive framework
1. The expression of Figure 1 is unclear. From the image, it appears that the input data are fed into the decoder. The paper should clarify why this component is referred to as the decoder rather than the encoder, and explicitly describe what the input data are. Moreover, the roles of Momentum Update and Embedding in the framework are not clearly explained. What does “Cost” represent in this figure? Is it equivalent to the loss function?
Additionally, regarding the left-hand side of the figure, I speculate that it corresponds to the grey-shaded part on the right-hand side. However, it is not clear how the output on the left is transmitted to the generator. This connection should be explained more explicitly.
2. Section 3 mainly discusses the quantile assignment, but it should also explain how this mechanism is made trainable and why it is considered optimal. These claims should be supported by theoretical justification or experimental evidence.
3. Dataset and Metrics: The introduction of the dataset and evaluation metrics is not the core contribution of the paper and could be moved to the appendix or combined with the related work section to improve focus.
4. Diffusion Model Performance: The diffusion process seems to fail under the proposed method, which may be influenced by the linear assignment mechanism. Diffusion models often struggle with simple linear interpolation in latent space, resulting in abrupt transitions, artifacts, or degenerate (e.g., grey) images. This appears to be a limitation of the current approach. However, it might be mitigated by adopting smoothed diffusion models [1] or related approaches that enforce smoother, more linear latent mappings.
[1] Smooth Diffusion: Crafting Smooth Latent Spaces in Diffusion Models
check with weakness |
Lightly AI-edited |
|
Generative Model via Quantile Assignment |
Soundness: 3: good
Presentation: 3: good
Contribution: 4: excellent
Rating: 8: accept, good paper
Confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked. |
The paper introduces NeuroSQL, a generative modeling framework that learns latent variables through a quantile-assignment process derived from optimal transport, eliminating the need for an encoder or discriminator. The model alternates between a generator update and an assignment step solved via the Hungarian algorithm. This approach aims to combine stable, deterministic optimization with the expressiveness of deep decoders. Experiments span MNIST, CelebA, AFHQ, and OASIS, across multiple generator backbones (ConvNet, ResNet, U-Net), showing competitive visual quality and efficient convergence under low-data conditions.
The replacement of encoder–decoder mappings with an assignment-based quantile mechanism is conceptually fresh and theoretically grounded. It bridges optimal transport with generative modeling in a unique and elegant way.
The method is particularly well-suited for data domains like neuroimaging, where dimensionality exceeds sample size, and the assignment cost is independent of feature dimensionality.
Avoiding adversarial losses makes the model stable and lightweight to train. The simplicity of using an L2-based reconstruction objective allows reproducibility even in constrained computing environments.
The experiments show meaningful improvements in visual quality and diversity under limited data, highlighting NeuroSQL’s advantage.
While the paper provides an overall complexity estimate, quantitative comparisons to VAEs, GANs, or diffusion models in terms of runtime, memory, and scalability would provide stronger evidence of its efficiency. The Hungarian step’s cubic cost could be limiting for very large batch sizes, although mini-batching is suggested as a practical solution.
The performance advantage over GANs and VAEs is not uniform—some settings show weaker results, suggesting NeuroSQL's strengths may be inconsistent.
Why the quantitative diffusion comparisons under similar compute budgets are missing?
Do you see challenges extending this approach to transformer-based or high-resolution settings? |
Fully AI-generated |
|
Generative Model via Quantile Assignment |
Soundness: 2: fair
Presentation: 2: fair
Contribution: 2: fair
Rating: 2: reject
Confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. |
The paper proposes NeuroSQL, a latent-variable generative model that removes the encoder/discriminator and instead assigns latent codes by solving a linear assignment problem between data and a fixed quantile lattice of the latent prior. Training alternates between fitting a single generator to currently assigned codes and re-solving the assignment with a cost based on a perceptual/structural image loss; a momentum update smooths the assigned codes across iterations. For d>1, the quantiles are built via center-outward multivariate ranks from optimal transport; practically, a low-discrepancy grid on the unit ball is used. Experiments under a small-compute regime compare NeuroSQL against VAE, GAN, and a budget-matched diffusion baseline on MNIST, CelebA, AFHQ, and OASIS, showing improved visual quality and quantitative scores.
The paper makes a meaningful attempt to resolve the disadvantages of mainstream generative models VAEs and GANs, by removing the encoder and discriminator modules and integrating statistical quantile learning for stable training. The approach may be of interest in certain domains of generative tasks.
1. The main issue of the proposed method is scalability. The optimization algorithm runs in O(n^3) time with n being the number of samples. While the paper mentioned approximation via mini-batches, no concrete evidence is provided to show if it still works with large datasets (and models). The paper compares the method with VAEs, GANs and diffusion models in the seemingly fair budgeted setting. However, the comparison is not sound in that other models scale easier and perform much better with more budget. The budget, 200 Google Colab compute units and 2000 training images, is too limited for practical generative tasks.
2. Experimental results are not convincing to show the advantage of the proposed method. Images in Figure 2 are in low resolution making it hard to compare the visual quality. Quantitative results in Appendix show high instability across latent dimensions. In particular, in many cases the FIDs for NeuroSQL, VAE and CAN change significantly and non-monotonically as the latent dimension increases.
1. How does the method perform in mini-batches settings?
2. For quantitative results, how many runs were executed? Is unstable and insufficient training the cause for the varying evaluation scores? |
Fully human-written |