|
GLLP: Graph Learning from Label Proportions |
Soundness: 3: good
Presentation: 3: good
Contribution: 3: good
Rating: 4: marginally below the acceptance threshold
Confidence: 2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked. |
This paper studies the problem of learning from label proportion (LLP) for graph data. In particular, the paper devises a very interesting method GLLP, which decomposes the LLP problems into two: one is to explore optima transfer (OT) to derive soft pseudo-labels for nodes based on a bag-level label distribution and the node-level prediction from GNN models; the other is to train the GNN model based on the derived pseudo-labels. GLLP has alternations between these two steps, pseudo-label generation and GNN training, which is rather effective to iteratively refine node-level predictions and remain consistent with bag-level proportions. Experimental evaluation gives clear evidence to demonstrate the strength of the GLLP over other different proposals.
-- The paper addresses an important problem of LLP for graph data.
-- It presents a first proposal for the research problem and the main idea is quite appealing to me.
-- The paper is well-written and the experimental results give strong support for the proposed method.
-- The theoretical analysis is Section 3.3 is quite interesting to me, but the proof in the appendix is rather difficult for me to understand.
-- There are quite some typos in the paper: (page 3, line 125; page 4, line 194; page 5, line 228, line 236, line 248, line 254, line 260; page 6, line 299, etc. also a few typos on page 12).
1. What is exactly the temperature \tau? In the experiments, it is simply set at 2. Why? How does its value impact the results?
2. In general, can you make the proof on page 12 more detailed?
Especially, how to expand Eq. 12 into Eqs. 26 and 27? I also don't understand the last sentence "Leveraging the assumption in Eq. 15, we can obtain the final conclusion". I thought Eq. 15 is what you aim to prove.
Once these are resolved, I am willing to improve my overall rating. |
Fully human-written |
|
GLLP: Graph Learning from Label Proportions |
Soundness: 2: fair
Presentation: 3: good
Contribution: 2: fair
Rating: 2: reject
Confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. |
This paper introduces Graph Learning from Label Proportions (GLLP), extending the Learning from Label Proportions (LLP) paradigm to graph-structured data for node classification tasks. The authors highlight that the bag-level supervision yields weak signals. To address this, they propose to leverage optimal transport and graph penalty terms, with soft pseudo-labels.
1. The paper study an underexplored LLP problem in graph domain.
2. The authors propose to generate pseudo-label based on graph structure inductive bias.
3. The writing is easy to understand.
* While the introduction emphasizes LLP's applicability in scenarios where node-level labels are infeasible or undesirable, the experiments rely on standard homophilic benchmarks like citation networks (e.g., Cora, CiteSeer), Amazon product graphs, and WikiCS, which do not inherently reflect these constraints. Labels are readily available in these datasets, potentially undermining the method's real-world validation.
* The proposed method generates pseudo-labels based solely on bag-level proportions and graph structure. Intuitively, for a given bag and its proportion, there is a vast solution space of possible node label assignments that are consistent with both the proportion and the graph's homophily. The paper would be strengthened by a deeper investigation into this ambiguity, such as an analysis of the stability of the pseudo-labels or the sensitivity of the final results to different initializations, which is currently lacking.
* Figure 3, intended to illustrate the Optimal Transport process, is not sufficiently detailed to enhance reader understanding.
* For Theorem 1, the critical homophily assumption is not formally defined in the main text. The statement "Under the homophily assumption..." is vague.
* The experimental setup follows prior LLP works by using random sampling to create bags. However, in a graph context, random sampling can be detrimental as it may arbitrarily fracture local community structures.
* Only evaluate a single GCN backbone.
* The code is not available.
* More evaluation on data with consistent and reasonable constraints.
* Detailed theorem presentation.
* Reproducible issues. |
Lightly AI-edited |
|
GLLP: Graph Learning from Label Proportions |
Soundness: 2: fair
Presentation: 3: good
Contribution: 2: fair
Rating: 2: reject
Confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. |
This paper introduces GLLP, a framework that extends Learning from Label Proportions (LLP) to graph-structured data, where nodes are interdependent. The method employs Optimal Transport with a homophily-aware cost to generate soft pseudo-labels for nodes, enabling effective node classification under distributional supervision. Theoretical analysis and experiments demonstrate that GLLP outperforms existing LLP baselines.
1. Applying LLP to graph structures is a novel contribution.
2. The proposed method is consistent with the theoretical analysis.
1. My main concern is the lack of real-world scenarios for the proposed graph LLP problem. The paper does not present convincing real-world applications, and the experiments are conducted only on synthetic datasets.
2. For the proposed graph LLP setting, the paper should discuss more carefully the role of edges — both between different bags and within each bag. Without such analysis, simply applying the LLP framework to graph data is not particularly meaningful.
3. Using only synthetic datasets in the experiments is insufficient to validate the practicality and relevance of the proposed graph LLP setting.
on weakness |
Lightly AI-edited |
|
GLLP: Graph Learning from Label Proportions |
Soundness: 3: good
Presentation: 3: good
Contribution: 3: good
Rating: 6: marginally above the acceptance threshold
Confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. |
This paper studied a new problem, i.e. LLP, in graph domain, where we only know the label distributions over each bag of nodes, rather than node-level labels. The authors proposed two-level losses, the first is bag-level KL loss, and the second is node-level supervison, got by OT. Experiments showed good performances of the technique.
1. The authors involved an interesting problem, Learning from Lable Proportion, into graph domain.
2. The theoretical and experimental stuff supports their techniques.
3. The organization is clear and easy to follow.
1. Learning from Label proportion sounds more like an industrial scenario. For users' privacy, we have to mask sensitive labels. Also in introduction, the authors mentioned online advertising. But all experments were done on non-industrial datasets. The used datasets, like Cora, Citeseer, never have the requirements of masking labels. So, I suggest the authors to try their techniques on some industrial cases.
2. The authors used optimal transport to get pssudo-labels. OP has a high cost O(N^2). I concern about scalability and efficiency if the authors compute OT every epoch.
As in Eq. (9), the cost matrix C is obtained based on logits, i.e., the GNN output. But if GNN is not well-trained, especially at the early stage, the quality of the cost matrix could be low. Why not try to decompose bag-level label proportions into node-level pesudo-signals, and use these signals, rather than logits, to construct cost matrix? |
Fully human-written |