|
Real-IKEA : Simulating What Robots Will Really See and Touch |
Soundness: 2: fair
Presentation: 2: fair
Contribution: 2: fair
Rating: 2: reject
Confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked. |
This paper proposes Real-IKEA, a dataset and simulation framework based on real IKEA furniture, which mitigates the asset gap, physics gap and vision gap in simulation environments. For asset generation, the authors combine reusable handles with base cabinet units to create the Real-IKEA dataset. Building on this dataset, the framework also enables high fidelity physical interaction simulation and high-quality visual rendering by reconstructing collision meshes with the COACD algorithm and using a hybrid rendering pipeline. In addition, the authors test different manipulation policies in their simulation environment for validation.
1. The proposed Real-IKEA dataset is valuable in robotics research as its assets closely match real-world asset distributions. This helps mitigate the sim-to-real gap in robotics policy deployment.
2. Multiple quantitative evaluations of asset quality and physical-interaction fidelity are helpful for understanding the characteristics of the proposed framework.
1. The paper does not present many novel contributions. From Section A.1, the object meshes were already available before this work, and the dataset creation is mostly mesh selection and segmentation. Also, for the claimed contributions in realistic physical interaction and visual rendering, the benefits come from using prior methods such as the COACD algorithm and combining existing renderers, which are not contributions of this paper itself.
2. The evaluation in Figure 6 is not fair, as the background colors in the baseline simulator and the real-world frames differ a lot, while the background color of the proposed framework and the real-world frames is similar, which would influence the FID evaluation. Since background color can be easily set in simulators, a fairer comparison is to set the background color of all simulators the same and report results in that setting.
3. There is no citation of the COACD algorithm on Line 254.
1. How do the proposed framework’s physical interaction realism and visual fidelity compare to prior simulation environments? Are there quantitative results?
2. Are the original meshes for the dataset publicly available and what is the detailed procedure of creating the dataset? These aspects should also be discussed in the paper. |
Fully human-written |
|
Real-IKEA : Simulating What Robots Will Really See and Touch |
Soundness: 3: good
Presentation: 3: good
Contribution: 2: fair
Rating: 2: reject
Confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. |
This paper introduces a new dataset of 3D assets of IKEA cabinets with realistic visuals and collision geometry, diverse coverage, and configurable joint resistance.
This dataset is useful and could help the community's research on contact-rich manipulation of articulated objects based on simulation. The more accurate collision geometry makes simulation more accurate and shrinks the sim-to-real gap.
The contribution of this paper is majorly the new dataset. It definitely has value, as I said, but there are few innovations or insights either to learning in general or robot manipulation. The construction of the more accurate collision geometry and the higher-fidelity visualization are using off-the-shelf methods. The physics parameters (the resistance) are manually specified, which is not grounded in the real data. This paper might be a better fit for robotics venues.
Is the bidirectional surface-deviation metric just chamfer distance?
I didn't quite understand Figure 6. How did you get the corresponding images in the real-world? The teleoperation is done in simulation, right?
A lot of details of the case study is missing. How is success defined? Are multiple attempts allowed? Does human teleoperator know the resistance level of the task in prior? How forceful is the grasp, or it is not in the parameter space of grasps? |
Fully human-written |
|
Real-IKEA : Simulating What Robots Will Really See and Touch |
Soundness: 2: fair
Presentation: 2: fair
Contribution: 2: fair
Rating: 2: reject
Confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked. |
This paper introduces Real-IKEA, an articulated object dataset for contact-rich articulated object manipulation. The assets are constructed from real IKEA furniture bases paired with 83 authentic IKEA handles and knobs, yielding 1,079 articulated configurations. The work emphasizes the three axis of sim2real gap: object geometry, physics realism, and visual fidelity, by COACD-based collision modeling and a hybrid rendering pipeline. The paper provides quantitative evaluations of collision modeling accuracy, visual realism, and manipulation success rates under varying joint resistance.
1. The use of real IKEA furniture bases and accurately modeled handles/knobs gives the dataset strong grounding in real-world distributions.
2. The bidirectional metric and use of the COACD algorithm for convex decomposition improves the collision fidelity for handles and knobs, which are shown to be important for contact-rich manipulation.
3. The two-stage rendering process (real-time + offline re-rendering) narrows the vision gap, supported by quantitative FID/EMD improvements and t-SNE visualizations.
4. The human-in-the-loop evaluation is interesting. The inclusion of teleoperation studies and human performance benchmarks provides insight into how accurate handle modeling influences manipulation strategies.
5. The case study on drawer opening under varying resistance levels is an interesting design that highlights the limitations of friction-dominated policies and the importance of accurate collision modeling.
1. Limited Downstream Validation: Although the dataset shows improved FID/EMD and collision accuracy, it remains unclear how much these improvements translate to real policy performance, e.g. visual sim2real manipulation policy transfer. Including at least one downstream comparison (training in Real-IKEA and testing on a real robot) would strengthen the claim.
2. Comparison Scope: While comparisons with PartNet-Mobility and AdaManip are provided, a direct quantitative or qualitative comparison with GAPartManip (ICRA 2024) or similar large-scale articulated manipulation datasets would help contextualize the novelty.
3. Missing experimental details: The experimental details for Tables 2 and 3 are incomplete. In particular, sample sizes and human study protocols are missing.
4. Missing related works: Since one major contribution of the paper is to mitigate the sim2real gap, it should include the Real2Sim2Real line of work in robotics that aim to mitigate the visual[1,2] and/or physics[3] gaps. Distinguish Real-IKEA to this body of work would clarify its contribution.
[1] Li, Xinhai, et al. "Robogsim: A real2sim2real robotic gaussian splatting simulator." arXiv preprint arXiv:2411.11839 (2024).
[2] Yu, Justin, et al. "Real2render2real: Scaling robot data without dynamics simulation or robot hardware." arXiv preprint arXiv:2505.09601 (2025).
[3] Lim, Vincent, et al. "Real2sim2real: Self-supervised learning of physical single-step dynamic actions for planar robot casting." 2022 International Conference on Robotics and Automation (ICRA). IEEE, 2022.
1. For Table 2 (FID/EMD):
How many real-world videos were used for computing the distribution distances? Were these captured in a controlled lighting environment, or across multiple real settings?
2. For Table 3 (Success Rates):
How many objects, human subjects, and trials per condition were used to compute the success-rate distributions?
3. How were the damping and friction parameters (2/5, 5/15, 10/30) determined. Is it through physical measurement on real furniture, or empirical tuning in simulation?
4. The paper mentioned the knobs and handles are systematically combined with the base cabinets, but there are missing details on how the attach locations and rotation axises are generated. |
Fully human-written |
|
Real-IKEA : Simulating What Robots Will Really See and Touch |
Soundness: 3: good
Presentation: 2: fair
Contribution: 3: good
Rating: 6: marginally above the acceptance threshold
Confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. |
The paper introduces Real-IKEA, a benchmark and simulation platform for robot manipulation. Utilizing realistic articulated assets curated from real-world IKEA furniture, employing MuJoCo for physics simulation, and carefully designing the collision meshes via CoACD convex decomposition, Real-IKEA makes a step forward in the field of articulated object manipulation simulation platforms. Authors further accompany this platform with a benchmark contribution to demonstrate the value of Real-IKEA in pushing forward the development of robot articulated object manipulation.
- Nice motivation. A realistic and high-performant robot articulated object manipulation platform is a key to advancing the development of robot manipulation policies. This paper identifies three critical challenges in robot manipulation policy sim-to-real transfer and contributes a more realistic simulation platform and benchmark to address the underlying problem.
- Good contributions. The paper makes a solid contribution in aspects of realistic furniture asset curation, accurate convex approximation, and realistic visual rendering.
- The major weakness is the lack of technical contribution. The authors have made a huge amount of efforts in building the benchmark. However, the technical contribution is weak. To improve the physical simulation fidelity, the paper incorporates two existing techniques/simulators, including CoACD convex approximation and the MuJoCo simulator, with no new decomposition strategies or simulation methods proposed. The solution to improve visual rendering is also to combine existing techniques.
- Restricted coverage of the asset and limited manipulation scenarios. Real-IKEA primarily focuses on drawer-style furniture. The benchmarked manipulation scenarios are mainly pulling out by contacting the handle. The asset is limited in diversity, compared to SAPIEN. Besides, the manipulation scenario is oversimplistic.
- How about the simulation efficiency when using fine convex decomposed meshes?
- Could it support parallel simulation?
- What does "reality see and touch" mean? Does Real-IKEA reconstruct materials from real-world furniture or does it support tactile simulation as well as tactile policy training? |
Fully human-written |