{
  "schemaVersion": 1,
  "benchmark": "robotwin_nvidia10",
  "description": "Verbatim published instructions for the eight RoboTwin examples used in the article. Baseline instructions come from the original run catalog; revised instructions come from the goal-spec rerun catalog. No instruction text is reconstructed or paraphrased.",
  "boundaryMarker": "\n\nPublic goal criteria derived from the benchmark: ",
  "baselineSource": {
    "repository": "https://github.com/tianqi-zh/robot-agent-gallery",
    "revision": "72d27cffba20b3bb5b56945f5f35f666f2568808",
    "path": "data/gallery.json",
    "url": "https://github.com/tianqi-zh/robot-agent-gallery/blob/72d27cffba20b3bb5b56945f5f35f666f2568808/data/gallery.json",
    "sha256": "71aa30f5b2c388f12e50cf951f7a8a3b9d7e59f132c0bd74f559dd7512aa5987",
    "run": "robotwin_nvidia10_full_v1",
    "manifestSha256": "348c4f7a97e28e763a9c46158e60d164c5a95708093dbfbc178b860416200694",
    "field": "benchmarks[id=robotwin_nvidia10].tasks[].episodes[sourceEpisodeKey].instruction"
  },
  "revisedSource": {
    "repository": "https://huggingface.co/spaces/Alan0928/robot-agent-gallery",
    "path": "data/gallery.json",
    "url": "https://huggingface.co/spaces/Alan0928/robot-agent-gallery/blob/main/data/gallery.json",
    "sha256": "aca0bfc4a669b553a8cf23730f5b1c5d5264790f5b2b50c61a0b1680013cf4bc",
    "run": "robotwin_goal_spec_full_v1",
    "manifestSha256": "5044e7c5eaed3389b689c0534c0863966d5ee1b3fbafc9e4990a0c85960a5f88",
    "field": "benchmarks[id=robotwin_nvidia10].tasks[].episodes[sourceEpisodeKey].instruction"
  },
  "notes": [
    "The goal-spec appendix includes explicit numerical tolerances and, in some tasks, quaternion targets; it is richer than the original task sentence.",
    "The added goal spec is separated from the revised episode instruction using boundaryMarker.",
    "Six of these eight examples preserve the baseline scene seed and base instruction. scan_object_r01 and dump_bin_bigbin_r00 use different resolved base instructions and scene seeds in the rerun.",
    "These examples retain native benchmark outcomes. A completion claim is not an independent human annotation or ground-truth success label."
  ],
  "cases": [
    {
      "episodeKey": "adjust_bottle_r00",
      "taskName": "adjust_bottle",
      "baselineInstruction": "Position the bottle with white neck and red top head-up and lift it",
      "revisedBaseInstruction": "Position the bottle with white neck and red top head-up and lift it",
      "addedGoalSpec": "Public goal criteria derived from the benchmark: bottle functional point height z > 0.90 m; functional bottle point must be on the right side of the table: x > 0.15 m.\nBefore finishing, verify: bottle is visibly upright/head-up; bottle is high enough; bottle is on the correct side.",
      "revisedInstruction": "Position the bottle with white neck and red top head-up and lift it\n\nPublic goal criteria derived from the benchmark: bottle functional point height z > 0.90 m; functional bottle point must be on the right side of the table: x > 0.15 m.\nBefore finishing, verify: bottle is visibly upright/head-up; bottle is high enough; bottle is on the correct side.",
      "baselineSeed": 100001,
      "revisedSeed": 100001,
      "sameSceneSeed": true,
      "sameBaseInstruction": true,
      "baselineBenchSuccess": false,
      "revisedBenchSuccess": true,
      "comparisonNote": "The baseline and revised records use the same resolved base instruction and seed."
    },
    {
      "episodeKey": "place_mouse_pad_r08",
      "taskName": "place_mouse_pad",
      "baselineInstruction": "Use the left arm to grab the dark gray mouse and place it on the Black colored mat.",
      "revisedBaseInstruction": "Use the left arm to grab the dark gray mouse and place it on the Black colored mat.",
      "addedGoalSpec": "Public goal criteria derived from the benchmark: mouse xy must be within about 1.5 cm x and 1.2 cm y of target; mouse orientation must match the pad orientation criterion; both grippers are open.\nBefore finishing, verify: mouse is centered and aligned on pad, not merely on top; hands have released.",
      "revisedInstruction": "Use the left arm to grab the dark gray mouse and place it on the Black colored mat.\n\nPublic goal criteria derived from the benchmark: mouse xy must be within about 1.5 cm x and 1.2 cm y of target; mouse orientation must match the pad orientation criterion; both grippers are open.\nBefore finishing, verify: mouse is centered and aligned on pad, not merely on top; hands have released.",
      "baselineSeed": 100008,
      "revisedSeed": 100008,
      "sameSceneSeed": true,
      "sameBaseInstruction": true,
      "baselineBenchSuccess": false,
      "revisedBenchSuccess": true,
      "comparisonNote": "The baseline and revised records use the same resolved base instruction and seed."
    },
    {
      "episodeKey": "rotate_qrcode_r00",
      "taskName": "rotate_qrcode",
      "baselineInstruction": "Grab the triangular stand with QR code, lift it from the table, and rotate it QR front",
      "revisedBaseInstruction": "Grab the triangular stand with QR code, lift it from the table, and rotate it QR front",
      "addedGoalSpec": "Public goal criteria derived from the benchmark: QR code orientation near quaternion [0.707, 0.707, 0, 0] within about 0.05 per component; QR code is down near table height, z < 0.75 m plus table bias; both grippers are open.\nBefore finishing, verify: QR face has the target orientation; hands have released.",
      "revisedInstruction": "Grab the triangular stand with QR code, lift it from the table, and rotate it QR front\n\nPublic goal criteria derived from the benchmark: QR code orientation near quaternion [0.707, 0.707, 0, 0] within about 0.05 per component; QR code is down near table height, z < 0.75 m plus table bias; both grippers are open.\nBefore finishing, verify: QR face has the target orientation; hands have released.",
      "baselineSeed": 100000,
      "revisedSeed": 100000,
      "sameSceneSeed": true,
      "sameBaseInstruction": true,
      "baselineBenchSuccess": false,
      "revisedBenchSuccess": true,
      "comparisonNote": "The baseline and revised records use the same resolved base instruction and seed."
    },
    {
      "episodeKey": "scan_object_r01",
      "taskName": "scan_object",
      "baselineInstruction": "Pick up the black tea-box with gold logo and the yellow scanner with buttons in parallel with the right arm and the left arm, then scan.",
      "revisedBaseInstruction": "Grab the smooth black tea-box and the scanner with flat reading surface at the same time and scan the smooth black tea-box with the scanner with flat reading surface.",
      "addedGoalSpec": "Public goal criteria derived from the benchmark: scanner forward ray aligns to object within about 2.5 cm; object-to-scanner distance along scan ray is >0 and <7 cm; both grippers are closed.\nBefore finishing, verify: scanner tip/window is aimed directly at the object at close range; both hands keep holding/closing as required.",
      "revisedInstruction": "Grab the smooth black tea-box and the scanner with flat reading surface at the same time and scan the smooth black tea-box with the scanner with flat reading surface.\n\nPublic goal criteria derived from the benchmark: scanner forward ray aligns to object within about 2.5 cm; object-to-scanner distance along scan ray is >0 and <7 cm; both grippers are closed.\nBefore finishing, verify: scanner tip/window is aimed directly at the object at close range; both hands keep holding/closing as required.",
      "baselineSeed": 100002,
      "revisedSeed": 100006,
      "sameSceneSeed": false,
      "sameBaseInstruction": false,
      "baselineBenchSuccess": false,
      "revisedBenchSuccess": false,
      "comparisonNote": "The baseline and revised records share an episode key but use different resolved base instructions and scene seeds; this is not a matched-scene instruction-only comparison."
    },
    {
      "episodeKey": "move_can_pot_r01",
      "taskName": "move_can_pot",
      "baselineInstruction": "Move the hand-held sauce container beside the round blue-lid kitchenpot for stovetop after lifting it",
      "revisedBaseInstruction": "Move the hand-held sauce container beside the round blue-lid kitchenpot for stovetop after lifting it",
      "addedGoalSpec": "Public goal criteria derived from the benchmark: can must be on the arm-dependent side of the pot; can and pot y positions align within about 3.5 cm; can roll orientation is near 90 degrees and pitch near 0; can is back down on the table; both grippers are open.\nBefore finishing, verify: can is beside the pot on the required side; can is lying in the expected orientation; hands have released.",
      "revisedInstruction": "Move the hand-held sauce container beside the round blue-lid kitchenpot for stovetop after lifting it\n\nPublic goal criteria derived from the benchmark: can must be on the arm-dependent side of the pot; can and pot y positions align within about 3.5 cm; can roll orientation is near 90 degrees and pitch near 0; can is back down on the table; both grippers are open.\nBefore finishing, verify: can is beside the pot on the required side; can is lying in the expected orientation; hands have released.",
      "baselineSeed": 100001,
      "revisedSeed": 100001,
      "sameSceneSeed": true,
      "sameBaseInstruction": true,
      "baselineBenchSuccess": false,
      "revisedBenchSuccess": false,
      "comparisonNote": "The baseline and revised records use the same resolved base instruction and seed."
    },
    {
      "episodeKey": "pick_dual_bottles_r00",
      "taskName": "pick_dual_bottles",
      "baselineInstruction": "Use each arm to grab one of the plastic drink bottle and the medium green bottle with capped neck.",
      "revisedBaseInstruction": "Use each arm to grab one of the plastic drink bottle and the medium green bottle with capped neck.",
      "addedGoalSpec": "Public goal criteria derived from the benchmark: each bottle xy is within about 10 cm of its assigned target pose; each bottle functional point height z > 0.89 m.\nBefore finishing, verify: both bottles are held/lifted; each bottle is over the correct side/target zone.",
      "revisedInstruction": "Use each arm to grab one of the plastic drink bottle and the medium green bottle with capped neck.\n\nPublic goal criteria derived from the benchmark: each bottle xy is within about 10 cm of its assigned target pose; each bottle functional point height z > 0.89 m.\nBefore finishing, verify: both bottles are held/lifted; each bottle is over the correct side/target zone.",
      "baselineSeed": 100000,
      "revisedSeed": 100000,
      "sameSceneSeed": true,
      "sameBaseInstruction": true,
      "baselineBenchSuccess": false,
      "revisedBenchSuccess": false,
      "comparisonNote": "The baseline and revised records use the same resolved base instruction and seed."
    },
    {
      "episodeKey": "place_dual_shoes_r00",
      "taskName": "place_dual_shoes",
      "baselineInstruction": "Move two the medium white shoe into the rectangular shoe-box with orange lid, ensure tips point left.",
      "revisedBaseInstruction": "Move two the medium white shoe into the rectangular shoe-box with orange lid, ensure tips point left.",
      "addedGoalSpec": "Public goal criteria derived from the benchmark: left and right shoes must land at the two target offsets around y=-0.13; each shoe xy tolerance about 5 cm; each shoe orientation near quaternion [0.5, 0.5, -0.5, -0.5]; shoe height is near shoebox support height; both grippers are open.\nBefore finishing, verify: both shoes are inside box, parallel and correctly oriented; hands have released.",
      "revisedInstruction": "Move two the medium white shoe into the rectangular shoe-box with orange lid, ensure tips point left.\n\nPublic goal criteria derived from the benchmark: left and right shoes must land at the two target offsets around y=-0.13; each shoe xy tolerance about 5 cm; each shoe orientation near quaternion [0.5, 0.5, -0.5, -0.5]; shoe height is near shoebox support height; both grippers are open.\nBefore finishing, verify: both shoes are inside box, parallel and correctly oriented; hands have released.",
      "baselineSeed": 100001,
      "revisedSeed": 100001,
      "sameSceneSeed": true,
      "sameBaseInstruction": true,
      "baselineBenchSuccess": false,
      "revisedBenchSuccess": false,
      "comparisonNote": "The baseline and revised records use the same resolved base instruction and seed."
    },
    {
      "episodeKey": "dump_bin_bigbin_r00",
      "taskName": "dump_bin_bigbin",
      "baselineInstruction": "Lift the simple white plastic trashbin using the arm to pour balls.",
      "revisedBaseInstruction": "Pick up the smooth tabletrashbin for small waste using the arm, pour balls.",
      "addedGoalSpec": "Public goal criteria derived from the benchmark: deskbin height z >= 1.0 m; every garbage sphere must end in the bin-height band, roughly z 0.13 to 0.25 m.\nBefore finishing, verify: all small items have left the desk bin; all contents are inside the big bin area.",
      "revisedInstruction": "Pick up the smooth tabletrashbin for small waste using the arm, pour balls.\n\nPublic goal criteria derived from the benchmark: deskbin height z >= 1.0 m; every garbage sphere must end in the bin-height band, roughly z 0.13 to 0.25 m.\nBefore finishing, verify: all small items have left the desk bin; all contents are inside the big bin area.",
      "baselineSeed": 100002,
      "revisedSeed": 100001,
      "sameSceneSeed": false,
      "sameBaseInstruction": false,
      "baselineBenchSuccess": false,
      "revisedBenchSuccess": false,
      "comparisonNote": "The baseline and revised records share an episode key but use different resolved base instructions and scene seeds; this is not a matched-scene instruction-only comparison."
    }
  ]
}