Explore hand configurations, inspect representative poses and recorded interactions, and compare geometric coverage with PCA reconstruction and compact pose banks.
The default representation is link_near_and_distance_joint. It combines joints, palm-object relations, object shape, 5 mm proximity indicators for 26 hand links, and size-normalized distances. Drag to rotate, scroll to zoom, and color points by source, cluster or object. Select a point or star to inspect its representative and available videos. Cluster numbers are specific to each representation.
| Representation | Silhouette ↑ | Restart ARI ↑ | Object NMI | PC1–2 variance |
|---|---|---|---|---|
| joint_only | 0.136 | 0.998 | 0.271 | 38.3% |
| legacy_full_relation | 0.101 | 0.523 | 0.130 | 28.8% |
| tip_relation_joint | 0.073 | 0.444 | 0.481 | 24.9% |
| link_near_joint | 0.075 | 0.698 | 0.391 | 25.9% |
| link_near_and_distance_joint | 0.087 | 0.997 | 0.400 | 27.8% |
Each representative is an observed sample nearest its cluster center. Compare its hand pose with a recorded teacher interaction from the same final cluster; these are different frames and references. The C4 representative has no link within 5 mm of the object, so these groups should not all be interpreted as stable grasp types.
dexcanvas/traj_00016388#segment2
Frequent object IDs: banana × 613 / cylinder3 × 383 / bowl × 254 / Plate_00 × 151
dexycb_lifted/dexycb_02_right_011_banana_20200813_152647_00
Historical lift + tracking: 198 / 2278. Current teacher examples: 1.
dexcanvas/traj_00018497#segment0
Frequent object IDs: cube2 × 531 / cube1 × 472 / Cuboid_02 × 123 / unknown × 105
dexycb_lifted/dexycb_07_right_025_mug_20200928_154547_00
Historical lift + tracking: 129 / 1560. Current teacher examples: 1.
dexycb_lifted/dexycb_02_right_004_sugar_box_20200813_145951_00
Frequent object IDs: unknown × 349 / banana × 57 / Cylinder_00 × 55 / cylinder3 × 53
hrdexdb_24f/toothbrush_holder-3
Historical lift + tracking: 107 / 624. Current teacher examples: 1.
oakink_native/scene_01__A003++seq__5a5d66e7a358b59c9fee__2023-04-21-20-23-45/stage21/part0
Frequent object IDs: unknown × 124 / Cuboid_01 × 107 / O02@0202@00001 × 60 / cylinder7 × 58
dexcanvas/traj_00021628#segment0
Historical lift + tracking: 8 / 644. Current teacher examples: 1.
dexcanvas/traj_00015753#segment2
Frequent object IDs: cube2 × 61 / cube1 × 50 / powerdrill × 47 / cylinder3 × 45
oakink_lifted/oakink_0005_right_A16013_00
Historical lift + tracking: 2 / 374. Current teacher examples: 1.
dexycb_lifted/dexycb_05_right_003_cracker_box_20200908_143617_00
Frequent object IDs: unknown × 698 / Ring_00 × 53 / Cylinder_00 × 4 / cylinder2 × 2
hot3d_v2/aria_P0001-b2bcbe28_right_whiteboard_marker_00
Historical lift + tracking: 123 / 761. Current teacher examples: 1.
Scenes are mapped to final clusters by exact trajectory key. These are saved HOLD frames, not the configurations used for clustering; not every scene has passed strict success filtering.
| Cluster | Samples | Lift success | Lift + pose tracking | Recorded examples |
|---|---|---|---|---|
| C0 | 2278 | 312 / 2278 (13.7%) | 198 / 2278 (8.7%) | 1 |
| C1 | 1560 | 203 / 1560 (13.0%) | 129 / 1560 (8.3%) | 1 |
| C2 | 624 | 236 / 624 (37.8%) | 107 / 624 (17.1%) | 1 |
| C3 | 644 | 81 / 644 (12.6%) | 8 / 644 (1.2%) | 1 |
| C4 | 374 | 9 / 374 (2.4%) | 2 / 374 (0.5%) | 1 |
| C5 | 761 | 324 / 761 (42.6%) | 123 / 761 (16.2%) | 1 |
One example per final cluster, using General teacher model_5200.pt. The wrist follows its reference while the teacher controls the fingers. These color recordings are matched by complete reference identity and trimmed to the original display durations. Their original rollout measurements remain linked separately.
Stable tail HOLD: in at least 95% of the final 30 simulation ticks, the object is lifted ≥2 cm and at least two target-contacting fingertips reach 0.5 N. These are selected examples, not an aggregate success rate or a five-second HOLD evaluation.
E1 indexes 1,029,112 poses, of which 1,015,044 satisfy URDF limits. A trajectory-key split and source-balanced sampling produce 8,867 fitting poses and 5,016 test poses. Sharpa uses 22 joint angles; the human analysis uses 60D hand-local landmarks normalized by hand length. These feature spaces are analyzed separately.
| Population / feature space | Input dimensions | Components for 95% | PC1–3 variance |
|---|---|---|---|
| Sharpa all (22D) | 22 | 13 | 61.2% |
| Sharpa HOI (22D) | 22 | 14 | 59.7% |
| SharpaDex free hand (22D) | 22 | 9 | 81.2% |
| Human normalized keypoints (60D) | 60 | 10 | 76.2% |
PCA-14 retains 96.1% of variance, yet fingertip FK reconstruction error after clipping still averages 10.1 mm, with a P95 of 26.7 mm. High variance retention does not guarantee precise contact geometry.
| PCA dimensions | Variance retained | Joint RMS / rad | Outside limits before clipping | FK mean / mm | FK P95 / mm |
|---|---|---|---|---|---|
| 4 | 69.1% | 0.1958 | 44.0% | 19.38 | 47.34 |
| 6 | 78.5% | 0.1599 | 57.6% | 18.06 | 45.17 |
| 8 | 85.4% | 0.1313 | 60.0% | 15.09 | 36.78 |
| 10 | 90.2% | 0.1070 | 70.0% | 12.48 | 31.22 |
| 14 | 96.1% | 0.0717 | 76.0% | 10.11 | 26.74 |
| 22 | 100.0% | 0.0000 | 0.0% | 0.00 | 0.00 |
All exported bank entries are observed poses within joint limits. KMeans centers are replaced by their nearest training samples; approximate K-Medoids alternates over a candidate pool. Coverage is the mean of source-wise median nearest-pose errors, in radians. Diversity uses pairwise L2 distances after joint-range normalization.
| Method | 6 | 12 | 24 | 48 | 96 | 192 |
|---|---|---|---|---|---|---|
| Random | 0.3237 | 0.2839 | 0.2597 | 0.2344 | 0.2151 | 0.1986 |
| Joint KMeans 22D | 0.2798 | 0.2430 | 0.2263 | 0.2104 | 0.1970 | 0.1846 |
| PCA KMeans 9D | 0.2880 | 0.2517 | 0.2308 | 0.2138 | 0.2012 | 0.1842 |
| PCA KMeans 14D | 0.2804 | 0.2447 | 0.2285 | 0.2095 | 0.1918 | 0.1811 |
| PCA KMeans 22D | 0.2761 | 0.2572 | 0.2264 | 0.2141 | 0.1949 | 0.1838 |
| PCA KMedoids 14D | 0.2871 | 0.2579 | 0.2301 | 0.2191 | 0.2003 | 0.1879 |
| Taxonomy | 0.3013 | 0.2815 | 0.2655 | 0.2483 | 0.2456 | 0.2390 |
| Method | K=24 | K=96 |
|---|---|---|
| Random | 0.2848 | 0.2423 |
| Joint KMeans 22D | 0.2583 | 0.2292 |
| PCA KMeans 14D | 0.2594 | 0.2275 |
| PCA KMedoids 14D | 0.2618 | 0.2277 |
| Taxonomy | 0.2933 | 0.2684 |
Only about 10–25% of sampled poses in some SPIDER source subsets pass the current URDF limit check; the cause remains unresolved. Exported bank entries have zero joint-limit violations, but their PhysX self-collision pass rate has not been established.