Sensors & Peripherals
Meta Quest 3 VR Teleop Specifications
Complete technical reference: UDP packet schema, field types and byte layout, end-to-end latency breakdown, key tunable parameters, and compatible arm specifications.
System Overview
Hardware & Runtime
| Headset | Meta Quest 3 (hand-tracking mode required) |
| Unity version | 2022.3 LTS or later · XR Hands package · OpenXR plugin |
| Host runtime | Python 3.10+ (Linux or macOS; no ROS required) |
| Network requirement | Same LAN subnet; Wi-Fi 6 access point recommended |
| Right hand UDP port | 8888 |
| Left hand UDP port | 8889 (bilateral control only; otherwise unused) |
| Packet rate (sender) | ~50 Hz from Unity (Unity's fixed update loop) |
| Robot command rate | 30 Hz default (configurable via CONTROL_HZ) |
| Primary SDK (Piper) | piper_sdk · python-can · CAN over USB adapter |
UDP Packet Schema
Each packet is a fixed-length 45-byte binary message. All multi-byte fields use little-endian byte order. Unknown bytes appended at the end are ignored by the receiver for forward compatibility.
Binary Packet Layout — 45 bytes total
| Field | Type | Bytes | Description |
|---|---|---|---|
| header | uint8[4] | 4 | Magic bytes 0x52 0x43 0x54 0x50 — ASCII "RCTP". Packets with a wrong magic value are silently dropped. |
| timestamp | float64 | 8 | Unity Time.realtimeSinceStartup in seconds. Used for jitter measurement; not interpreted by the robot controller. |
| pos_x | float32 | 4 | End-effector X position in metres, VR world frame (right-handed, Y-up). Converted to mm robot frame by transform_position(). |
| pos_y | float32 | 4 | End-effector Y position in metres, VR world frame. |
| pos_z | float32 | 4 | End-effector Z position in metres, VR world frame. |
| rot_x, rot_y, rot_z, rot_w | float32 × 4 | 16 | End-effector rotation as a unit quaternion in VR world frame. Converted to Euler (roll/pitch/yaw) by quat_to_euler() before passing to the robot SDK. |
| gripper | float32 | 4 | Gripper openness: 0.0 = fully closed, 1.0 = fully open. Derived from pinch strength via VRGripperController.cs. |
| flags | uint8 | 1 | Bit 0: tracking valid (1 = hand detected). Bit 1: emergency stop requested (1 = operator pressed Menu). Remaining bits reserved. |
Byte offset summary: header 0–3, timestamp 4–11, pos_x 12–15, pos_y 16–19, pos_z 20–23, rot_x 24–27, rot_y 28–31, rot_z 32–35, rot_w 36–39, gripper 40–43, flags 44.
Python struct format string for unpacking:
import struct PACKET_MAGIC = b'\x52\x43\x54\x50' # "RCTP" PACKET_FMT = '<4sdfffffffff f B' # ^ ^ ^^^^^^^^^^ ^ ^ # | | pos xyz | flags (uint8) # | timestamp gripper (float32) # header (4 bytes) rot xyzw (float32×4) def parse_packet(data: bytes) -> dict | None: if len(data) < struct.calcsize(PACKET_FMT): return None fields = struct.unpack_from(PACKET_FMT, data) header, ts, px, py, pz, rx, ry, rz, rw, gripper, flags = fields if header != PACKET_MAGIC: return None return { "timestamp": ts, "position": (px, py, pz), "rotation": (rx, ry, rz, rw), "gripper": gripper, "valid": bool(flags & 0x01), "estop": bool(flags & 0x02), }
End-to-End Latency
Measured on a Wi-Fi 6 LAN with the AgileX Piper arm at 30 Hz control rate and 25% speed limit. Values are typical; actual latency depends on Wi-Fi conditions and robot servo rate.
~20 ms
Quest 3 Tracking Pipeline
Hand pose sensor fusion to Unity callback
<5 ms
UDP Transit
Wi-Fi 6 LAN; up to ~15 ms on older Wi-Fi
<2 ms
Python Parse & Queue
Struct unpack + queue insert
30–80 ms
Robot Trajectory Execution
Depends on speed limit and move distance
Typical Total End-to-End
| Wi-Fi 6 LAN, 25% speed | 50–120 ms |
| Wi-Fi 5 LAN, 25% speed | 80–150 ms |
| Software e-stop response (Bit 1 path) | 1 control cycle (~33 ms at 30 Hz) |
| Tracking-loss detection | Immediate (flags byte in current packet) |
Keep speed limits conservative to reduce latency feel.
The dominant perceptual latency component is robot trajectory execution. A lower
SPEED_PERCENT makes the system feel more predictable even though actual round-trip time is similar. Start at 25% and raise only after motion is fully validated.
Key Tunable Parameters
Unity Inspector (VRHandPoseSender)
| positionOffset (m) | (0, 0, 0.3) — Piper starting point |
| rotationOffset (deg) | (0, 90, 0) — Piper starting point |
| scaleFactor | 0.75 — scales hand motion to Piper workspace |
| Workspace X clamp (mm) | ±400 (Piper); ±600 (xArm6) |
| Workspace Z clamp (mm) | 50 – 700 (Piper); 0 – 900 (xArm6) |
| UDP target port | 8888 (right hand) / 8889 (left hand) |
Python Server (teleoperation_main.py)
| CONTROL_HZ | 30 Hz — robot command rate; match to servo rate |
| QUEUE_MAXSIZE | 3 — frames buffered between receiver and controller threads |
piper_controller.py (AgileX Piper)
| SPEED_PERCENT | 25% — maximum joint speed fraction; raise gradually |
| X_MIN / X_MAX (mm) | −400 / +400 |
| Y_MIN / Y_MAX (mm) | −400 / +400 |
| Z_MIN / Z_MAX (mm) | 50 / 700 |
| GRIPPER_MAX_UM (µm) | 70 000 — 70 mm physical opening |
| SDK position units | Micrometres (integer) — multiply mm values by 1 000 |
| SDK orientation units | Millidegrees (integer) — multiply degree values by 1 000 |
| CAN interface | can0 (default) — activate with can_activate.sh before running |
Compatible Arms at SVRC
| Arm | DOF | SDK / Interface | Controller File | Status |
|---|---|---|---|---|
| AgileX Piper | 6 | piper_sdk · CAN over USB | piper_controller.py | Production-ready |
| OpenArm | 7 | SocketCAN / ROS 2 MoveIt2 | openarm_controller.py | Beta |
| DK1 Bimanual Kit | Dual 6 | Ports 8888 + 8889 simultaneously | dk1_controller.py | Beta |
| VLAI L1 | Dual 6 | ROS 2 bridge · TCP/IP | vlai_l1_controller.py | In development |
Software Requirements
Quest 3 / Unity Side
| Unity | 2022.3 LTS or later |
| XR Hands package | com.unity.xr.hands ≥ 1.3 |
| OpenXR plugin | com.unity.xr.openxr ≥ 1.9 |
| Meta XR SDK | Optional but recommended for passthrough |
| Hand tracking mode | Enabled in Quest system settings (Movement Tracking) |
Control PC / Python Side
| Python | 3.10 or later (uses union type syntax) |
| piper_sdk | pip install piper_sdk |
| python-can | pip install python-can |
| CAN interface | USB-to-CAN adapter; activated with can_activate.sh can0 1000000 |
| Standard library only | socket, struct, queue, threading, signal, time |