RL Deployment FAQ

Tip

If the robot itself misbehaves (motors in error, controllers do not start, change_controllers times out, e-stop), open the Kangaroo Doctor at http://kangaroo-5c:8001 first. This FAQ covers policy and container problems.

Docker → Real Robot: Setting ROS_IP

If ROS commands from the Docker container hang (e.g. Waiting for Action Server ...), ROS_IP is missing. Without it the robot does not know how to communicate back to the container.

export ROS_IP=$(hostname -I | awk '{print $1}')
echo $ROS_IP  # should be 10.68.0.209

Afterwards all ROS commands from the container work normally.

Complete setup for Docker → robot:

export ROS_MASTER_URI=http://kangaroo-5c:11311
export ROS_IP=$(hostname -I | awk '{print $1}')
source /catkin_ws/devel/setup.bash

rosrun pal_policy_deployer debug_obs.py _model:=v56 _task=loco

/catkin_ws/src/kangaroo_robot/kangaroo_controller_configuration/config/local_joint_control/impedance_control/

On the real robot you need to overwrite the 10 files (5 joints × left+right) in /opt/pal/ferrum/share/...

Changes for 7DOF + Gripper Simulation

Problem

The subscriber_controller failed with:

gains_param.size() = 26 != actual_js_state_.name.size() = 34

And:

file does not exist [.../impedance_control/arm_left_5_joint.yaml]

The configs were designed for 4-DOF arms only; the real robot has 7-DOF + gripper (34 joints).

Fix 1: gains YAML extended

File: /catkin_ws/src/subscriber_controller/config/kangaroo_subscriber_controller.yaml

8 missing joints added: - arm_left_5_joint, arm_left_6_joint, arm_left_7_joint - arm_right_5_joint, arm_right_6_joint, arm_right_7_joint - gripper_left_finger_joint, gripper_right_finger_joint

All with p: 0.0, i: 0.0, d: 0.0, i_clamp: 0.0

Fix 2: Impedance Control YAMLs created

Directory: /catkin_ws/src/kangaroo_robot/kangaroo_controller_configuration/config/local_joint_control/impedance_control/

6 new files created (same values as arm_*_4_joint.yaml): - arm_left_5_joint.yaml - arm_left_6_joint.yaml - arm_left_7_joint.yaml - arm_right_5_joint.yaml - arm_right_6_joint.yaml - arm_right_7_joint.yaml

Content of each file:

stiffness_factor: 26.177
damping_factor: 1.666
feedforward_prev_cmd: true
integral_factor: 0.0

Gripper joints do not need an impedance YAML — they use no_control.launch.

Starting simulation (7DOF + Gripper)

roslaunch kangaroo_pal_physics_simulator kangaroo_pal_physics_simulator.launch \
  with_arms:=true arm_type:=7dof with_pelvis:=true end_effector_type:=gripper

Starting subscriber controller

source /catkin_ws/devel/setup.bash
roslaunch subscriber_controller kangaroo_subscriber_controller.launch simulation:=true

Important: the workspace must be sourced so that rospack find subscriber_controller finds the catkin_ws version (not /opt/pal/ferrum).

On the real robot

The same 6 impedance YAML files must also be copied to /opt/pal/ferrum/share/kangaroo_controller_configuration/config/local_joint_control/impedance_control/.

roslaunch pal_policy_deployer kang_rl_deployer.launch model:=v56 task:=loco lab:=mjlab

Why the Policy Works for Both 4-DOF and 7-DOF Arms

Principle: name-based matching

The policy communicates exclusively via joint names, not via indices or the number of joints.

The node reads joint_names from the ONNX metadata at startup:

config.joint_names = splitString(metadata_map["joint_names"], ',');

In the JointState callback only these names are looked up:

if (msg->name[i] == config.joint_names[k]) {
    term->current[k] = msg->position[i] - config.default_joint_pos[k];
}

The action command also contains only the policy joints:

cmd_msg.name = config.action_joint_names;  // only what the policy knows

What happens to the "unknown" joints

Joints	Trained with 4-DOF policy	Trained with 7-DOF policy
arm_*_1–4	Policy controls them	Policy controls them
arm_*_5–7	Policy ignores them → fallback	Policy controls them
gripper_*	Policy ignores them → fallback	Policy ignores them → fallback

Fallback = subscriber_controller holds the joints at their init position: τ = K*(q_init - q) - D*q̇

This means a policy trained only with 4-DOF arms simply "leaves" joints 5/6/7 hanging — they are held at initial position by the PD controller.

Preparing the Real Robot (7DOF + Gripper)

The real robot loads configs from /opt/pal/ferrum/share/ — not from catkin_ws. The following files must be copied manually:

1. Impedance Control YAMLs (6 new files)

DEST=/opt/pal/ferrum/share/kangaroo_controller_configuration/config/local_joint_control/impedance_control
SRC=/catkin_ws/src/kangaroo_robot/kangaroo_controller_configuration/config/local_joint_control/impedance_control

cp $SRC/arm_left_5_joint.yaml  $DEST/
cp $SRC/arm_left_6_joint.yaml  $DEST/
cp $SRC/arm_left_7_joint.yaml  $DEST/
cp $SRC/arm_right_5_joint.yaml $DEST/
cp $SRC/arm_right_6_joint.yaml $DEST/
cp $SRC/arm_right_7_joint.yaml $DEST/

Content of each file (same as arm_*_4_joint):

stiffness_factor: 26.177
damping_factor: 1.666
feedforward_prev_cmd: true
integral_factor: 0.0

2. Subscriber Controller gains YAML (34 joints)

cp /catkin_ws/src/subscriber_controller/config/kangaroo_subscriber_controller.yaml \
   /opt/pal/ferrum/share/subscriber_controller/config/kangaroo_subscriber_controller.yaml

This file was extended from 26 to 34 joints (arm_5/6/7 + gripper added).

Why these two files?

File	Problem without fix
`impedance_control/arm_*_5/6/7_joint.yaml`	Launch aborts: `file does not exist`
`kangaroo_subscriber_controller.yaml`	Controller fails to load: `gains_param.size() = 26 != 34`

Deploy on real robot

# Source workspace so catkin_ws packages take priority
source /catkin_ws/devel/setup.bash

# Start policy (starts subscriber_controller automatically)
roslaunch pal_policy_deployer kang_rl_deployer.launch model:=v56 task:=loco lab:=mjlab

Note: When roslaunch is started from inside the Docker container (with ROS_MASTER_URI=http://kangaroo-5c:11311) and catkin_ws is sourced, YAMLs are automatically loaded from catkin_ws — no manual copying to the robot needed.

Policy Model: What Must Be in the Folder?

Minimum content

models/loco/loco_v60_mjlab/
└── 2026-04-01_07-08-05.onnx   ← only required file

The node automatically searches for the first .onnx file in the directory:

pkg_root/models/<task>/<task>_<version>_mjlab/

What is inside the ONNX model?

All relevant parameters are embedded as metadata in the model:

Metadata key	Content
`joint_names`	List of all joints the policy knows
`default_joint_pos`	Zero position per joint (= action_offset)
`action_scale`	Scaling factor per action dimension
`observation_names`	Order of observation terms
`joint_stiffness`	K values (optional, informational)
`joint_damping`	D values (optional, informational)

Observation normalization and action scaling

Raw sensor → ONNX model → Raw output → action_scale * output + default_joint_pos → q_des
              ↑
     Normalization internal (mean/std embedded at training export)

Observations (node sends raw, no preprocessing):

Term	Content	Dim
`joint_pos`	`q - default_joint_pos`	n_joints
`joint_vel`	`q̇` measured	n_joints
`base_ang_vel`	IMU angular velocity	3
`projected_gravity`	gravity vector in body frame (from IMU)	3
`command`	`vx, vy, wz` from `/cmd_vel`	3
`actions`	last policy output (raw, without scaling)	n_actions

Normalization: obs_normalization: true in training → mean/std embedded in ONNX, the node does nothing.

Action scaling in node:

action[i] = config.action_scale[i] * net_output[i] + config.action_offset[i];
//                                                      └── default_joint_pos

Clipping: clip_actions: null → no clipping.

Safety: subscriber_controller Without Active Policy

What happens at startup without a policy?

subscriber_controller stores q_init (current position at startup)
Sends q_des = q_init to the motor driver
Motor driver holds position via DirectImpedanceControl: τ = stiffness_factor * (q_init - q) - damping_factor * q̇
The gains in kangaroo_subscriber_controller.yaml are all 0 → no extra PD

feedforward_prev_cmd — motor driver parameter (unverified)

On the real robot all joints are configured with feedforward_prev_cmd: true:

{damping_factor: 6.366, feedforward_prev_cmd: true, integral_factor: 0.0, stiffness_factor: 100.0}

Note: feedforward_prev_cmd is not present in the ROS-side ImpedanceControlParams struct (verified in direct_impedance_control.h). The parameter is passed to the motor driver firmware — what it does with it cannot be verified from source code (closed-source firmware).

The name suggests that the motor driver adds the previous torque command as feedforward (τ_t = K*(q_des-q) - D*q̇ + τ_(t-1)), but this is an assumption based on the parameter name.