Action Map Policy: From High-Dimensional Action Regression to Pixel Classification, A New Paradigm for Robotic Manipulation Learning
1 day ago / Read about 0 minute
Author:小编   

Doctoral student Haojie Huang from Northeastern University proposed a new paradigm for robotic manipulation learning called Action Map Policy (AMP). This paradigm transforms high-dimensional 3D actions into pixel trajectories in image space, converting action prediction into a pixel-level classification problem. It models complex multimodal action distributions using a cross-entropy objective function, enabling the output of a complete action probability distribution through a single forward pass. This approach achieves a unified input-output space and enhances inference speed. AMP employs an encoder-decoder architecture in the shape of X-Net, fusing information from multiple perspectives through a multi-view Transformer. It completes end-to-end training in image space without requiring explicit 3D trajectory supervision, and recovers 3D trajectories through triangulation during inference. Experimental results show that in fine-grained visual perception and multimodal distribution expression experiments, AMP achieved a 100% success rate. In three real-world tasks involving making coffee, toasting bread, and steaming eggs, AMP's success rate was 50% to 70% higher than that of Diffusion Policy and ACT, with a single inference time of approximately 13.80 milliseconds, significantly faster than DiffPo's 93.53 milliseconds. Additionally, it achieved a positional reconstruction error of approximately 1 millimeter and a rotational error of 1.3 degrees, making it more lightweight and efficient than the World Action Model.