AmazingHand

This project is built upon the initial Amazing Hand project.

Amazing Debugging.zip

Code Download

Clone the official open-source code repository https://github.com/pollen-robotics/AmazingHand.git. Please note that there might be errors or omissions in the official open-source code.

git clone https://github.com/pollen-robotics/AmazingHand.git

Environment Installation

Install Rust, uv, dora-rs according to your system.

Install Rust: https://www.rust-lang.org/tools/install

Windows Rust Environment Variable Setup (Important!): Refer to https://zhuanlan.zhihu.com/p/1958936613276087180

Linux Environment Variable Setup:

The first installation may require Visual Studio Installer.

Configure Cargo Mirror Source Create a config.toml configuration file in the .cargo folder and configure the Tsinghua crates.io-index mirror. This will make Cargo use Tsinghua University's mirror source to download crates.

[source.crates-io]
replace-with = 'tuna'

[source.tuna]
registry = "https://mirrors.tuna.tsinghua.edu.cn/git/crates.io-index.git"

Install uv: https://docs.astral.sh/uv/getting-started/installation/

Windows: Open PowerShell terminal, copy and paste the following command to install. Linux Environment Variable Setup:

Install dora-rs: Please refer to https://dora-rs.ai/docs/guides/Installation/installing Linux Environment Variable Setup:

Wiring Method

Power supply requires at least 5V3A. Connect the servo driver board externally via USB to the computer.

Example Demonstration

Check Servo Driver Board Port Number

• Windows system is generally COM11. You can find the servo driver board's port number via Device Manager or the Feite Servo Upper Computer software.

• Ubuntu/Linux system is generally /dev/ttyACM0 Check the servo driver board's port via command line:

ls /dev/ttyUSB* /dev/ttyACM*

sudo chmod 666 /dev/ttyACM*

sudo usermod -aG dialout $USER

If you cannot find the directory with ls /dev/ttyUSB* /dev/ttyACM* in a virtual machine, check if the smart hand is connected to the host computer at the bottom right corner of the virtual machine. If so, choose to disconnect it and connect it to the virtual machine.

Modify the Port Number in the Code ① Find the main.rs code file in the AmazingHand-main\Demo\AHControl\src directory. Open it with a text editor and modify it to the port number found on your host machine (COM* for Windows, /dev/ttyACM* for Ubuntu/Linux systems).

② Find the corresponding example file. For the Right Smart Hand, find dataflow_tracking_real_right.yml in the AmazingHand-main\Demo directory.

For the Left Smart Hand, find dataflow_tracking_real_left.yml in the AmazingHand-main\Demo directory.

For Dual Smart Hands, find dataflow_tracking_real_2hands.yml in the AmazingHand-main\Demo directory.

Open it in text format and modify it to the port number found on your host machine (COM* for Windows, /dev/ttyACM* for Ubuntu/Linux systems).

Code Deployment

• Open the Demo folder. Windows system: Type Powershell in the directory address bar and press Enter to open it. Start the daemon process (required every time): Linux system: Open a terminal directly in the directory. Start the daemon process (required every time):

dora up

• Then run the following command from the terminal in this directory to create a virtual environment (run once during environment setup!! Running again will overwrite the virtual environment!!):

uv venv --python 3.12

• Activate the virtual environment (required every time). Please input and run according to your system:

# Windows
Set-ExecutionPolicy -ExecutionPolicy Bypass -Scope Process

.venv\Scripts\activate

# Linux
source .venv/bin/activate

Ensure the terminal has activated the virtual environment!

• Execute dependency synchronization. Navigate to the AHControl folder.

cd AHControl

cargo build --release

• Then type cd .. and press Enter to return to the Demo directory!. Navigate to the AHSimulation folder.

# Synchronize dependencies in the uv environment
cd AHSimulation

uv sync

• Then type cd .. and press Enter to return to the Demo directory!. Navigate to the HandTracking folder.

cd HandTracking

uv sync

Running Results

• Open the Demo folder! Type Powershell in the directory address bar and press Enter to open it. Start the daemon process (required every time):

dora up

• Activate the virtual environment (required every time). Please input and run according to your system:

Windows commands to activate the virtual environment:

Set-ExecutionPolicy -ExecutionPolicy Bypass -Scope Process

.venv\Scripts\activate

Linux commands to activate the virtual environment:

source .venv/bin/activate

Simulation Environment

• Run the webcam hand tracking demo only in the simulation environment:

dora build dataflow_tracking_simu.yml --uv   # (Execute only once)

dora run dataflow_tracking_simu.yml --uv

Real Hardware Run (Hand Tracking)

• Run the webcam hand tracking demo using real hardware:

Right Smart Hand

dora build dataflow_tracking_real_right.yml --uv   # (Execute only once)

dora run dataflow_tracking_real_right.yml --uv

Left Smart Hand

dora build dataflow_tracking_real_left.yml --uv   # (Execute only once)

dora run dataflow_tracking_real_left.yml --uv

Dual Smart Hands (Note: both connected to one servo driver board)

dora build dataflow_tracking_real_2hands.yml --uv   # (Execute only once)

dora run dataflow_tracking_real_2hands.yml --uv

Simple Example to Control Simulated Finger Angles

• Run a simple example to control finger angles in simulation:

dora build dataflow_angle_simu.yml --uv   # (Execute only once)

dora run dataflow_angle_simu.yml --uv

Description

• AHControl contains a dora-rs node to control the motors and some utility tools for motor configuration.
• AHSimulation contains a dora-rs node to simulate hand movements and obtain inverse kinematics.
• HandTracking contains a dora-rs node to track hands from a webcam and use it as a target for controlling AH!.

Notes

1. Mediapipe Version Issue pyproject.toml is configured with mediapipe>=0.10.14, but the installed mediapipe package lacks the solutions submodule. This is likely due to mediapipe version incompatibility with Python 3.12 (higher versions of mediapipe have issues with Python 3.12 support), or corrupted package files during installation.

uv pip uninstall mediapipe

uv pip install mediapipe==0.10.14

2. Dora Version Incompatibility, Message Format (v0.7.0 vs v0.8.0)

Answer: ① First, delete only the corresponding dependency packages in C:\\Users\\[YourUsername]\\.cargo\\registry\\src\\github.xxxxxxxx\\!

dora-message-0.7.0 (Core! This is the old message format folder, must delete)
dora-core-0.4.1
dora-node-api-0.4.1
dora-arrow-convert-0.4.1
dora-metrics-0.4.1
dora-tracing-0.4.1
const-random-macro-0.1.16 (Auxiliary library Dora depends on, delete along with the old version)

② Open the Demo/AHControl folder and modify Cargo.toml to dora-node-api="0.5.0" dora-message="0.8.0" ③ In the terminal, navigate to the AHControl directory and run cargo build --release again. ④ Then follow the "Real Hardware Run" steps again to re-run build.

Modify the corresponding versions based on the actual error. For example, if dora-message requires 0.6.0, change it to dora-node-api="0.4.0", dora-message="0.6.0".

3. Missing OpenCV Dependency Library

Enter the following command in the HandTracking directory:

python -m pip install opencv-contrib-python numpy mediapipe -i https://mirrors.aliyun.com/pypi/simple/

4. Enable Camera Permissions (Computer)

5. Virtual Machine 22.04 Camera Access Refer to https://blog.csdn.net/qq_19731521/article/details/124954288