🐝 LineBee — Line-Following Robot

EC6090 Robotics & Automation Mini Project — University of Jaffna

An autonomous line-following mobile robot with color-based obstacle avoidance and pick-and-place. Supports two line-following modes:

• PID Mode (default) — traditional PID controller, runs directly on ESP32
• RL Mode (optional) — Q-Learning trained in simulation, exported to ESP32

Both modes can be tested in Pygame simulation before deploying to hardware.

---

🏗 Architecture

┌─────────────────────────────────────────────────────────┐
│                   PC SIMULATION                          │
│                                                          │
│  ┌──────────────────┐    ┌──────────────────────────┐   │
│  │  PID Test         │    │  RL Training              │   │
│  │  (pid_test.py)    │    │  (train.py → visualize)  │   │
│  │  Test ESP32 PID   │    │  Train Q-Learning agent  │   │
│  │  logic in Pygame  │    │  then export Q-table     │   │
│  └──────────────────┘    └──────────────────────────┘   │
└─────────────────────────────────────────────────────────┘
            ↓ tune PID gains              ↓ q_table.h
┌─────────────────────────────────────────────────────────┐
│                   ESP32 FIRMWARE                         │
│  ┌──────────┐  ┌───────────────┐  ┌──────────────┐     │
│  │  Sensors │→ │ PID or RL     │→ │   Motors     │     │
│  │  IR /    │  │ Line Follower │  │   Servo      │     │
│  │  Color   │  │ + State       │  │   Gripper    │     │
│  │          │  │   Machine     │  │              │     │
│  └──────────┘  └───────────────┘  └──────────────┘     │
└─────────────────────────────────────────────────────────┘

---

🤖 Robot Dimensions

        ← ── ── 10cm IR Array ── ── →
  Front  [S1]     [S2]   [S3]    [S4]
          ●        ●      ●       ●
        -4cm    -1.5cm   +1.5cm    +4cm  ← distance from centre (firmware .env / robot.py)

   ← 2.5cm → Color Sensor (gripper base)
   ← 6.0cm → Gripper Tip (max claw gap: 5.5cm)

              ← 7.5cm back → Wheel Axis (13cm between wheels)
              ← 19.0cm back → Caster Wheel

Sensor spacing is configurable — see ⚙️ Configuring IR Sensor Distances below.

---

🚀 Quick Start

Option A: PID Mode — Without RL (Recommended)

PID mode works out of the box. No training needed.

Step 1: Test in Simulation (Optional but Recommended)

cd simulation
pip install -r requirements.txt

# Run PID simulation on the simple linear track (good starting point)
python3 pid_test.py --track linear

# Test on the default loop track
python3 pid_test.py

# Test generalization on different track layouts:
# python3 pid_test.py --track eval_1
# python3 pid_test.py --track eval_2
# python3 pid_test.py --track eval_3

Interactive controls in PID simulation:

Key	Action
SPACE	Pause / Resume
R	Reset robot to start
↑ / ↓	Increase / Decrease Kp
← / →	Decrease / Increase Kd
1 - 9	Set base speed (1=slow, 9=fast)
Q / ESC	Quit

Use the simulation to find good PID values, then update firmware/.env (see below).

Step 2: Flash ESP32

cd firmware
pip install platformio   # if not installed
pio run -t upload
pio device monitor

Step 3: Run the Robot

1. Place robot at START on the arena
2. Press the push button
3. Robot autonomously follows the line using PID

---

Option B: RL Mode — With Q-Learning

RL mode requires training in simulation first.

Step 1: Train the Agent

cd simulation
pip install -r requirements.txt

# Train on the simple linear track (fastest convergence — great for first run)
python3 train.py --track linear

# Train on the default loop track
python3 train.py

# Train on all tracks for maximum generalization:
# python3 train.py --track all
# (includes: default, linear, eval_1, eval_2, eval_3)

# Note: The Q-table is saved to q_tables/ with a timestamp
# (e.g. q_table_20260320_230000.npy) — old models are never overwritten.

Step 2: Watch the Trained Agent

cd simulation

# Automatically loads the MOST RECENTLY trained Q-table:
python3 visualize.py

# Evaluate on specific tracks:
python3 visualize.py --track linear
python3 visualize.py --track eval_1
python3 visualize.py --track eval_2
python3 visualize.py --track eval_3

# Load a SPECIFIC older model:
# python3 visualize.py --model q_table_20260320_230000.npy

Step 3: Export Q-Table to Firmware

cd simulation

# Exports the MOST RECENTLY trained Q-table to firmware/src/q_table.h
python3 export_qtable.py

# Export a SPECIFIC model:
# python3 export_qtable.py --model q_table_20260320_230000.npy

Step 4: Enable RL in Firmware

Use firmware/examples/main-RL.cpp as your main.cpp, or add the RL logic to handleLineFollowing():

// REPLACE THIS (PID mode):
bool lineDetected = line_follower_update();

// WITH THIS (RL mode — discrete actions mapped to config.h speeds):
int lineReadings[4];
sensors_read_line(lineReadings);
uint8_t lineState = sensors_get_line_state(lineReadings);
uint16_t stateIndex = lineState * 6 + (uint8_t)color * 2 + (carryingCube ? 1 : 0);
if (stateIndex < Q_NUM_STATES) {
    uint8_t action = Q_POLICY[stateIndex];
    switch (action) {
      case 0: motors_set(SPEED_FULL, SPEED_FULL); break;       // FWD
      case 1: motors_set(SPEED_SLOW, SPEED_FAST); break;       // SL
      case 2: motors_set(SPEED_FAST, SPEED_SLOW); break;       // SR
      case 3: motors_set(-SPEED_TURN, SPEED_TURN); break;      // HL
      case 4: motors_set(SPEED_TURN, -SPEED_TURN); break;      // HR
      case 5: motors_stop(); break;                            // STOP
      case 6: motors_set(-SPEED_REVERSE, -SPEED_REVERSE); break; // REV
    }
} else {
    motors_stop();
}
bool lineDetected = (lineState != 0);

Also add #include "q_table.h" at the top of main.cpp. The exported q_table.h contains the trained action policy, and you configure the resulting motor speeds manually in firmware/src/config.h using SPEED_FULL, SPEED_FAST, etc.

Step 5: Flash ESP32

cd firmware
pio run -t upload

---

⚙️ Configuration via .env

Sensor geometry, PID gains, and motor ranges for the firmware are controlled via a plain-text .env file. No firmware source code editing is required for these settings.

How it works

firmware/.env   → read by platformio.ini → injected as -D flags → config.h

Firmware: firmware/.env

# ─── IR Sensor Geometry ───────────────────────────────────────
IR_OUTER_DIST_CM=4.0       # S1/S4 distance from centre (cm)
IR_INNER_DIST_CM=2.0       # S2/S3 distance from centre (cm)

# ─── PID Gains ────────────────────────────────────────────────
PID_KP=30.0
PID_KI=0.0
PID_KD=20.0
PID_BASE_SPEED=160

# ─── Line Detection ───────────────────────────────────────────
LINE_THRESHOLD=2000        # ADC threshold (12-bit, 0–4095)

After editing PID/sensor values, rebuild and re-flash:

cd firmware
pio run -t upload

Simulation Config: simulation/robot.py

The PC simulation does not use a .env file. To configure sensor geometries for training or testing, open simulation/robot.py and edit the configuration block at the very top of the file:

IR_OUTER_DIST_CM = 4.0   # S1 (far left) & S4 (far right)
IR_INNER_DIST_CM = 2.0   # S2 (inner left) & S3 (inner right)

After modifying this, running train.py or pid_test.py will instantly reflect the changes. Keep these values identical to your firmware's .env for accurate transfers!

---

🗺 Available Training Tracks

Track	--track flag	Description
Linear	linear	★ Simple straight line — fastest convergence, great for first training run
Default	default	Oval + curves — the primary training loop
Eval 1	eval_1	S-curve layout
Eval 2	eval_2	U-turn layout
Eval 3	eval_3	Complex zig-zag loop
All	all	Interleave all 5 tracks for maximum generalization

Use --track linear first to verify the robot can learn at all, then progress to harder layouts.

---

🔄 Quick Comparison: PID vs RL

	PID Mode	RL Mode
Setup time	Instant — flash and go	~5 min training + export
Simulation test	python3 pid_test.py	python3 train.py then python3 visualize.py
Algorithm	PID controller	Q-Learning lookup table
Tuning	Adjust Kp, Ki, Kd, speed	Adjust learning rate, episodes, rewards
Line following	Smooth, continuous correction	Trained policy table mapping to manual speeds
Adaptability	Fixed logic, works on any track	Learns from specific arena layout
Code change	None (default)	Modify handleLineFollowing() in main.cpp
Best for	Quick deployment, simple tracks	Complex arenas, research projects

---

📁 Project Structure

LineBee/
├── simulation/             # PC simulation environment
│   ├── pid_test.py         # ★ Test PID logic (same as ESP32)
│   ├── train.py            # ★ Train RL agent (supports --track linear/all/...)
│   ├── visualize.py        # Watch trained RL agent
│   ├── export_qtable.py    # Export RL policy → C header
│   ├── arena.py            # Pygame arena with Bézier tracks (5 layouts)
│   ├── robot.py            # Virtual robot — ★ edit IR sensor geometry here
│   ├── sensors.py          # Simulated IR + color sensors
│   ├── rl_agent.py         # Q-learning agent (96 states × 7 discrete actions)
│   ├── requirements.txt    # Python dependencies
│   └── q_tables/           # Timestamped trained models (auto-created)
├── firmware/               # ESP32 code (PlatformIO)
│   ├── .env                # ★ IR sensor distances + PID gains (edit here)
│   ├── platformio.ini      # Reads .env and injects as build flags
│   ├── examples/
│   │   ├── calibrate.cpp     # ★ Auto-calibrate per-sensor thresholds
│   │   ├── ir_test.cpp       # Raw IR sensor value test
│   │   ├── motor_test.cpp    # Motor direction/speed test
│   │   ├── servo_test.cpp    # Gripper open/close test
│   │   ├── color_test.cpp    # Color detection test
│   │   ├── main-pid-linefollower.cpp  # Standalone PID example
│   │   ├── main-normal.cpp   # Basic line follower + obstacle avoidance
│   │   └── main-RL.cpp       # RL mode (Q-table based)
│   └── src/
│       ├── main.cpp          # State machine (PID default / RL optional)
│       ├── config.h          # Pins + constants (reads .env values via #ifndef)
│       ├── thresholds.h      # ★ Per-sensor calibrated thresholds (auto-generated)
│       ├── motors.cpp/h      # L298N motor control
│       ├── sensors.cpp/h     # 4-way IR line sensors (uses thresholds.h)
│       ├── color_sensor.cpp/h # TCS34725 color detection
│       ├── gripper.cpp/h     # MG-995 servo gripper
│       ├── line_follower.cpp/h # PID line following
│       ├── obstacle_avoid.cpp/h # Color-based avoidance
│       └── q_table.h         # Stub (PID) or trained policy (RL)
└── docs/
    └── wiring_guide.md     # Wiring + robot layout diagram

---

🔌 Hardware

Component	Purpose
ESP32 NodeMCU	Microcontroller
L298N Motor Driver	DC motor control
2× DC Gear Motors + Wheels	Drive system (13cm apart)
4-Way IR Line Sensor	Line detection (10cm array, sensor spacing via .env)
TCS34725 Color Sensor	Red/green detection (at gripper base)
MG-995 Servo	Gripper actuator (5.5cm max gap)
Push Button	Start trigger

See docs/wiring_guide.md for complete wiring and robot layout.

---

🧪 Component Testing Guide (Calibration)

Before running the full autonomous code, it is highly recommended to test each hardware component individually using the provided test scripts in firmware/examples/.

How to Run a Test (Arduino IDE)

1. Install ESP32 Boards (if you haven't already):
   • Go to Arduino → Preferences (Mac) or File → Preferences (Windows).
   • In "Additional Boards Manager URLs", paste: https://dl.espressif.com/dl/package_esp32_index.json
   • Go to Tools → Board → Boards Manager, search for "esp32" by Espressif Systems, and click Install.
2. Go to Tools → Board → ESP32 Arduino and select ESP32 Dev Module.
3. Create a New Sketch (File → New).
4. Copy all the contents of the desired test script (e.g., firmware/examples/ir_test.cpp) and paste it into your new Arduino sketch.
5. If testing the servo, go to Sketch → Include Library → Manage Libraries and install ESP32Servo.
6. If testing the color sensor, install Adafruit TCS34725 and Adafruit BusIO from the Library Manager.
7. Connect your ESP32, select your COM Port (Tools → Port), and hit Upload.
8. Open the Serial Monitor and set the baud rate to 115200.

How to Run a Test (PlatformIO / VS Code)

1. Open the LineBee/firmware/ folder in VS Code / PlatformIO.
2. Open src/main.cpp. Select everything and Delete it.
3. Open the desired test script (e.g., examples/ir_test.cpp), copy all its contents, and paste them into src/main.cpp.
4. Run pio run -t upload to flash the ESP32.
5. Watch the results in the Serial Monitor: pio device monitor.
6. When finished testing, remember to restore the original main.cpp (e.g., via git restore src/main.cpp).

1. Sensor Calibration (examples/calibrate.cpp) ⭐ Do This First!

The calibration sketch measures each sensor's response on the line and floor, then computes per-sensor thresholds automatically.

Steps:

1. Copy examples/calibrate.cpp → src/main.cpp
2. Upload: pio run -t upload
3. Open Serial Monitor at 115200 baud
4. Place all sensors on the LINE → press BUTTON (samples for 5 seconds)
5. Move all sensors to the FLOOR → press BUTTON (samples for 5 seconds)
6. Copy the generated thresholds.h output from the Serial Monitor
7. Paste it into firmware/src/thresholds.h (replace entire file)
8. Restore your original src/main.cpp (e.g., git restore src/main.cpp)
9. Rebuild: pio run -t upload

The calibration output:

• Shows real-time sensor readings while you position the robot
• Displays a progress bar during sampling
• Rates each sensor's quality (★ Excellent / ● Good / ◐ OK / ⚠ Poor)
• Provides a live verification mode after calibration
• Supports re-calibration by pressing the button again

Why per-sensor thresholds? Each sensor may have slightly different sensitivity due to manufacturing variation, mounting angle, or distance from the surface. Per-sensor calibration ensures all 4 sensors detect the line accurately.

2. IR Array Test (examples/ir_test.cpp)

• Goal: Verify that all sensors correctly detect line vs floor using calibrated thresholds.
• Process: Open Serial Monitor. Output shows S1:xxxx[L] S2:xxxx[_] ... where L = on line, _ = off line.
• Note: If readings seem wrong, re-run the calibration sketch.

3. Servo Gripper Test (examples/servo_test.cpp)

• Goal: Verify gripper opens wide enough and closes securely.
• Process: The code loops, opening and closing every 2 seconds. Keep hands clear!
• Fix: If the gap is too small or too tight, edit GRIPPER_OPEN and GRIPPER_CLOSE in firmware/src/config.h.

4. Color Sensor Test (examples/color_test.cpp)

• Goal: Reliable Red and Green cube detection.
• Process: Place Red and Green cubes directly under the color sensor (~1-2cm distance). Monitor the Serial output for Detected: RED or Detected: GREEN.
• Fix: If it says Unknown, adjust the RED_* and GREEN_* thresholds in config.h.

5. Motors Test (examples/motor_test.cpp)

• Goal: Ensure both wheels spin to move the robot forward.
• Process: Elevate the robot so the wheels can spin freely. The test script runs both motors FORWARD, stops, then REVERSE.
• Fix: If one wheel spins backwards, swap its two wires on the L298N.

---

⚙️ Tuning Guide

IR Sensor Distances

For firmware, edit firmware/.env. For simulation, edit simulation/robot.py:

# Outer sensors (S1 far-left, S4 far-right) distance from centre
IR_OUTER_DIST_CM=4.0

# Inner sensors (S2 inner-left, S3 inner-right) distance from centre
IR_INNER_DIST_CM=2.0

Rebuild firmware after editing: pio run -t upload

PID Gains (PID Mode)

Tune in simulation first (python3 pid_test.py --track linear), then update firmware/.env:

PID_KP=30.0    # Proportional — ↑/↓ keys in simulation
PID_KI=0.0     # Integral — start at 0
PID_KD=20.0    # Derivative — ←/→ keys in simulation
PID_BASE_SPEED=160  # Base motor speed (0-255)

Quick tuning process:

1. Set PID_KI=0, PID_KD=0. Increase PID_KP until robot follows line but oscillates
2. Increase PID_KD until oscillation stops
3. If robot drifts on straights, add small PID_KI (e.g. 0.5)
4. Adjust PID_BASE_SPEED for desired speed

RL Hyperparameters (RL Mode)

Adjust in simulation/train.py:

NUM_EPISODES = 10000        # More = better policy (but slower)
LEARNING_RATE = 0.5         # How fast Q-values update
DISCOUNT_FACTOR = 0.95      # Future reward importance
EPSILON_DECAY = 0.9985      # Exploration decay rate

RL Motor Speeds (Firmware)

The Q-learning agent outputs discrete actions (0-6). You can manually tune how fast the robot moves for each action by editing the presets in firmware/src/config.h:

#define SPEED_FULL      200   // Action 0: Forward (both wheels)
#define SPEED_FAST      180   // Actions 1,2: Slight turn (faster wheel)
#define SPEED_SLOW      100   // Actions 1,2: Slight turn (slower wheel)
#define SPEED_TURN      160   // Actions 3,4: Hard pivot turns
#define SPEED_REVERSE   150   // Action 6: Reverse
#define SPEED_MEDIUM    140   // Line-loss recovery

Tips:

• If the robot turns too sharply during line following, decrease SPEED_SLOW and increase SPEED_FAST.
• If your motors stall below a certain PWM (e.g., 40), ensure all speed values are high enough.

Sensor Thresholds (Both Modes)

Recommended: Auto-calibration — see Sensor Calibration above.

The calibration sketch generates per-sensor thresholds in firmware/src/thresholds.h:

// Auto-generated by calibrate.cpp
#define SENSOR_1_THRESHOLD  1850   // Far Left   (GPIO 34)
#define SENSOR_2_THRESHOLD  2100   // Inner Left (GPIO 35)
#define SENSOR_3_THRESHOLD  1920   // Inner Right (GPIO 32)
#define SENSOR_4_THRESHOLD  2050   // Far Right  (GPIO 33)

Note: The old single LINE_THRESHOLD in .env is no longer used by the sensor code. All line detection now uses per-sensor thresholds from thresholds.h.

---

📝 License

University of Jaffna — EC6090 Robotics & Automation Course Project.