Architecture
Laptop to Pi to motors
Hackathon Fall 2026
A Raspberry Pi 5 robot, driven from a laptop, that moves, turns, explores, speaks, and senses.
arpitran/hackathon-fall-2026
The idea
Plan
| # | Milestone | Status |
|---|---|---|
| M0 | Prep: order parts, bring up the Pi 5 | done |
| M1 | Eyes on the Pi: person detection on the board | todo |
| M2 | Hardware bring-up: L298N on the Pi's GPIO, all four motors turn | done |
| M3 | Wheels: Car class + drive CLI | in progress |
| M4 | Laptop control: FastAPI + WebSocket + watchdog | todo |
| M5 | Sentry vision: live stream with person boxes | todo |
| M6 | Stretch: voice (talking loop works on the Mac) or obstacle sensing | in progress |
| M7 | Demo video + docs | todo |
Hardware
Raspberry Pi 5 with active cooler
Pi 5 GPIO pins drive the L298N directly, no co-processor
L298N driver, four BO motors and wheels (skid steering), separate 4×AA battery pack (6V)
Logitech USB webcam
USB speaker (the Pi 5 has no 3.5mm jack)
Doubled cardboard chassis, breadboard for prototyping
Architecture
Wiring
Header pins
Working today
| Pi pin | GPIO | L298N | Side |
|---|---|---|---|
| 11 | GPIO17 | IN1 | left |
| 13 | GPIO27 | IN2 | left |
| 15 | GPIO22 | IN3 | right |
| 16 | GPIO23 | IN4 | right |
| 14 | GND | GND | shared ground |
Verified on the real car on 2026-09-21: all four motors turn, both sides, both directions. The drive code is tested and runs as drive-car.
Electronics
A Pi pin puts out 3.3 V and should supply about 16 mA at most. The resistor decides how much current flows.
Red and yellow LEDs drop about 2 V. Blue, white and green drop about 3 V, so use a smaller resistor (100 Ω) if one looks dim. Drag the slider; press Esc to get the arrow keys back.
Vision
A YOLO11n detector runs on webcam frames with tracking and a live FPS counter. It is the base for the sentry view in M5.
uv run --extra vision detect-people
M6 · Voice
Push-to-talk. Every model runs locally: no cloud, no API key, nothing leaves the machine.
uv run talk-car
Measured on the Mac (M3 Max)
Cold start matters: the first Whisper call took 16.6 s, every call after took 0.4 s. So the loop warms the models up before the first “press Enter”.
A faster voice cut the spoken reply from 8.7 s to 5.1 s.
Local models
| Model | Job | Pi 5 (7.9 GB, CPU only) | Mac (M3 Max) |
|---|---|---|---|
| MiniCPM-V 1B | eyes | 3.9 s, correct | 1.4 s, correct |
| Gemma 4 8B | brain | too big: 9.6 GB model | 0.6 s |
| Qwen 3.5 0.8B | brain | 11 tok/s, chatty toy | not needed |
| Moondream | eyes | wrong: invented flowers | wrong, same answer |
| Whisper small.en | ears | tiny/base installed, not timed yet | 0.4 s |
So the Mac is the brain for now, and the Pi is the body: camera, mic, speaker and motors.
What we learned
| What happened | What we did |
|---|---|
| The official Ollama installer 404’d on the Pi | Unpacked the release by hand and added a systemd service |
| Piper produced no audio | It broke when installed under a very long path; a short path fixed it |
| A webcam capture command hung for 5 minutes | ffmpeg waited for 15 output frames; -update 1 fixes it |
| The mic took 1.5 s to open, so first words were lost | The listening blip now plays once audio is really flowing |
| Vision made up a battery level of 92% | The persona now says it has no sensors and refuses to invent numbers |
Demo
Drive the car from the laptop, watch the person boxes, then talk to it.
Fallback: recorded demo video
Next