shadowcontrol

A real-time vision-based teleoperation system that lets an operator control a humanoid robot through body movements. Inspired by the "shadow boxing" concept from Real Steel, the system tracks your pose via webcam and maps it to robot joint angles in real time with ~130ms latency.

Key Features: MediaPipe BlazePose tracking • Plane projection for joint angle retargeting • Gimbal lock detection • Exponential smoothing & rate limiting • Built in 1 week at CMU Build18 2025

Tech Stack: Python • MediaPipe • OpenCV • Serial Communication • K-Scale Zeroth-01 • Feetech Servos

piano hand

An autonomous robotic hand capable of playing the piano. The system integrates perception (optical music recognition), algorithms (Viterbi-based finger planning), hardware (custom 3D-printed hand with servo actuation), and simulation (ROS-based testing environment).

Key Features: Sheet music scanning and parsing • Optimal finger positioning algorithms • Custom tendon-driven mechanism • Multi-agent coordination • Real-time servo control

Tech Stack: C++ • Python • ROS2 • Arduino/ESP32 • Computer Vision • 3D CAD/Printing

promptmove ai

A natural language-driven embodied AI system enabling people with disabilities to control robots in simulated domestic environments through conversational interfaces. Built on VirtualHome simulation platform with LLM-based task planning for household assistance tasks.

Key Features: Voice & text command interface • GPT-4 powered task understanding • Real-time 3D environment simulation • Multi-agent coordination • Speech transcription • Automated video generation

Tech Stack: Python • OpenAI API • VirtualHome • Streamlit • Speech Recognition