MediaPipe로 만드는 웹캠 손 추적 게임 웹앱
MediaPipe로 만드는 웹캠 손 추적 게임
안녕하세요! 이번 포스트에서는 웹캠을 사용하여 사용자의 손 움직임을 인식하고, 이를 통해 즐길 수 있는 간단한 드래그 앤 드롭 게임을 만드는 방법을 소개합니다. Google의 MediaPipe 라이브러리를 활용하여 별도의 장비 없이도 실시간 손 추적 기능을 구현하고, 순수 JavaScript와 Tailwind CSS로 인터랙티브한 웹 애플리케이션을 구축하는 전체 과정을 다룹니다.
🎮 프로젝트 개요
이 게임의 목표는 간단합니다. 화면에 나타나는 여러 색상의 사각형 중 같은 색을 가진 사각형끼리 손으로 집어 겹치게 만들어 없애는 것입니다.
<마이너리티 리포트 웹게임! 🖐️>
- 웹캠 앞에서 검지와 중지로 집은 채, 같은 색 블록을 합치면 끝!
- 총 3라운드로 구성됩니다.
- 창체 활동이나 쉬는 시간, 누가 더 빨리 3라운드를 통과하는지 간단하게 친구들과 함께 기록 대결 어떨까요?
- 상대적으로 큰 화면인 PC 환경(웹캠)에서 구동(모바일 X)
- 프로젝트 페이지 바로가기
🛠️ 주요 기술 스택
- HTML5: 웹 페이지의 기본 구조를 담당합니다.
<video>
태그로 웹캠 영상을 받고,<canvas>
태그에 게임 화면을 그립니다. - Tailwind CSS: 유틸리티 우선 CSS 프레임워크로, 빠르고 효율적인 UI 스타일링을 위해 사용했습니다.
- JavaScript (ES Modules): 게임의 모든 로직과 동적 기능을 구현합니다.
- MediaPipe Hands: 웹캠 영상에서 실시간으로 손의 랜드마크(관절 위치)를 감지하는 핵심 라이브러리입니다.
💻 전체 코드 (All in one)
아래 코드를 index.html
파일로 저장하고 웹 브라우저에서 열면 바로 게임을 실행해볼 수 있습니다. (카메라 접근 권한 허용 필요)
<!DOCTYPE html>
<html lang="ko">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Hand Tracking Drag & Drop Game</title>
<script src="[https://cdn.tailwindcss.com](https://cdn.tailwindcss.com)"></script>
<link rel="preconnect" href="[https://fonts.googleapis.com](https://fonts.googleapis.com)">
<link rel="preconnect" href="[https://fonts.gstatic.com](https://fonts.gstatic.com)" crossorigin>
<link href="[https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700&display=swap](https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700&display=swap)" rel="stylesheet">
<script src="[https://cdn.jsdelivr.net/npm/@mediapipe/camera_utils/camera_utils.js](https://cdn.jsdelivr.net/npm/@mediapipe/camera_utils/camera_utils.js)" crossorigin="anonymous"></script>
<script src="[https://cdn.jsdelivr.net/npm/@mediapipe/control_utils/control_utils.js](https://cdn.jsdelivr.net/npm/@mediapipe/control_utils/control_utils.js)" crossorigin="anonymous"></script>
<script src="[https://cdn.jsdelivr.net/npm/@mediapipe/drawing_utils/drawing_utils.js](https://cdn.jsdelivr.net/npm/@mediapipe/drawing_utils/drawing_utils.js)" crossorigin="anonymous"></script>
<script src="[https://cdn.jsdelivr.net/npm/@mediapipe/hands/hands.js](https://cdn.jsdelivr.net/npm/@mediapipe/hands/hands.js)" crossorigin="anonymous"></script>
<style>
body {
font-family: 'Inter', sans-serif;
}
/* 로딩 스피너 스타일 */
.loader {
border: 8px solid #f3f3f3;
border-radius: 50%;
border-top: 8px solid #3498db;
width: 60px;
height: 60px;
animation: spin 2s linear infinite;
}
@keyframes spin {
0% { transform: rotate(0deg); }
100% { transform: rotate(360deg); }
}
.screen-overlay {
/* 오버레이 화면 스타일: 더 어둡게 하여 집중도 향상 */
@apply absolute inset-0 flex flex-col items-center justify-center bg-black bg-opacity-80 z-20 hidden;
}
</style>
</head>
<body class="bg-gray-900 text-white flex flex-col items-center justify-center min-h-screen p-4 overflow-hidden">
<div class="w-full max-w-4xl mx-auto text-center">
<h1 class="text-3xl md:text-4xl font-bold mb-2">손 추적 드래그 & 드롭 게임</h1>
<p class="text-gray-400 mb-4">같은 색 사각형끼리 겹쳐서 모두 없애보세요!</p>
<div id="controls" class="my-4">
<button id="startButton" class="bg-blue-600 hover:bg-blue-700 text-white font-bold py-3 px-6 rounded-lg shadow-lg transition-transform transform hover:scale-105">
게임 시작
</button>
</div>
<div id="container" class="relative w-full aspect-video bg-gray-800 rounded-lg overflow-hidden shadow-2xl hidden">
<video id="webcam" class="hidden"></video>
<canvas id="outputCanvas" class="absolute top-0 left-0 w-full h-full"></canvas>
<div id="loading" class="absolute inset-0 flex flex-col items-center justify-center bg-black bg-opacity-50 z-30">
<div class="loader"></div>
<p class="mt-4 text-lg">카메라와 모델을 로딩 중입니다...</p>
</div>
<div id="roundStartScreen" class="screen-overlay">
<h2 id="roundText" class="font-bold text-5xl sm:text-7xl">Round 1</h2>
</div>
<div id="roundClearScreen" class="screen-overlay">
<h2 id="roundClearText" class="font-bold text-green-400 mb-4 text-4xl sm:text-6xl">Round 1 Clear!</h2>
<p class="text-xl sm:text-3xl">기록: <span id="roundClearTime" class="font-bold">0.00초</span></p>
</div>
<div id="gameOverFrame" class="absolute top-1/2 left-1/2 -translate-x-1/2 -translate-y-1/2 bg-gray-800 p-8 rounded-xl shadow-2xl border border-gray-600 flex flex-col items-center justify-center z-20 hidden w-11/12 max-w-md">
<h2 class="font-bold text-yellow-400 mb-4 text-4xl sm:text-5xl text-center">게임 클리어!</h2>
<p class="text-xl sm:text-2xl mb-2">최종 합산 기록:</p>
<p id="finalTime" class="font-bold mb-8 text-3xl sm:text-5xl">0.00초</p>
<button id="restartButton" class="bg-green-600 hover:bg-green-700 text-white font-bold py-3 px-6 rounded-lg shadow-lg transition-transform transform hover:scale-105 w-full">
처음부터 다시 시작
</button>
</div>
</div>
</div>
<script type="module">
// DOM 요소
const videoElement = document.getElementById('webcam');
const canvasElement = document.getElementById('outputCanvas');
const canvasCtx = canvasElement.getContext('2d');
const startButton = document.getElementById('startButton');
const container = document.getElementById('container');
const loading = document.getElementById('loading');
const gameOverFrame = document.getElementById('gameOverFrame');
const finalTimeElement = document.getElementById('finalTime');
const restartButton = document.getElementById('restartButton');
const roundStartScreen = document.getElementById('roundStartScreen');
const roundText = document.getElementById('roundText');
const roundClearScreen = document.getElementById('roundClearScreen');
const roundClearText = document.getElementById('roundClearText');
const roundClearTime = document.getElementById('roundClearTime');
// 게임 상태 변수
let gameActive = false;
let roundActive = false;
let roundStartTime = 0;
let currentRound = 1;
const totalRounds = 3;
let roundTimes = [];
// 사각형 클래스
class DragRect {
constructor(posCenter, size, color) {
this.posCenter = posCenter; this.size = size; this.color = color;
this.originalColor = color; this.isDragging = false;
}
update(cursor) {
const [cx, cy] = this.posCenter; const [w, h] = this.size;
const [cursorX, cursorY] = cursor;
if (cursorX > cx - w / 2 && cursorX < cx + w / 2 && cursorY > cy - h / 2 && cursorY < cy + h / 2) {
this.posCenter = cursor; this.isDragging = true; return true;
}
return false;
}
reset() { this.isDragging = false; }
draw(ctx) {
const [cx, cy] = this.posCenter; const [w, h] = this.size;
const x = cx - w / 2; const y = cy - h / 2;
ctx.fillStyle = this.isDragging ? 'rgba(0, 255, 0, 0.7)' : this.color;
ctx.fillRect(x, y, w, h);
ctx.strokeStyle = 'white'; ctx.lineWidth = 4; const cornerLength = Math.min(20, w/4);
ctx.beginPath();
ctx.moveTo(x, y + cornerLength); ctx.lineTo(x, y); ctx.lineTo(x + cornerLength, y);
ctx.moveTo(x + w - cornerLength, y); ctx.lineTo(x + w, y); ctx.lineTo(x + w, y + cornerLength);
ctx.moveTo(x + w, y + h - cornerLength); ctx.lineTo(x + w, y + h); ctx.lineTo(x + w - cornerLength, y + h);
ctx.moveTo(x + cornerLength, y + h); ctx.lineTo(x, y + h); ctx.lineTo(x, y + h - cornerLength);
ctx.stroke();
}
}
let rectList = [];
let hands;
// 충돌 계산 함수
function checkCollision(rect1, rect2) {
if (rect1.color !== rect2.color) return false;
const [cx1, cy1] = rect1.posCenter; const [w1, h1] = rect1.size;
const x1_left = cx1 - w1 / 2, x1_right = cx1 + w1 / 2;
const y1_top = cy1 - h1 / 2, y1_bottom = cy1 + h1 / 2;
const [cx2, cy2] = rect2.posCenter; const [w2, h2] = rect2.size;
const x2_left = cx2 - w2 / 2, x2_right = cx2 + w2 / 2;
const y2_top = cy2 - h2 / 2, y2_bottom = cy2 + h2 / 2;
const overlapX = Math.max(0, Math.min(x1_right, x2_right) - Math.max(x1_left, x2_left));
const overlapY = Math.max(0, Math.min(y1_bottom, y2_bottom) - Math.max(y1_top, y2_top));
const overlapArea = overlapX * overlapY;
const rect1Area = w1 * h1;
return (overlapArea / rect1Area) >= 0.4;
}
// 메인 렌더링 루프
function onResults(results) {
if (!canvasElement.width || !canvasElement.height) return;
canvasCtx.save();
canvasCtx.clearRect(0, 0, canvasElement.width, canvasElement.height);
canvasCtx.translate(canvasElement.width, 0); canvasCtx.scale(-1, 1);
canvasCtx.drawImage(results.image, 0, 0, canvasElement.width, canvasElement.height);
canvasCtx.restore();
if (roundActive) {
let isGrabbing = false; let cursorPosition = null; let draggedRect = null;
if (results.multiHandLandmarks && results.multiHandLandmarks.length > 0) {
const lm = results.multiHandLandmarks[0];
const ix = lm[8].x * canvasElement.width, iy = lm[8].y * canvasElement.height; // 검지 끝
const mx = lm[12].x * canvasElement.width, my = lm[12].y * canvasElement.height; // 중지 끝
const distance = Math.sqrt(Math.pow(ix - mx, 2) + Math.pow(iy - my, 2));
if (distance < 65) { // 두 손가락 사이의 거리가 특정 값보다 가까우면 '잡기'로 인식
isGrabbing = true; cursorPosition = [canvasElement.width - ix, iy];
}
}
if (isGrabbing) {
for (const rect of rectList) {
if (rect.update(cursorPosition)) { draggedRect = rect; break; }
}
} else { rectList.forEach(rect => rect.reset()); }
if (draggedRect) {
const toRemove = new Set();
for (const rect of rectList) {
if (rect !== draggedRect && checkCollision(draggedRect, rect)) {
toRemove.add(draggedRect); toRemove.add(rect);
}
}
if (toRemove.size > 0) { rectList = rectList.filter(rect => !toRemove.has(rect)); }
}
}
rectList.forEach(rect => rect.draw(canvasCtx));
if (roundActive) {
const elapsedTime = ((Date.now() - roundStartTime) / 1000).toFixed(2);
canvasCtx.save();
canvasCtx.font = "bold 32px Inter";
canvasCtx.fillStyle = "yellow";
canvasCtx.textAlign = "right";
canvasCtx.shadowColor = 'black';
canvasCtx.shadowBlur = 7;
canvasCtx.fillText(`Round ${currentRound} | ${elapsedTime}s`, canvasElement.width - 20, 50);
canvasCtx.restore();
if (rectList.length === 0) {
roundActive = false;
gameActive = false;
const finalRoundTime = parseFloat(elapsedTime);
roundTimes.push(finalRoundTime);
showRoundClearScreen(finalRoundTime);
}
}
}
function showRoundClearScreen(time) {
roundClearText.textContent = `Round ${currentRound} Clear!`;
roundClearTime.textContent = `${time.toFixed(2)}초`;
roundClearScreen.style.display = 'flex';
setTimeout(() => {
roundClearScreen.style.display = 'none';
currentRound++;
if (currentRound > totalRounds) {
endGame();
} else {
startNewRound();
}
}, 2000);
}
function initializeRects(round) {
rectList = [];
const rows = 4; const cols = 2;
const canvasArea = canvasElement.width * canvasElement.height;
// 라운드가 올라갈수록 사각형이 작아져 난이도 상승
const coverage = round === 1 ? 0.6 : (round === 2 ? 0.3 : 0.1);
const singleRectArea = (canvasArea * coverage) / (rows * cols);
const rectSize = Math.sqrt(singleRectArea);
const colors = ['rgba(255, 0, 255, 0.7)', 'rgba(0, 255, 255, 0.7)', 'rgba(255, 255, 0, 0.7)', 'rgba(255, 0, 0, 0.7)'];
const xMargin = (canvasElement.width - (cols * rectSize)) / (cols + 1);
const yMargin = (canvasElement.height - (rows * rectSize)) / (rows + 1);
for (let r = 0; r < rows; r++) {
for (let c = 0; c < cols; c++) {
const x = xMargin * (c + 1) + rectSize * c + rectSize / 2;
const y = yMargin * (r + 1) + rectSize * r + rectSize / 2;
rectList.push(new DragRect([x, y], [rectSize, rectSize], colors[r]));
}
}
}
function startNewRound() {
roundText.textContent = `Round ${currentRound}`;
roundStartScreen.style.display = 'flex';
setTimeout(() => {
roundStartScreen.style.display = 'none';
initializeRects(currentRound);
roundActive = true;
gameActive = true;
roundStartTime = Date.now();
}, 2000);
}
function endGame() {
gameActive = false;
const totalTime = roundTimes.reduce((acc, time) => acc + time, 0);
finalTimeElement.textContent = `${totalTime.toFixed(2)}초`;
gameOverFrame.style.display = 'flex';
}
function startGame() {
currentRound = 1;
roundTimes = [];
rectList = [];
gameOverFrame.style.display = 'none';
roundClearScreen.style.display = 'none';
roundStartScreen.style.display = 'none';
container.classList.remove('hidden');
startButton.classList.add('hidden');
loading.style.display = 'flex';
if (!hands) {
hands = new Hands({ locateFile: (file) => `https://cdn.jsdelivr.net/npm/@mediapipe/hands/${file}` });
hands.setOptions({ maxNumHands: 1, modelComplexity: 1, minDetectionConfidence: 0.7, minTrackingConfidence: 0.7 });
hands.onResults(onResults);
const camera = new Camera(videoElement, {
onFrame: async () => {
if (videoElement.readyState >= 3) {
if (loading.style.display !== 'none') {
loading.style.display = 'none';
canvasElement.width = videoElement.videoWidth;
canvasElement.height = videoElement.videoHeight;
startNewRound();
}
if(gameActive) await hands.send({ image: videoElement });
}
}, width: 1280, height: 720
});
navigator.mediaDevices.getUserMedia({ video: { facingMode: 'user', width: { ideal: 1280 }, height: { ideal: 720 } } })
.then(stream => {
videoElement.srcObject = stream;
videoElement.onloadedmetadata = () => camera.start();
}).catch(err => {
console.error("카메라 접근 오류:", err); alert("카메라에 접근할 수 없습니다. 권한을 확인해주세요.");
});
} else {
loading.style.display = 'none';
startNewRound();
}
}
startButton.addEventListener('click', startGame);
restartButton.addEventListener('click', startGame);
</script>
</body>
</html>
![]() |
![]() |