7 분 소요

MediaPipe로 만드는 웹캠 손 추적 게임

안녕하세요! 이번 포스트에서는 웹캠을 사용하여 사용자의 손 움직임을 인식하고, 이를 통해 즐길 수 있는 간단한 드래그 앤 드롭 게임을 만드는 방법을 소개합니다. Google의 MediaPipe 라이브러리를 활용하여 별도의 장비 없이도 실시간 손 추적 기능을 구현하고, 순수 JavaScript와 Tailwind CSS로 인터랙티브한 웹 애플리케이션을 구축하는 전체 과정을 다룹니다.

🎮 프로젝트 개요

이 게임의 목표는 간단합니다. 화면에 나타나는 여러 색상의 사각형 중 같은 색을 가진 사각형끼리 손으로 집어 겹치게 만들어 없애는 것입니다.

<마이너리티 리포트 웹게임! 🖐️>

  • 웹캠 앞에서 검지와 중지로 집은 채, 같은 색 블록을 합치면 끝!
  • 총 3라운드로 구성됩니다.
  • 창체 활동이나 쉬는 시간, 누가 더 빨리 3라운드를 통과하는지 간단하게 친구들과 함께 기록 대결 어떨까요?
  • 상대적으로 큰 화면인 PC 환경(웹캠)에서 구동(모바일 X)
  • 프로젝트 페이지 바로가기

🛠️ 주요 기술 스택

  • HTML5: 웹 페이지의 기본 구조를 담당합니다. <video> 태그로 웹캠 영상을 받고, <canvas> 태그에 게임 화면을 그립니다.
  • Tailwind CSS: 유틸리티 우선 CSS 프레임워크로, 빠르고 효율적인 UI 스타일링을 위해 사용했습니다.
  • JavaScript (ES Modules): 게임의 모든 로직과 동적 기능을 구현합니다.
  • MediaPipe Hands: 웹캠 영상에서 실시간으로 손의 랜드마크(관절 위치)를 감지하는 핵심 라이브러리입니다.

💻 전체 코드 (All in one)

아래 코드를 index.html 파일로 저장하고 웹 브라우저에서 열면 바로 게임을 실행해볼 수 있습니다. (카메라 접근 권한 허용 필요)

<!DOCTYPE html>
<html lang="ko">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Hand Tracking Drag & Drop Game</title>
    <script src="[https://cdn.tailwindcss.com](https://cdn.tailwindcss.com)"></script>
    <link rel="preconnect" href="[https://fonts.googleapis.com](https://fonts.googleapis.com)">
    <link rel="preconnect" href="[https://fonts.gstatic.com](https://fonts.gstatic.com)" crossorigin>
    <link href="[https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700&display=swap](https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700&display=swap)" rel="stylesheet">
    <script src="[https://cdn.jsdelivr.net/npm/@mediapipe/camera_utils/camera_utils.js](https://cdn.jsdelivr.net/npm/@mediapipe/camera_utils/camera_utils.js)" crossorigin="anonymous"></script>
    <script src="[https://cdn.jsdelivr.net/npm/@mediapipe/control_utils/control_utils.js](https://cdn.jsdelivr.net/npm/@mediapipe/control_utils/control_utils.js)" crossorigin="anonymous"></script>
    <script src="[https://cdn.jsdelivr.net/npm/@mediapipe/drawing_utils/drawing_utils.js](https://cdn.jsdelivr.net/npm/@mediapipe/drawing_utils/drawing_utils.js)" crossorigin="anonymous"></script>
    <script src="[https://cdn.jsdelivr.net/npm/@mediapipe/hands/hands.js](https://cdn.jsdelivr.net/npm/@mediapipe/hands/hands.js)" crossorigin="anonymous"></script>
    <style>
        body {
            font-family: 'Inter', sans-serif;
        }
        /* 로딩 스피너 스타일 */
        .loader {
            border: 8px solid #f3f3f3;
            border-radius: 50%;
            border-top: 8px solid #3498db;
            width: 60px;
            height: 60px;
            animation: spin 2s linear infinite;
        }
        @keyframes spin {
            0% { transform: rotate(0deg); }
            100% { transform: rotate(360deg); }
        }
        .screen-overlay {
            /* 오버레이 화면 스타일: 더 어둡게 하여 집중도 향상 */
            @apply absolute inset-0 flex flex-col items-center justify-center bg-black bg-opacity-80 z-20 hidden;
        }
    </style>
</head>
<body class="bg-gray-900 text-white flex flex-col items-center justify-center min-h-screen p-4 overflow-hidden">

    <div class="w-full max-w-4xl mx-auto text-center">
        <h1 class="text-3xl md:text-4xl font-bold mb-2">손 추적 드래그 & 드롭 게임</h1>
        <p class="text-gray-400 mb-4">같은 색 사각형끼리 겹쳐서 모두 없애보세요!</p>
        
        <div id="controls" class="my-4">
            <button id="startButton" class="bg-blue-600 hover:bg-blue-700 text-white font-bold py-3 px-6 rounded-lg shadow-lg transition-transform transform hover:scale-105">
                게임 시작
            </button>
        </div>

        <div id="container" class="relative w-full aspect-video bg-gray-800 rounded-lg overflow-hidden shadow-2xl hidden">
            <video id="webcam" class="hidden"></video>
            <canvas id="outputCanvas" class="absolute top-0 left-0 w-full h-full"></canvas>
            
            <div id="loading" class="absolute inset-0 flex flex-col items-center justify-center bg-black bg-opacity-50 z-30">
                <div class="loader"></div>
                <p class="mt-4 text-lg">카메라와 모델을 로딩 중입니다...</p>
            </div>
            <div id="roundStartScreen" class="screen-overlay">
                <h2 id="roundText" class="font-bold text-5xl sm:text-7xl">Round 1</h2>
            </div>
            <div id="roundClearScreen" class="screen-overlay">
                <h2 id="roundClearText" class="font-bold text-green-400 mb-4 text-4xl sm:text-6xl">Round 1 Clear!</h2>
                <p class="text-xl sm:text-3xl">기록: <span id="roundClearTime" class="font-bold">0.00초</span></p>
            </div>
            
            <div id="gameOverFrame" class="absolute top-1/2 left-1/2 -translate-x-1/2 -translate-y-1/2 bg-gray-800 p-8 rounded-xl shadow-2xl border border-gray-600 flex flex-col items-center justify-center z-20 hidden w-11/12 max-w-md">
                <h2 class="font-bold text-yellow-400 mb-4 text-4xl sm:text-5xl text-center">게임 클리어!</h2>
                <p class="text-xl sm:text-2xl mb-2">최종 합산 기록:</p>
                <p id="finalTime" class="font-bold mb-8 text-3xl sm:text-5xl">0.00초</p>
                <button id="restartButton" class="bg-green-600 hover:bg-green-700 text-white font-bold py-3 px-6 rounded-lg shadow-lg transition-transform transform hover:scale-105 w-full">
                    처음부터 다시 시작
                </button>
            </div>
        </div>
    </div>

    <script type="module">
        // DOM 요소
        const videoElement = document.getElementById('webcam');
        const canvasElement = document.getElementById('outputCanvas');
        const canvasCtx = canvasElement.getContext('2d');
        const startButton = document.getElementById('startButton');
        const container = document.getElementById('container');
        const loading = document.getElementById('loading');
        const gameOverFrame = document.getElementById('gameOverFrame');
        const finalTimeElement = document.getElementById('finalTime');
        const restartButton = document.getElementById('restartButton');
        const roundStartScreen = document.getElementById('roundStartScreen');
        const roundText = document.getElementById('roundText');
        const roundClearScreen = document.getElementById('roundClearScreen');
        const roundClearText = document.getElementById('roundClearText');
        const roundClearTime = document.getElementById('roundClearTime');

        // 게임 상태 변수
        let gameActive = false;
        let roundActive = false;
        let roundStartTime = 0;
        let currentRound = 1;
        const totalRounds = 3;
        let roundTimes = [];

        // 사각형 클래스
        class DragRect {
            constructor(posCenter, size, color) {
                this.posCenter = posCenter; this.size = size; this.color = color;
                this.originalColor = color; this.isDragging = false;
            }
            update(cursor) {
                const [cx, cy] = this.posCenter; const [w, h] = this.size;
                const [cursorX, cursorY] = cursor;
                if (cursorX > cx - w / 2 && cursorX < cx + w / 2 && cursorY > cy - h / 2 && cursorY < cy + h / 2) {
                    this.posCenter = cursor; this.isDragging = true; return true;
                }
                return false;
            }
            reset() { this.isDragging = false; }
            draw(ctx) {
                const [cx, cy] = this.posCenter; const [w, h] = this.size;
                const x = cx - w / 2; const y = cy - h / 2;
                ctx.fillStyle = this.isDragging ? 'rgba(0, 255, 0, 0.7)' : this.color;
                ctx.fillRect(x, y, w, h);
                ctx.strokeStyle = 'white'; ctx.lineWidth = 4; const cornerLength = Math.min(20, w/4);
                ctx.beginPath();
                ctx.moveTo(x, y + cornerLength); ctx.lineTo(x, y); ctx.lineTo(x + cornerLength, y);
                ctx.moveTo(x + w - cornerLength, y); ctx.lineTo(x + w, y); ctx.lineTo(x + w, y + cornerLength);
                ctx.moveTo(x + w, y + h - cornerLength); ctx.lineTo(x + w, y + h); ctx.lineTo(x + w - cornerLength, y + h);
                ctx.moveTo(x + cornerLength, y + h); ctx.lineTo(x, y + h); ctx.lineTo(x, y + h - cornerLength);
                ctx.stroke();
            }
        }

        let rectList = [];
        let hands;

        // 충돌 계산 함수
        function checkCollision(rect1, rect2) {
            if (rect1.color !== rect2.color) return false;
            const [cx1, cy1] = rect1.posCenter; const [w1, h1] = rect1.size;
            const x1_left = cx1 - w1 / 2, x1_right = cx1 + w1 / 2;
            const y1_top = cy1 - h1 / 2, y1_bottom = cy1 + h1 / 2;
            const [cx2, cy2] = rect2.posCenter; const [w2, h2] = rect2.size;
            const x2_left = cx2 - w2 / 2, x2_right = cx2 + w2 / 2;
            const y2_top = cy2 - h2 / 2, y2_bottom = cy2 + h2 / 2;
            const overlapX = Math.max(0, Math.min(x1_right, x2_right) - Math.max(x1_left, x2_left));
            const overlapY = Math.max(0, Math.min(y1_bottom, y2_bottom) - Math.max(y1_top, y2_top));
            const overlapArea = overlapX * overlapY;
            const rect1Area = w1 * h1;
            return (overlapArea / rect1Area) >= 0.4;
        }

        // 메인 렌더링 루프
        function onResults(results) {
            if (!canvasElement.width || !canvasElement.height) return;
            canvasCtx.save();
            canvasCtx.clearRect(0, 0, canvasElement.width, canvasElement.height);
            canvasCtx.translate(canvasElement.width, 0); canvasCtx.scale(-1, 1);
            canvasCtx.drawImage(results.image, 0, 0, canvasElement.width, canvasElement.height);
            canvasCtx.restore();

            if (roundActive) {
                let isGrabbing = false; let cursorPosition = null; let draggedRect = null;
                if (results.multiHandLandmarks && results.multiHandLandmarks.length > 0) {
                    const lm = results.multiHandLandmarks[0];
                    const ix = lm[8].x * canvasElement.width, iy = lm[8].y * canvasElement.height; // 검지 끝
                    const mx = lm[12].x * canvasElement.width, my = lm[12].y * canvasElement.height; // 중지 끝
                    const distance = Math.sqrt(Math.pow(ix - mx, 2) + Math.pow(iy - my, 2));
                    if (distance < 65) { // 두 손가락 사이의 거리가 특정 값보다 가까우면 '잡기'로 인식
                        isGrabbing = true; cursorPosition = [canvasElement.width - ix, iy];
                    }
                }
                if (isGrabbing) {
                    for (const rect of rectList) {
                        if (rect.update(cursorPosition)) { draggedRect = rect; break; }
                    }
                } else { rectList.forEach(rect => rect.reset()); }
                if (draggedRect) {
                    const toRemove = new Set();
                    for (const rect of rectList) {
                        if (rect !== draggedRect && checkCollision(draggedRect, rect)) {
                            toRemove.add(draggedRect); toRemove.add(rect);
                        }
                    }
                    if (toRemove.size > 0) { rectList = rectList.filter(rect => !toRemove.has(rect)); }
                }
            }
            rectList.forEach(rect => rect.draw(canvasCtx));

            if (roundActive) {
                const elapsedTime = ((Date.now() - roundStartTime) / 1000).toFixed(2);
                canvasCtx.save();
                canvasCtx.font = "bold 32px Inter";
                canvasCtx.fillStyle = "yellow";
                canvasCtx.textAlign = "right";
                canvasCtx.shadowColor = 'black';
                canvasCtx.shadowBlur = 7;
                canvasCtx.fillText(`Round ${currentRound} | ${elapsedTime}s`, canvasElement.width - 20, 50);
                canvasCtx.restore();

                if (rectList.length === 0) {
                    roundActive = false;
                    gameActive = false; 
                    const finalRoundTime = parseFloat(elapsedTime);
                    roundTimes.push(finalRoundTime);
                    showRoundClearScreen(finalRoundTime);
                }
            }
        }
        
        function showRoundClearScreen(time) {
            roundClearText.textContent = `Round ${currentRound} Clear!`;
            roundClearTime.textContent = `${time.toFixed(2)}초`;
            roundClearScreen.style.display = 'flex';

            setTimeout(() => {
                roundClearScreen.style.display = 'none';
                currentRound++;
                if (currentRound > totalRounds) {
                    endGame();
                } else {
                    startNewRound();
                }
            }, 2000);
        }

        function initializeRects(round) {
            rectList = [];
            const rows = 4; const cols = 2;
            const canvasArea = canvasElement.width * canvasElement.height;
            // 라운드가 올라갈수록 사각형이 작아져 난이도 상승
            const coverage = round === 1 ? 0.6 : (round === 2 ? 0.3 : 0.1);
            const singleRectArea = (canvasArea * coverage) / (rows * cols);
            const rectSize = Math.sqrt(singleRectArea);
            const colors = ['rgba(255, 0, 255, 0.7)', 'rgba(0, 255, 255, 0.7)', 'rgba(255, 255, 0, 0.7)', 'rgba(255, 0, 0, 0.7)'];
            const xMargin = (canvasElement.width - (cols * rectSize)) / (cols + 1);
            const yMargin = (canvasElement.height - (rows * rectSize)) / (rows + 1);
            for (let r = 0; r < rows; r++) {
                for (let c = 0; c < cols; c++) {
                    const x = xMargin * (c + 1) + rectSize * c + rectSize / 2;
                    const y = yMargin * (r + 1) + rectSize * r + rectSize / 2;
                    rectList.push(new DragRect([x, y], [rectSize, rectSize], colors[r]));
                }
            }
        }
        
        function startNewRound() {
            roundText.textContent = `Round ${currentRound}`;
            roundStartScreen.style.display = 'flex';
            setTimeout(() => {
                roundStartScreen.style.display = 'none';
                initializeRects(currentRound);
                roundActive = true;
                gameActive = true; 
                roundStartTime = Date.now();
            }, 2000);
        }
        
        function endGame() {
            gameActive = false; 
            const totalTime = roundTimes.reduce((acc, time) => acc + time, 0);
            finalTimeElement.textContent = `${totalTime.toFixed(2)}초`;
            gameOverFrame.style.display = 'flex';
        }

        function startGame() {
            currentRound = 1;
            roundTimes = [];
            rectList = []; 
            
            gameOverFrame.style.display = 'none';
            roundClearScreen.style.display = 'none';
            roundStartScreen.style.display = 'none';
            
            container.classList.remove('hidden');
            startButton.classList.add('hidden');
            loading.style.display = 'flex';
            
            if (!hands) {
                hands = new Hands({ locateFile: (file) => `https://cdn.jsdelivr.net/npm/@mediapipe/hands/${file}` });
                hands.setOptions({ maxNumHands: 1, modelComplexity: 1, minDetectionConfidence: 0.7, minTrackingConfidence: 0.7 });
                hands.onResults(onResults);
                const camera = new Camera(videoElement, {
                    onFrame: async () => {
                        if (videoElement.readyState >= 3) {
                            if (loading.style.display !== 'none') {
                                loading.style.display = 'none';
                                canvasElement.width = videoElement.videoWidth;
                                canvasElement.height = videoElement.videoHeight;
                                startNewRound();
                            }
                            if(gameActive) await hands.send({ image: videoElement });
                        }
                    }, width: 1280, height: 720
                });
                navigator.mediaDevices.getUserMedia({ video: { facingMode: 'user', width: { ideal: 1280 }, height: { ideal: 720 } } })
                    .then(stream => {
                        videoElement.srcObject = stream;
                        videoElement.onloadedmetadata = () => camera.start();
                    }).catch(err => {
                        console.error("카메라 접근 오류:", err); alert("카메라에 접근할 수 없습니다. 권한을 확인해주세요.");
                    });
            } else {
                loading.style.display = 'none'; 
                startNewRound();
            }
        }

        startButton.addEventListener('click', startGame);
        restartButton.addEventListener('click', startGame);
    </script>
</body>
</html>
표
표