Skip to main content

Luna — Voice AI Avatar

A web app that combines ElevenLabs conversational AI with a real-time animated avatar (Luna), natural lip sync, and a custom Node server. Built with Next.js, React, and Socket.IO.

Live demo
  • next.js
  • node.js
  • socket.io
  • react.js
  • eleven labs

Overview

Single-page web app: user taps "Start Voice Mode", grants microphone access, and talks to Luna. The avatar animates in sync with the AI's voice; the user can mute, end the call, or trigger tools (time, alarm, brightness, etc.) via conversation.

Architecture

Next.js handles the UI and API routes (signed URL, optional tool sync). A custom server.js runs the Next request handler and mounts Socket.IO on the same HTTP server. The front end connects to the socket on the same origin unless overridden by env. One signed URL per conversation; reconnection and visibility logic keep the session and mic state consistent.

Challenges

Keeping the voice session and microphone reliable when the tab was backgrounded; avoiding duplicate audio streams on reconnect; and merging the Socket.IO server into the same process and port as the web app so deployment stays a single service.