diff --git a/assets/devices.png b/assets/devices.png deleted file mode 100644 index 6b0210b..0000000 Binary files a/assets/devices.png and /dev/null differ diff --git a/assets/elato-alien.png b/assets/elato-alien.png deleted file mode 100644 index 63dd175..0000000 Binary files a/assets/elato-alien.png and /dev/null differ diff --git a/assets/logos.png b/assets/logos.png deleted file mode 100644 index c69a572..0000000 Binary files a/assets/logos.png and /dev/null differ diff --git a/assets/star-history.png b/assets/star-history.png deleted file mode 100644 index 58db4c0..0000000 Binary files a/assets/star-history.png and /dev/null differ diff --git a/assets/tables.png b/assets/tables.png deleted file mode 100644 index ffe96bc..0000000 Binary files a/assets/tables.png and /dev/null differ diff --git a/assets/usecases.png b/assets/usecases.png deleted file mode 100644 index f3e005f..0000000 Binary files a/assets/usecases.png and /dev/null differ diff --git a/docs/Deploy.md b/docs/Deploy.md deleted file mode 100644 index eeb77a2..0000000 --- a/docs/Deploy.md +++ /dev/null @@ -1,23 +0,0 @@ -# Deploying guide - -If you are ready to launch your next AI venture that helps people in a meaningful way, this is the guide for you. And we're here to support you in that journey. If you get stuck, join our Discord channel where AI enthusiasts from all around the world gather to discuss new ideas and help solve problems together. This is your invite: https://discord.gg/KJWxDPBRUj - -## How to deploy - -1. Elato consists of 4 main parts: - 1. the database hosted on supabase, - 2. the frontend that runs with NextJS, - 3. the server that runs on Deno edge functions and - 4. the firmware the runs on your ESP32 Arduino device. - -2. Setup your Supabase db. Follow the instructions here: [Supabase Setup and Usage Guide](../supabase/README.md) - -3. Setup your NextJS frontend. Follow the instructions here: [NextJS Frontend Setup and Usage Guide](../frontend-nextjs/README.md) - -4. Setup your Deno edge server. Follow the instructions here: [Deno Server Setup and Usage Guide](../server-deno/README.md) - -5. Setup your ESP32 Arduino device. Follow the instructions here: [ESP32 Arduino Device Setup and Usage Guide](../firmware-arduino/README.md) - -6. Putting it all together. Now you should have all the individual parts working together. For a single device, you can do not need to configure any device code settings on Supabase. For multiple devices, refer to the [Multiple Devices](MultipleDevices.md) guide. - -7. For use cases and new ideas, refer to the [Use Cases](Usecases.md) guide. \ No newline at end of file diff --git a/docs/ElevenLabs.md b/docs/ElevenLabs.md deleted file mode 100644 index 59fe872..0000000 --- a/docs/ElevenLabs.md +++ /dev/null @@ -1,50 +0,0 @@ -## Setup - -1. **Environment Variables** - Add your ElevenLabs API key to your `.env` file: - ``` - ELEVENLABS_API_KEY=your_elevenlabs_api_key_here - ``` - -2. **Agent Configuration** - - Create an agent in the Eleven Labs dashboard - - Copy the agent ID - - On the Elato UI, Click `+ Create new` and create an Eleven Labs character with a `title` and the `agentId` - -## How it Works - -1. **Connection Flow**: - - ESP32 connects to your Deno server via WebSocket - - Server authenticates the user and gets their personality configuration - - If provider is "elevenlabs", server requests a signed URL from Eleven Labs API - - Server establishes WebSocket connection to Eleven Labs using the signed URL - - Server acts as a relay between ESP32 and Eleven Labs - -2. **Audio Processing**: - - IMPORTANT: In your Eleven Labs Agent Settings > Voice > TTS Output Format > Set this to PCM 24kHz. - - Currently the AI Agent must speak first. You can change this behaviour in `Audio.cpp`. - - ESP32 sends PCM16 audio data (binary) to server (If you change this, you will also want to change your Eleven Labs Audio input settings) - - Server converts to base64 and forwards to Eleven Labs - - Eleven Labs sends back base64 audio data - - Server converts to PCM16, encodes with Opus, and sends to ESP32 - -3. **Message Types**: - - **From ElevenLabs**: `audio`, `user_transcript`, `agent_response`, `ping`, `conversation_end` - - **To ElevenLabs**: `audio`, `pong`, `interrupt` - - **To ESP32**: Various server messages like `RESPONSE.CREATED`, `RESPONSE.COMPLETE` - -## Usage - -Once configured, the Eleven Labs provider works exactly like OpenAI and Gemini. The server will automatically route to the Eleven Labs implementation when the provider is set to "elevenlabs". - -## Differences from OpenAI/Gemini - -- **No system prompts**: Eleven Labs agents are configured in their dashboard through your account (and api key via a signedUrl) -- **Agent-based**: More advanced with workflow handling and tool calling with MCP through the Eleven Labs dashboard - -## Troubleshooting - -1. **Connection Issues**: Ensure your Eleven Labs API key is valid and has access to Conversational AI -2. **Agent Not Found**: Verify the agent ID is correct and the agent exists in your Eleven Labs account -3. **Audio Issues**: Check that the AI is speaking first. And that TTS Output is 24kHz pcm not 16kHz pcm (which is default) -4. **Transcription Missing**: Ensure your Eleven Labs agent has transcription enabled in the dashboard \ No newline at end of file diff --git a/docs/MultipleDevices.md b/docs/MultipleDevices.md deleted file mode 100644 index ab41980..0000000 --- a/docs/MultipleDevices.md +++ /dev/null @@ -1,15 +0,0 @@ -## 🤖🤖🤖 Getting Started with multiple devices - -1. Register your device by adding your ESP32 Device's MAC Address and a unique user code to the `devices` table in Supabase. -> **Pro Tip:** To find your ESP32-S3 Device's MAC Address, build and upload `test/print_mac_address_test.cpp` using PlatformIO and view the serial monitor. - - -2. Register your user account to this device by adding your unique user code to the [Settings page](http://localhost:3000/home/settings) in the NextJS Frontend. This links your device to your account. - - -3. Set DEV_MODE to `False` in your `frontend-nextjs/.env.local` env variable. - -> **Pro Tip:** If you're testing locally, you can keep enabled the `DEV_MODE` macro in the `Config.h` file of your firmware and the Deno server env variable to use your local IP addresses for testing. - - -4. Now you can register multiple devices to your account by repeating the process above. \ No newline at end of file diff --git a/docs/Usecases.md b/docs/Usecases.md deleted file mode 100644 index f1b60fe..0000000 --- a/docs/Usecases.md +++ /dev/null @@ -1,43 +0,0 @@ -# [WIP] Here are some use cases for the Elato AI device - -Build Custom AI Toys - -*Cute AI plush toys for kids, elderly, and more!* - -## Healthcare ideas: - -1. **Kid-friendly hospital buddies**: Plush toys that help kids feel less scared during hospital stays with stories and games when they need them most. - -2. **Companions for older adults**: Someone to chat with for seniors living alone - they can share stories, play favorite songs, or just be there for a quick hello. - -3. **Mental workout partners**: Fun voice games that keep elderly minds sharp through conversations, trivia, and memory challenges. - -4. **Speech practice friends**: Judgment-free characters that help kids practice their speech therapy exercises without feeling self-conscious. - -5. **Meditation guides**: Calming voices that walk you through relaxation exercises whenever you need a mental break. - -## Learning & play: - -1. **Language conversation partners**: Practice your Spanish, French or any language with a patient friend who never gets tired of helping you get better. - -2. **Story co-creators**: Toys that build stories with your kids, adapting the plot based on what your child says next. - -3. **Educational characters**: Like a dinosaur that actually explains paleontology when asked, or a solar system model that talks about each planet. - -4. **Social skills practice**: A safe space for kids to practice conversations, especially helpful for children on the autism spectrum. - -## Helpful assistants: - -1. **Simple communication hubs**: Easy voice tools for people who struggle with technology to call family with simple commands. - -2. **Museum guides**: Bring exhibits to life by letting visitors chat with historical figures or characters. - -3. **Custom helpers**: Voice systems designed specifically for people with mobility challenges. - -## Creative applications: - -1. **Interactive exhibits**: Imagine talking to Einstein at a science museum or a dinosaur at a natural history exhibit. - -2. **Training scenarios**: Role-play customer service situations or practice difficult conversations. - -3. **Personality for your smart home**: Give your home automation a friendly character that feels more like talking to a helpful roommate than a robot. \ No newline at end of file