admin-valentin/AIvoices
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

added quickstart details

Browse source
This commit is contained in:
akdeb 2025-04-25 15:48:14 +01:00
parent ebc56bb229
commit 7e7c2c0c4a
6 changed files with 110 additions and 53 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

53
README.md
View file

@ -44,57 +44,66 @@ Control your ESP32 AI device from your phone with the Elato AI webapp.
|:---:|:---:|:---:|
| *Select from a list of AI characters* | *Talk to your AI with real-time responses* | *Create personalized AI characters* |
## Getting Started
## 🚀 Quick Start
1. Install [Supabase CLI](https://supabase.com/docs/guides/local-development/cli/getting-started) and set up your Local Supabase Backend. From the root directory, run:
1. Ensure `DEV_MODE` is set to `True` in your frontend, and server environment variables and enabled in the `Config.h` file of your firmware. This will allow you to skip the device registration process and speed your way to testing the Realtime API AI chat.
2. Install [Supabase CLI](https://supabase.com/docs/guides/local-development/cli/getting-started) and set up your Local Supabase Backend. From the root directory, run:
```bash
brew install supabase/tap/supabase
supabase start # Starts your local Supabase server with the default migrations and seed data.
```
2. Set up your NextJS Frontend. ([See the Frontend README](frontend-nextjs/README.md)) From the `frontend-nextjs` directory, run the following commands. (**Login creds:** Email: admin@elatoai.com, Password: admin)
3. Set up your NextJS Frontend. ([See the Frontend README](frontend-nextjs/README.md)) From the `frontend-nextjs` directory, run the following commands. (**Login creds:** Email: admin@elatoai.com, Password: admin)
```bash
cd frontend-nextjs
npm install
# Set your environment variables
cp .env.example .env.local
# NEXT_PUBLIC_SUPABASE_ANON_KEY=<your_supabase_anon_key>
# OPENAI_API_KEY=<your_openai_api_key>
# In .env.local, set your environment variables
# NEXT_PUBLIC_SUPABASE_ANON_KEY=<your-supabase-anon-key>
# OPENAI_API_KEY=<your-openai-api-key>
# Run the development server
npm run dev
```
3. Add your ESP32-S3 Device MAC Address to the [Settings page](http://localhost:3000/home/settings) in the NextJS Frontend. This links your device to your account.
To find your ESP32-S3 Device's MAC Address, build and upload `test/print_mac_address_test.cpp` using PlatformIO.
4. Start the Deno server. ([See the Deno server README](server-deno/README.md))
```bash
# Navigate to the server directory
cd server-deno
# Set your environment variables
cp .env.example .env
# NEXT_PUBLIC_SUPABASE_ANON_KEY=<your_supabase_anon_key>
# OPENAI_API_KEY=<your_openai_api_key>
# In .env, set your environment variables
# SUPABASE_KEY=<your-supabase-anon-key>
# OPENAI_API_KEY=<your-openai-api-key>
# Run the server at port 8000
deno run -A --env-file=.env main.ts
```
5. Add your OpenAI API Key in the `server-deno/.env` and `frontend-nextjs/.env.local` file.
```
OPENAI_API_KEY=<your_openai_api_key>
```
5. In `Config.cpp` set `ws_server` and `backend_server` to your local IP address. Run `ifconfig` in your console and find `en0` -> `inet` -> `192.168.1.100` (it may be different for your Wifi network). This tells the ESP32 device to connect to your NextJS frontend and Deno server running on your local machine. All services should be on the same Wifi network.
6. Set up your ESP32 Arduino Client. ([See the ESP32 README](firmware-arduino/README.md)) On PlatformIO, first `Build` the project, then `Upload` the project to your ESP32.
6. Build and upload the firmware to your ESP32 device. The ESP32 should open an `ELATO-DEVICE` captive portal to connect to Wifi. Connect to it and go to `http://192.168.4.1` to configure the device wifi.
7. The ESP32 should open an AP `ELATO-DEVICE` to connect to Wifi. Connect to it and go to `http://192.168.4.1` to configure the device wifi.
7. Once your Wifi credentials are configured, turn the device off and on again and it should connect to your Wifi and your server.
8. Once your Wifi is configured, turn the device off and on again and it should connect to your Wifi and the Deno edge server.
8. Now you can talk to your AI Character!
9. Now you can talk to your AI Character!
## 📦 Getting Started with multiple devices
1. Register your device by adding your ESP32 Device's MAC Address and a unique user code to the `devices` table in Supabase.
> **Pro Tip:** To find your ESP32-S3 Device's MAC Address, build and upload `test/print_mac_address_test.cpp` using PlatformIO.
2. Register your user account to this device by adding your unique user code to the [Settings page](http://localhost:3000/home/settings) in the NextJS Frontend. This links your device to your account.
3. Set DEV_MODE to `False` in your frontend and server environment variables.
> **Pro Tip:** If you're testing locally, you can enable the `DEV_MODE` macro in the `Config.h` file of your firmware to use your local IP addresses.
4. Now you can register multiple devices to your account by repeating the process above.
## Project Architecture

7
firmware-arduino/README.md
View file

@ -57,6 +57,13 @@ This firmware turns your ESP32 device into a WebSocket audio client for Elato, e
- In the firmware project, set `DEV_MODE` in Config.cpp
- Update the WebSocket server IP to your local IP address
## NVS Storage
We store the following data in Non-Volatile Storage (NVS) on the ESP32:
1. **Auth token**: The supabase auth token that is used to authenticate the device with the backend server.
2. **Factory reset**: Whether the device has been factory reset.
3. **Wifi credentials**: The wifi credentials of the device.
## First-Time Setup
1. Power on your ESP32

81
frontend-nextjs/app/api/generate_auth_token/route.ts
View file

@ -1,8 +1,9 @@
import { NextResponse } from 'next/server';
import { NextResponse } from "next/server";
import jwt from "jsonwebtoken";
import { createClient } from '@/utils/supabase/server';
import { createClient } from "@/utils/supabase/server";
const ALGORITHM = "HS256";
const isDevMode = process.env.DEV_MODE === "True";
interface TokenPayload {
[key: string]: any;
@ -12,71 +13,107 @@ const createSupabaseToken = (
jwtSecretKey: string,
data: TokenPayload,
// Set expiration to null for no expiration, or use a very large number like 10 years
expireDays: number | null = 3650 // Default to 10 years
expireDays: number | null = 3650, // Default to 10 years
): string => {
const toEncode = {
aud: 'authenticated',
role: 'authenticated',
aud: "authenticated",
role: "authenticated",
sub: data.user_id,
email: data.email,
// Only include exp if expireDays is not null
...(expireDays && {
exp: Math.floor(Date.now() / 1000) + (expireDays * 86400)
exp: Math.floor(Date.now() / 1000) + (expireDays * 86400),
}),
user_metadata: {
...data
}
...data,
},
};
const encodedJwt = jwt.sign(toEncode, jwtSecretKey, {
algorithm: ALGORITHM
algorithm: ALGORITHM,
});
return encodedJwt;
};
const getUserByMacAddress = async (macAddress: string) => {
const supabase = createClient();
const { data, error } = await supabase.from('devices').select('*, user:user_id(*)').eq('mac_address', macAddress).single();
const { data, error } = await supabase.from("devices").select(
"*, user:user_id(*)",
).eq("mac_address", macAddress).single();
if (error) {
throw new Error(error.message);
}
return data.user;
};
const getDevUser = async () => {
const supabase = createClient();
const { data, error } = await supabase.from("users").select("*").eq(
"email",
"admin@elatoai.com",
).single();
if (error) {
throw new Error(error.message);
}
return data;
};
export async function GET(req: Request) {
try {
const { searchParams } = new URL(req.url);
const macAddress = searchParams.get('macAddress');
const macAddress = searchParams.get("macAddress");
if (!macAddress) {
return NextResponse.json(
{ error: 'MAC address is required' },
{ status: 400 }
{ error: "MAC address is required" },
{ status: 400 },
);
}
const user = await getUserByMacAddress(macAddress);
/**
* If `DEV_MODE` is true, we use the dev user.
* Otherwise, we use the user by mac address.
*
* Steps to register your device:
* 1: Register the device `mac_address` and `user_code` in the `devices` tables.
* 2: Make sure the user adds the `user_code` to their account in Settings to link the device to their `user_id`.
* 3: When `DEV_MODE` is false, we then fetch the user by `mac_address`.
*/
let user;
if (isDevMode) {
user = await getDevUser();
} else {
user = await getUserByMacAddress(macAddress);
}
if (!user) {
return NextResponse.json(
{ error: 'User not found' },
{ status: 400 }
{ error: "User not found" },
{ status: 400 },
);
}
const payload = {
email: user.email,
user_id: user.user_id,
created_time: new Date()
created_time: new Date(),
};
const token = createSupabaseToken(process.env.JWT_SECRET_KEY!, payload, null);
const token = createSupabaseToken(
process.env.JWT_SECRET_KEY!,
payload,
null,
);
return NextResponse.json({ token });
} catch (error) {
return NextResponse.json(
{ error: error instanceof Error ? error.message : 'Internal server error' },
{ status: 500 }
{
error: error instanceof Error
? error.message
: "Internal server error",
},
{ status: 500 },
);
}
}
}

11
frontend-nextjs/app/components/Settings/AppSettings.tsx
View file

@ -88,6 +88,8 @@ const AppSettings: React.FC<AppSettingsProps> = ({
});
}
const isDevMode = process.env.DEV_MODE === "True";
return (
<>
<GeneralUserForm
@ -102,8 +104,9 @@ const AppSettings: React.FC<AppSettingsProps> = ({
<h2 className="text-lg font-semibold border-b border-gray-200 pb-2">
Device settings
</h2>
{isDevMode && <div className="flex flex-col text-purple-500 text-xs gap-2">You don't need to register your device because you're in dev mode.</div>}
<div className="flex flex-col gap-6">
{process.env.DEV_MODE != "True" && <div className="flex flex-col gap-2">
<div className="flex flex-col gap-2">
<div className="flex flex-row items-center gap-2">
<Label className="text-sm font-medium text-gray-700">
Register your device
@ -119,7 +122,7 @@ const AppSettings: React.FC<AppSettingsProps> = ({
<div className="flex flex-row items-center gap-2 mt-2">
<Input
value={deviceCode}
disabled={isConnected}
disabled={isConnected || isDevMode}
onChange={(e) => setDeviceCode(e.target.value)}
placeholder={isConnected ? "**********" : "Enter your device code"}
maxLength={100}
@ -127,7 +130,7 @@ const AppSettings: React.FC<AppSettingsProps> = ({
<Button
size="sm"
variant="outline"
disabled={isConnected}
disabled={isConnected || isDevMode}
onClick={async () => {
const result = await connectUserToDevice(selectedUser.user_id, deviceCode);
if (!result) {
@ -145,7 +148,7 @@ const AppSettings: React.FC<AppSettingsProps> = ({
"Enter your device code to register it."
}
</p>
</div>}
</div>
<div className="flex flex-col gap-2 mt-2">
<Label className="text-sm font-medium text-gray-700">
Logged in as

2
server-deno/.env.example
View file

@ -1,6 +1,6 @@
# Supabase keys
SUPABASE_URL=http://127.0.0.1:54321
SUPABASE_KEY=<SUPABASE_KEY>
SUPABASE_KEY=<YOUR-SUPABASE-ANON-KEY>
JWT_SECRET_KEY=super-secret-jwt-token-with-at-least-32-characters-long
# Encryption Key (useful for encrypting secrets in the database)

9
server-deno/main.ts
View file

@ -112,12 +112,13 @@ wss.on("connection", async (ws: WSWebSocket, payload: IPayload) => {
});
}
// send user details to client
// when DEV_MODE is true, we send the default values 100, false, false
ws.send(
JSON.stringify({
type: "auth",
volume_control: user.device?.volume,
is_ota: user.device?.is_ota,
is_reset: user.device?.is_reset,
volume_control: user.device?.volume ?? 100,
is_ota: user.device?.is_ota ?? false,
is_reset: user.device?.is_reset ?? false,
}),
);
@ -160,7 +161,7 @@ wss.on("connection", async (ws: WSWebSocket, payload: IPayload) => {
ws.send(JSON.stringify({
type: "server",
msg: "RESPONSE.COMPLETE",
volume_control: device.volume,
volume_control: device.volume ?? 100,
}));
} else {
// Fall back to just sending the complete message if there's an error