Initial PWA assistant proof of concept

This commit is contained in:
2026-04-26 21:50:39 +02:00
commit fb8cc4d086
12 changed files with 901 additions and 0 deletions

4
.env.example Normal file
View File

@@ -0,0 +1,4 @@
GROQ_API_KEY=your_groq_api_key
GMAIL_EMAIL=opensquaredgeneva@gmail.com
GMAIL_PASSWORD=your_gmail_app_password
APP_SECRET=change_me_to_a_long_random_string

10
.gitignore vendored Normal file
View File

@@ -0,0 +1,10 @@
.env
.venv/
venv/
__pycache__/
*.py[cod]
*.log
*.sqlite
*.db
.DS_Store
Thumbs.db

125
README.md Normal file
View File

@@ -0,0 +1,125 @@
# OpenSquared Assistant PWA
POC de PWA mobile pour parler a l'assistant depuis un telephone.
## Fonctionnalites
- Interface mobile installable sur Android via "Ajouter a l'ecran d'accueil".
- Bouton micro avec transcription Groq Whisper.
- Reponse LLM Groq.
- Lecture audio automatique de la reponse apres interaction utilisateur.
- Envoi d'emails Gmail avec carnet de contacts `contacts.json`.
## Configuration locale
```powershell
cd pwa_assistant
copy .env.example .env
```
Remplir `.env` :
```text
GROQ_API_KEY=...
GMAIL_EMAIL=...
GMAIL_PASSWORD=...
APP_SECRET=...
```
Installer :
```powershell
C:\Users\baron\AppData\Local\Programs\Python\Python313\python.exe -m pip install -r requirements.txt
```
Lancer :
```powershell
C:\Users\baron\AppData\Local\Programs\Python\Python313\python.exe app.py
```
Puis ouvrir :
```text
http://localhost:8080
```
Sur telephone, le micro necessite HTTPS sauf cas particulier localhost.
## Deploiement VPS Hostinger
Exemple Ubuntu :
```bash
cd /var/www
git clone <ton-repo-ou-upload> opensquared-assistant
cd opensquared-assistant/pwa_assistant
python3 -m venv .venv
. .venv/bin/activate
pip install -r requirements.txt
cp .env.example .env
nano .env
```
Service systemd :
```ini
[Unit]
Description=OpenSquared Assistant PWA
After=network.target
[Service]
WorkingDirectory=/var/www/opensquared-assistant/pwa_assistant
EnvironmentFile=/var/www/opensquared-assistant/pwa_assistant/.env
ExecStart=/var/www/opensquared-assistant/pwa_assistant/.venv/bin/gunicorn -w 2 -b 127.0.0.1:8080 app:app
Restart=always
RestartSec=3
[Install]
WantedBy=multi-user.target
```
Commandes :
```bash
sudo nano /etc/systemd/system/opensquared-assistant.service
sudo systemctl daemon-reload
sudo systemctl enable --now opensquared-assistant
sudo systemctl status opensquared-assistant
```
Nginx :
```nginx
server {
server_name assistant.ton-domaine.com;
location / {
proxy_pass http://127.0.0.1:8080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
```
HTTPS avec Certbot :
```bash
sudo certbot --nginx -d assistant.ton-domaine.com
```
## Installation Android
1. Ouvrir l'URL HTTPS dans Chrome Android.
2. Menu `...`.
3. `Ajouter a l'ecran d'accueil` ou `Installer l'application`.
4. Ouvrir l'icone `Assistant`.
5. Appuyer une fois sur `Demarrer` pour autoriser micro/audio.
## Notes
- Groq Orpheus TTS doit etre accepte dans la console Groq pour fonctionner :
https://console.groq.com/playground?model=canopylabs%2Forpheus-v1-english
- La lecture audio automatique depend toujours des regles du navigateur mobile, mais elle marche mieux dans une PWA apres un tap utilisateur que dans Telegram.

237
app.py Normal file
View File

@@ -0,0 +1,237 @@
import base64
import json
import os
import smtplib
import tempfile
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
from pathlib import Path
from dotenv import load_dotenv
from flask import Flask, jsonify, request, send_from_directory
from groq import Groq
load_dotenv()
ROOT = Path(__file__).resolve().parent
STATIC_DIR = ROOT / "static"
CONTACTS_FILE = ROOT / "contacts.json"
GROQ_API_KEY = os.getenv("GROQ_API_KEY", "")
GMAIL_EMAIL = os.getenv("GMAIL_EMAIL", "")
GMAIL_PASSWORD = os.getenv("GMAIL_PASSWORD", "")
CHAT_MODEL = "llama-3.3-70b-versatile"
STT_MODEL = "whisper-large-v3-turbo"
TTS_MODEL = "canopylabs/orpheus-v1-english"
TTS_VOICE = "troy"
app = Flask(__name__, static_folder=str(STATIC_DIR), static_url_path="")
app.secret_key = os.getenv("APP_SECRET", "dev-secret-change-me")
client = Groq(api_key=GROQ_API_KEY)
histories = {}
SYSTEM_PROMPT = """Tu es l'assistant personnel vocal de Laurent.
Tu peux discuter normalement en francais et aider a envoyer des emails via Gmail.
Quand l'utilisateur veut envoyer un email, collecte les informations manquantes :
destinataire, sujet, contenu.
Quand tu as toutes les informations pour envoyer un email, reponds uniquement avec ce JSON :
{"action": "envoyer_email", "destinataire": "...", "sujet": "...", "corps": "..."}
Pour tout le reste, reponds en francais naturel, avec des phrases courtes.
En mode voiture, sois bref : une ou deux phrases maximum."""
def charger_contacts():
try:
return json.loads(CONTACTS_FILE.read_text(encoding="utf-8"))
except FileNotFoundError:
return {"laurent": "barontil@hotmail.com"}
def construire_system_prompt():
contacts = charger_contacts()
lignes = [f"- {nom.title()} : {email}" for nom, email in contacts.items()]
return SYSTEM_PROMPT + "\n\nContacts connus :\n" + "\n".join(lignes)
def resoudre_contact(destinataire):
contacts = charger_contacts()
valeur = destinataire.strip()
return contacts.get(valeur.lower(), valeur)
def extraire_action_email(reponse):
decodeur = json.JSONDecoder()
for index, caractere in enumerate(reponse):
if caractere != "{":
continue
try:
data, _ = decodeur.raw_decode(reponse[index:])
except json.JSONDecodeError:
continue
if isinstance(data, dict) and data.get("action") == "envoyer_email":
return data
return None
def envoyer_email(destinataire, sujet, corps):
destinataire = resoudre_contact(destinataire)
msg = MIMEMultipart()
msg["From"] = GMAIL_EMAIL
msg["To"] = destinataire
msg["Subject"] = sujet
msg.attach(MIMEText(corps, "plain", "utf-8"))
with smtplib.SMTP("smtp.gmail.com", 587) as serveur:
serveur.starttls()
serveur.login(GMAIL_EMAIL, GMAIL_PASSWORD)
serveur.sendmail(GMAIL_EMAIL, destinataire, msg.as_string())
return destinataire
def demander_au_llm(session_id, texte):
historique = histories.setdefault(session_id, [])
historique.append({"role": "user", "content": texte})
del historique[:-20]
reponse = client.chat.completions.create(
model=CHAT_MODEL,
messages=[{"role": "system", "content": construire_system_prompt()}] + historique,
temperature=0.45,
max_tokens=900,
)
contenu = reponse.choices[0].message.content
historique.append({"role": "assistant", "content": contenu})
return contenu
def transcrire_audio(fichier):
suffix = Path(fichier.filename or "voice.webm").suffix or ".webm"
with tempfile.NamedTemporaryFile(delete=False, suffix=suffix) as tmp:
fichier.save(tmp)
chemin = tmp.name
try:
with open(chemin, "rb") as audio:
transcription = client.audio.transcriptions.create(
model=STT_MODEL,
file=audio,
language="fr",
response_format="text",
)
return transcription if isinstance(transcription, str) else transcription.text
finally:
try:
os.remove(chemin)
except OSError:
pass
def generer_audio(texte):
texte = texte.strip()
if len(texte) > 900:
texte = texte[:897] + "..."
reponse_audio = client.audio.speech.create(
model=TTS_MODEL,
voice=TTS_VOICE,
input=texte,
response_format="wav",
speed=1.0,
)
with tempfile.NamedTemporaryFile(delete=False, suffix=".wav") as tmp:
chemin = tmp.name
try:
reponse_audio.write_to_file(chemin)
audio = Path(chemin).read_bytes()
return "data:audio/wav;base64," + base64.b64encode(audio).decode("ascii")
finally:
try:
os.remove(chemin)
except OSError:
pass
def traiter_message(session_id, texte, avec_audio=False):
reponse = demander_au_llm(session_id, texte)
action = extraire_action_email(reponse)
email_sent = False
if action:
destinataire = envoyer_email(action["destinataire"], action["sujet"], action["corps"])
reponse = f"Email envoye a {destinataire}."
email_sent = True
audio_url = None
if avec_audio:
audio_url = generer_audio(reponse)
return {
"reply": reponse,
"audioUrl": audio_url,
"emailSent": email_sent,
}
@app.get("/")
def index():
return send_from_directory(STATIC_DIR, "index.html")
@app.get("/health")
def health():
return jsonify({"ok": True})
@app.post("/api/chat")
def chat():
data = request.get_json(force=True)
texte = (data.get("text") or "").strip()
session_id = data.get("sessionId") or "default"
avec_audio = bool(data.get("withAudio"))
if not texte:
return jsonify({"error": "Message vide"}), 400
try:
return jsonify(traiter_message(session_id, texte, avec_audio))
except Exception as e:
print(f"Erreur /api/chat : {e}", flush=True)
return jsonify({"error": str(e)}), 500
@app.post("/api/voice")
def voice():
fichier = request.files.get("audio")
session_id = request.form.get("sessionId") or "default"
if not fichier:
return jsonify({"error": "Audio manquant"}), 400
try:
texte = transcrire_audio(fichier).strip()
resultat = traiter_message(session_id, texte, avec_audio=True)
resultat["transcript"] = texte
return jsonify(resultat)
except Exception as e:
print(f"Erreur /api/voice : {e}", flush=True)
return jsonify({"error": str(e)}), 500
@app.post("/api/reset")
def reset():
data = request.get_json(silent=True) or {}
session_id = data.get("sessionId") or "default"
histories.pop(session_id, None)
return jsonify({"ok": True})
if __name__ == "__main__":
app.run(host="0.0.0.0", port=int(os.getenv("PORT", "8080")), debug=True)

3
contacts.json Normal file
View File

@@ -0,0 +1,3 @@
{
"laurent": "barontil@hotmail.com"
}

4
requirements.txt Normal file
View File

@@ -0,0 +1,4 @@
Flask==3.0.3
groq==1.2.0
gunicorn==22.0.0
python-dotenv==1.0.1

180
static/app.js Normal file
View File

@@ -0,0 +1,180 @@
const conversation = document.querySelector("#conversation");
const statusDot = document.querySelector("#statusDot");
const statusText = document.querySelector("#statusText");
const textInput = document.querySelector("#textInput");
const sendButton = document.querySelector("#sendButton");
const voiceButton = document.querySelector("#voiceButton");
const voiceLabel = document.querySelector("#voiceLabel");
const resetButton = document.querySelector("#resetButton");
const audioToggle = document.querySelector("#audioToggle");
const sessionId = getSessionId();
let recorder = null;
let chunks = [];
let isRecording = false;
let audioUnlocked = false;
if ("serviceWorker" in navigator) {
navigator.serviceWorker.register("/sw.js").catch(() => {});
}
sendButton.addEventListener("click", sendText);
voiceButton.addEventListener("click", toggleVoice);
resetButton.addEventListener("click", resetConversation);
textInput.addEventListener("keydown", (event) => {
if (event.key === "Enter" && !event.shiftKey) {
event.preventDefault();
sendText();
}
});
function getSessionId() {
const key = "opensquared_assistant_session";
let value = localStorage.getItem(key);
if (!value) {
value = crypto.randomUUID();
localStorage.setItem(key, value);
}
return value;
}
function setStatus(text, mode = "idle") {
statusText.textContent = text;
statusDot.classList.toggle("live", mode === "live");
statusDot.classList.toggle("error", mode === "error");
}
function addBubble(text, role = "assistant") {
const bubble = document.createElement("article");
bubble.className = `bubble ${role}`;
bubble.textContent = text;
conversation.appendChild(bubble);
conversation.scrollTop = conversation.scrollHeight;
return bubble;
}
async function unlockAudio() {
if (audioUnlocked) return;
const audio = new Audio();
audio.muted = true;
try {
await audio.play();
} catch {
// Mobile browsers only need the attempt to happen inside a tap handler.
}
audioUnlocked = true;
}
async function sendText() {
const text = textInput.value.trim();
if (!text) return;
textInput.value = "";
addBubble(text, "user");
setStatus("Analyse en cours", "live");
try {
const response = await fetch("/api/chat", {
method: "POST",
headers: {"Content-Type": "application/json"},
body: JSON.stringify({
sessionId,
text,
withAudio: audioToggle.checked,
}),
});
const data = await response.json();
if (!response.ok) throw new Error(data.error || "Erreur serveur");
await handleAssistantResponse(data);
} catch (error) {
addBubble(error.message, "error");
setStatus("Erreur", "error");
}
}
async function toggleVoice() {
await unlockAudio();
if (isRecording) {
recorder.stop();
return;
}
try {
const stream = await navigator.mediaDevices.getUserMedia({audio: true});
const mimeType = pickMimeType();
recorder = new MediaRecorder(stream, mimeType ? {mimeType} : undefined);
chunks = [];
recorder.ondataavailable = (event) => {
if (event.data.size) chunks.push(event.data);
};
recorder.onstop = async () => {
stream.getTracks().forEach((track) => track.stop());
isRecording = false;
voiceButton.classList.remove("recording");
voiceLabel.textContent = "Demarrer";
await sendVoiceBlob(new Blob(chunks, {type: recorder.mimeType || "audio/webm"}));
};
recorder.start();
isRecording = true;
voiceButton.classList.add("recording");
voiceLabel.textContent = "Stop";
setStatus("Je t'ecoute", "live");
} catch (error) {
addBubble("Micro indisponible : " + error.message, "error");
setStatus("Micro indisponible", "error");
}
}
function pickMimeType() {
const types = ["audio/webm;codecs=opus", "audio/webm", "audio/ogg;codecs=opus"];
return types.find((type) => MediaRecorder.isTypeSupported(type)) || "";
}
async function sendVoiceBlob(blob) {
addBubble("Message vocal envoye", "user");
setStatus("Transcription", "live");
const form = new FormData();
const extension = blob.type.includes("ogg") ? "ogg" : "webm";
form.append("sessionId", sessionId);
form.append("audio", blob, `voice.${extension}`);
try {
const response = await fetch("/api/voice", {method: "POST", body: form});
const data = await response.json();
if (!response.ok) throw new Error(data.error || "Erreur serveur");
if (data.transcript) addBubble(data.transcript, "user");
await handleAssistantResponse(data);
} catch (error) {
addBubble(error.message, "error");
setStatus("Erreur", "error");
}
}
async function handleAssistantResponse(data) {
addBubble(data.reply || "", "assistant");
setStatus(data.emailSent ? "Email envoye" : "Pret", data.emailSent ? "live" : "idle");
if (audioToggle.checked && data.audioUrl) {
try {
const audio = new Audio(data.audioUrl);
await audio.play();
} catch {
addBubble("Audio pret, mais le navigateur a bloque la lecture automatique.", "error");
}
}
}
async function resetConversation() {
await fetch("/api/reset", {
method: "POST",
headers: {"Content-Type": "application/json"},
body: JSON.stringify({sessionId}),
}).catch(() => {});
conversation.innerHTML = "";
addBubble("Nouvelle conversation. Je t'ecoute.", "assistant");
setStatus("Conversation reinitialisee", "live");
}

8
static/icons/icon.svg Normal file
View File

@@ -0,0 +1,8 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512">
<rect width="512" height="512" rx="112" fill="#111827"/>
<circle cx="256" cy="236" r="132" fill="#22c55e"/>
<path d="M219 166c0-18 15-33 33-33h8c18 0 33 15 33 33v101c0 18-15 33-33 33h-8c-18 0-33-15-33-33V166Z" fill="#f8fafc"/>
<path d="M173 234c0 46 37 83 83 83s83-37 83-83" fill="none" stroke="#f8fafc" stroke-width="28" stroke-linecap="round"/>
<path d="M256 317v56M210 373h92" stroke="#f8fafc" stroke-width="28" stroke-linecap="round"/>
<path d="M356 144l31-31M389 203h44M356 262l31 31" stroke="#38bdf8" stroke-width="24" stroke-linecap="round"/>
</svg>

After

Width:  |  Height:  |  Size: 632 B

55
static/index.html Normal file
View File

@@ -0,0 +1,55 @@
<!doctype html>
<html lang="fr">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover">
<meta name="theme-color" content="#111827">
<meta name="apple-mobile-web-app-capable" content="yes">
<meta name="apple-mobile-web-app-title" content="Assistant">
<title>OpenSquared Assistant</title>
<link rel="manifest" href="/manifest.webmanifest">
<link rel="icon" href="/icons/icon.svg" type="image/svg+xml">
<link rel="stylesheet" href="/styles.css">
</head>
<body>
<main class="app-shell">
<header class="topbar">
<div>
<p class="eyebrow">OpenSquared</p>
<h1>Assistant vocal</h1>
</div>
<button class="icon-button" id="resetButton" type="button" aria-label="Nouvelle conversation">
<span aria-hidden="true"></span>
</button>
</header>
<section class="status-band" id="statusBand">
<span class="status-dot" id="statusDot"></span>
<span id="statusText">Pret a demarrer</span>
</section>
<section class="conversation" id="conversation" aria-live="polite">
<article class="bubble assistant">
Bonjour Laurent. Appuie sur Demarrer, puis parle-moi naturellement.
</article>
</section>
<section class="composer">
<textarea id="textInput" rows="2" placeholder="Ecrire ou utiliser le micro"></textarea>
<div class="controls">
<button class="secondary-button" id="sendButton" type="button">Envoyer</button>
<button class="primary-button" id="voiceButton" type="button">
<span class="mic-icon" aria-hidden="true"></span>
<span id="voiceLabel">Demarrer</span>
</button>
</div>
<label class="toggle-row">
<input id="audioToggle" type="checkbox" checked>
<span>Lire les reponses a voix haute</span>
</label>
</section>
</main>
<script src="/app.js" type="module"></script>
</body>
</html>

View File

@@ -0,0 +1,19 @@
{
"name": "OpenSquared Assistant",
"short_name": "Assistant",
"description": "Assistant vocal personnel pour envoyer des emails et piloter des actions.",
"start_url": "/",
"scope": "/",
"display": "standalone",
"background_color": "#0d1117",
"theme_color": "#111827",
"orientation": "portrait",
"icons": [
{
"src": "/icons/icon.svg",
"sizes": "any",
"type": "image/svg+xml",
"purpose": "any maskable"
}
]
}

225
static/styles.css Normal file
View File

@@ -0,0 +1,225 @@
:root {
color-scheme: dark;
--bg: #0d1117;
--panel: #151b23;
--panel-strong: #1f2937;
--text: #f8fafc;
--muted: #9ca3af;
--line: rgba(255,255,255,.1);
--accent: #22c55e;
--accent-2: #38bdf8;
--danger: #fb7185;
}
* {
box-sizing: border-box;
}
body {
margin: 0;
min-height: 100svh;
font-family: Inter, ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
background:
radial-gradient(circle at 10% 0%, rgba(56, 189, 248, .18), transparent 32%),
radial-gradient(circle at 90% 10%, rgba(34, 197, 94, .16), transparent 30%),
var(--bg);
color: var(--text);
}
button,
textarea {
font: inherit;
}
.app-shell {
width: min(100%, 520px);
min-height: 100svh;
margin: 0 auto;
display: grid;
grid-template-rows: auto auto 1fr auto;
padding: max(18px, env(safe-area-inset-top)) 16px max(16px, env(safe-area-inset-bottom));
gap: 14px;
}
.topbar {
display: flex;
align-items: center;
justify-content: space-between;
gap: 16px;
}
.eyebrow {
margin: 0 0 4px;
color: var(--accent-2);
font-size: 12px;
font-weight: 700;
text-transform: uppercase;
}
h1 {
margin: 0;
font-size: 30px;
line-height: 1.05;
letter-spacing: 0;
}
.icon-button {
width: 44px;
height: 44px;
border: 1px solid var(--line);
border-radius: 8px;
color: var(--text);
background: rgba(255,255,255,.06);
}
.status-band {
display: flex;
align-items: center;
gap: 10px;
min-height: 44px;
padding: 0 14px;
border: 1px solid var(--line);
border-radius: 8px;
background: rgba(21, 27, 35, .82);
color: var(--muted);
}
.status-dot {
width: 10px;
height: 10px;
border-radius: 999px;
background: var(--muted);
}
.status-dot.live {
background: var(--accent);
box-shadow: 0 0 0 6px rgba(34, 197, 94, .12);
}
.status-dot.error {
background: var(--danger);
}
.conversation {
overflow-y: auto;
padding: 4px 2px 12px;
display: flex;
flex-direction: column;
gap: 10px;
}
.bubble {
max-width: 88%;
padding: 12px 14px;
border-radius: 8px;
line-height: 1.38;
white-space: pre-wrap;
word-break: break-word;
}
.bubble.user {
align-self: flex-end;
background: #2563eb;
color: white;
}
.bubble.assistant {
align-self: flex-start;
background: var(--panel);
border: 1px solid var(--line);
}
.bubble.error {
align-self: flex-start;
background: rgba(251, 113, 133, .12);
border: 1px solid rgba(251, 113, 133, .35);
}
.composer {
display: grid;
gap: 10px;
padding-top: 4px;
}
textarea {
width: 100%;
resize: none;
border: 1px solid var(--line);
border-radius: 8px;
padding: 12px;
color: var(--text);
background: rgba(21, 27, 35, .96);
outline: none;
}
textarea:focus {
border-color: rgba(56, 189, 248, .75);
}
.controls {
display: grid;
grid-template-columns: .8fr 1.2fr;
gap: 10px;
}
.primary-button,
.secondary-button {
height: 56px;
border: 0;
border-radius: 8px;
color: var(--text);
font-weight: 800;
}
.primary-button {
display: flex;
align-items: center;
justify-content: center;
gap: 10px;
background: linear-gradient(135deg, #16a34a, #0ea5e9);
}
.primary-button.recording {
background: linear-gradient(135deg, #dc2626, #f97316);
}
.secondary-button {
background: var(--panel-strong);
border: 1px solid var(--line);
}
.mic-icon {
width: 14px;
height: 22px;
border: 2px solid currentColor;
border-radius: 10px;
position: relative;
}
.mic-icon::after {
content: "";
position: absolute;
left: 50%;
bottom: -9px;
width: 18px;
height: 9px;
border-bottom: 2px solid currentColor;
border-left: 2px solid currentColor;
border-right: 2px solid currentColor;
border-radius: 0 0 10px 10px;
transform: translateX(-50%);
}
.toggle-row {
display: flex;
align-items: center;
gap: 10px;
color: var(--muted);
font-size: 14px;
}
.toggle-row input {
width: 20px;
height: 20px;
accent-color: var(--accent);
}

31
static/sw.js Normal file
View File

@@ -0,0 +1,31 @@
const CACHE_NAME = "opensquared-assistant-v1";
const ASSETS = [
"/",
"/styles.css",
"/app.js",
"/manifest.webmanifest",
"/icons/icon.svg"
];
self.addEventListener("install", (event) => {
event.waitUntil(caches.open(CACHE_NAME).then((cache) => cache.addAll(ASSETS)));
self.skipWaiting();
});
self.addEventListener("activate", (event) => {
event.waitUntil(
caches.keys().then((keys) =>
Promise.all(keys.filter((key) => key !== CACHE_NAME).map((key) => caches.delete(key)))
)
);
self.clients.claim();
});
self.addEventListener("fetch", (event) => {
if (event.request.method !== "GET") return;
event.respondWith(
caches.match(event.request).then((cached) => {
return cached || fetch(event.request);
})
);
});