Add hands-free PWA mode and systemd service

2026-04-27 08:20:51 +02:00
parent 606a925724
commit f876b3a635
9 changed files with 339 additions and 70 deletions
--- a/.env.example
+++ b/.env.example
@@ -2,3 +2,4 @@ GROQ_API_KEY=your_groq_api_key
 GMAIL_EMAIL=opensquaredgeneva@gmail.com
 GMAIL_PASSWORD=your_gmail_app_password
 APP_SECRET=change_me_to_a_long_random_string
+TTS_PROVIDER=browser
--- a/DEPLOY_NOTES.md
+++ b/DEPLOY_NOTES.md
@@ -18,6 +18,16 @@ fb8cc4d Initial PWA assistant proof of concept
 1b9933d Fix voice button audio unlock hang
 ```

+Derniers changements locaux a deployer:
+
+```text
+- Service systemd `opensquared-assistant` pour tourner hors session Putty.
+- Raccourci optionnel `/usr/local/bin/openbot`.
+- Mode mains libres: `Demarrer` ecoute en continu, envoi apres 2 secondes de silence.
+- TTS par defaut via voix navigateur `fr-FR` pour eviter l'accent anglais Groq.
+- Cache PWA: opensquared-assistant-v5.
+```
+
 ## VPS

 ```text
@@ -49,6 +59,37 @@ source .venv/bin/activate
 pip install -r requirements.txt
 ```

+## Service systemd
+
+Installer ou mettre a jour le service:
+
+```bash
+cd /root/openbot
+cp deploy/opensquared-assistant.service /etc/systemd/system/opensquared-assistant.service
+systemctl daemon-reload
+systemctl enable --now opensquared-assistant
+```
+
+Commandes quotidiennes:
+
+```bash
+systemctl start opensquared-assistant
+systemctl stop opensquared-assistant
+systemctl restart opensquared-assistant
+systemctl status opensquared-assistant
+journalctl -u opensquared-assistant -f
+```
+
+Raccourci optionnel:
+
+```bash
+install -m 755 /root/openbot/deploy/openbot /usr/local/bin/openbot
+openbot start
+openbot stop
+openbot restart
+openbot logs
+```
+
 ## HTTPS et Nginx Docker

 Le VPS utilise deja un `docker-compose.yml` avec un conteneur `nginx`.
@@ -130,8 +171,11 @@ Le bouton `Demarrer` etait non reactif sur navigateur parce que `unlockAudio()`
 Correctif applique:

 - `unlockAudio()` utilise maintenant `AudioContext`.
- `index.html` charge `/app.js?v=4`.
- Service worker: `opensquared-assistant-v4`.
+- `index.html` charge `/app.js?v=5`.
+- Service worker: `opensquared-assistant-v5`.
+- Le bouton `Demarrer` lance le mode mains libres.
+- Le message vocal est envoye automatiquement apres 2 secondes de silence.
+- La voix par defaut est celle du navigateur en `fr-FR`, via `TTS_PROVIDER=browser`.

 Si le bouton ne reagit pas apres un deploiement:

--- a/README.md
+++ b/README.md
@@ -24,6 +24,7 @@ GROQ_API_KEY=...
 GMAIL_EMAIL=...
 GMAIL_PASSWORD=...
 APP_SECRET=...
+TTS_PROVIDER=browser
 ```

 Installer :
@@ -48,12 +49,12 @@ Sur telephone, le micro necessite HTTPS sauf cas particulier localhost.

 ## Deploiement VPS Hostinger

-Exemple Ubuntu :
+Exemple Ubuntu pour le VPS actuel (`/root/openbot`) :

 ```bash
-cd /var/www
-git clone <ton-repo-ou-upload> opensquared-assistant
-cd opensquared-assistant/pwa_assistant
+cd /root
+git clone https://gitea.open-squared.tech/admin/openbot.git openbot
+cd openbot
 python3 -m venv .venv
 . .venv/bin/activate
 pip install -r requirements.txt
@@ -61,31 +62,31 @@ cp .env.example .env
 nano .env
 ```

-Service systemd :
-
-```ini
-[Unit]
-Description=OpenSquared Assistant PWA
-After=network.target
-
-[Service]
-WorkingDirectory=/var/www/opensquared-assistant/pwa_assistant
-EnvironmentFile=/var/www/opensquared-assistant/pwa_assistant/.env
-ExecStart=/var/www/opensquared-assistant/pwa_assistant/.venv/bin/gunicorn -w 2 -b 127.0.0.1:8080 app:app
-Restart=always
-RestartSec=3
-
-[Install]
-WantedBy=multi-user.target
-```
-
-Commandes :
+Service systemd pour que l'app continue apres fermeture de Putty :

 ```bash
-sudo nano /etc/systemd/system/opensquared-assistant.service
-sudo systemctl daemon-reload
-sudo systemctl enable --now opensquared-assistant
-sudo systemctl status opensquared-assistant
+cp deploy/opensquared-assistant.service /etc/systemd/system/opensquared-assistant.service
+systemctl daemon-reload
+systemctl enable --now opensquared-assistant
+```
+
+Commandes faciles :
+
+```bash
+systemctl start opensquared-assistant
+systemctl stop opensquared-assistant
+systemctl restart opensquared-assistant
+systemctl status opensquared-assistant
+journalctl -u opensquared-assistant -f
+```
+
+Optionnel, installer le raccourci `openbot` :
+
+```bash
+install -m 755 deploy/openbot /usr/local/bin/openbot
+openbot status
+openbot restart
+openbot logs
 ```

 Nginx :
@@ -95,7 +96,7 @@ server {
    server_name assistant.ton-domaine.com;

    location / {
-        proxy_pass http://127.0.0.1:8080;
+        proxy_pass http://172.17.0.1:8080;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
@@ -116,10 +117,14 @@ sudo certbot --nginx -d assistant.ton-domaine.com
 2. Menu `...`.
 3. `Ajouter a l'ecran d'accueil` ou `Installer l'application`.
 4. Ouvrir l'icone `Assistant`.
-5. Appuyer une fois sur `Demarrer` pour autoriser micro/audio.
+5. Appuyer une fois sur `Demarrer` pour lancer le mode mains libres.
+6. Parler naturellement : apres 2 secondes de silence, le message est envoye.
+7. Appuyer sur `Arreter` pour couper l'ecoute continue.

 ## Notes

 - Groq Orpheus TTS doit etre accepte dans la console Groq pour fonctionner :
  https://console.groq.com/playground?model=canopylabs%2Forpheus-v1-english
+- Par defaut `TTS_PROVIDER=browser` utilise la voix francaise du telephone pour eviter l'accent anglais du modele Groq Orpheus.
+- Pour revenir au TTS Groq, mettre `TTS_PROVIDER=groq` dans `.env`.
 - La lecture audio automatique depend toujours des regles du navigateur mobile, mais elle marche mieux dans une PWA apres un tap utilisateur que dans Telegram.
--- a/app.py
+++ b/app.py
@@ -25,6 +25,7 @@ CHAT_MODEL = "llama-3.3-70b-versatile"
 STT_MODEL = "whisper-large-v3-turbo"
 TTS_MODEL = "canopylabs/orpheus-v1-english"
 TTS_VOICE = "troy"
+TTS_PROVIDER = os.getenv("TTS_PROVIDER", "browser").lower()

 app = Flask(__name__, static_folder=str(STATIC_DIR), static_url_path="")
 app.secret_key = os.getenv("APP_SECRET", "dev-secret-change-me")
@@ -42,6 +43,7 @@ Quand tu as toutes les informations pour envoyer un email, reponds uniquement av
 {"action": "envoyer_email", "destinataire": "...", "sujet": "...", "corps": "..."}

 Pour tout le reste, reponds en francais naturel, avec des phrases courtes.
+N'utilise pas d'anglais sauf si l'utilisateur le demande explicitement.
 En mode voiture, sois bref : une ou deux phrases maximum."""


@@ -170,7 +172,7 @@ def traiter_message(session_id, texte, avec_audio=False):
        email_sent = True

    audio_url = None
-    if avec_audio:
+    if avec_audio and TTS_PROVIDER == "groq":
        audio_url = generer_audio(reponse)

    return {
@@ -211,13 +213,14 @@ def chat():
 def voice():
    fichier = request.files.get("audio")
    session_id = request.form.get("sessionId") or "default"
+    avec_audio = request.form.get("withAudio", "false").lower() == "true"

    if not fichier:
        return jsonify({"error": "Audio manquant"}), 400

    try:
        texte = transcrire_audio(fichier).strip()
-        resultat = traiter_message(session_id, texte, avec_audio=True)
+        resultat = traiter_message(session_id, texte, avec_audio=avec_audio)
        resultat["transcript"] = texte
        return jsonify(resultat)
    except Exception as e:
@@ -234,4 +237,8 @@ def reset():


 if __name__ == "__main__":
-    app.run(host="0.0.0.0", port=int(os.getenv("PORT", "8080")), debug=True)
+    app.run(
+        host="0.0.0.0",
+        port=int(os.getenv("PORT", "8080")),
+        debug=os.getenv("FLASK_DEBUG", "0") == "1",
+    )
--- a/deploy/openbot
+++ b/deploy/openbot
@@ -0,0 +1,18 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+SERVICE="opensquared-assistant"
+ACTION="${1:-status}"
+
+case "$ACTION" in
+  start|stop|restart|status)
+    systemctl "$ACTION" "$SERVICE"
+    ;;
+  logs)
+    journalctl -u "$SERVICE" -f
+    ;;
+  *)
+    echo "Usage: openbot {start|stop|restart|status|logs}" >&2
+    exit 2
+    ;;
+esac
--- a/deploy/opensquared-assistant.service
+++ b/deploy/opensquared-assistant.service
@@ -0,0 +1,14 @@
+[Unit]
+Description=OpenSquared Assistant PWA
+After=network.target
+
+[Service]
+WorkingDirectory=/root/openbot
+EnvironmentFile=/root/openbot/.env
+ExecStart=/root/openbot/.venv/bin/gunicorn -w 2 -b 0.0.0.0:8080 app:app
+Restart=always
+RestartSec=3
+User=root
+
+[Install]
+WantedBy=multi-user.target
--- a/static/app.js
+++ b/static/app.js
@@ -8,11 +8,26 @@ const voiceLabel = document.querySelector("#voiceLabel");
 const resetButton = document.querySelector("#resetButton");
 const audioToggle = document.querySelector("#audioToggle");

+const SILENCE_DELAY_MS = 2000;
+const MIN_VOICE_MS = 450;
+const MAX_UTTERANCE_MS = 30000;
+const VOICE_THRESHOLD = 0.025;
+
 const sessionId = getSessionId();
 let recorder = null;
 let chunks = [];
-let isRecording = false;
 let audioUnlocked = false;
+let handsFreeMode = false;
+let processingVoice = false;
+let mediaStream = null;
+let audioContext = null;
+let analyser = null;
+let analyserData = null;
+let monitorFrame = null;
+let speechDetected = false;
+let speechStartedAt = 0;
+let silenceStartedAt = null;
+let utteranceStartedAt = 0;

 if ("serviceWorker" in navigator) {
  navigator.serviceWorker.register("/sw.js").catch(() => {});
@@ -89,7 +104,7 @@ async function sendText() {
      body: JSON.stringify({
        sessionId,
        text,
-        withAudio: audioToggle.checked,
+        withAudio: false,
      }),
    });
    const data = await response.json();
@@ -102,6 +117,14 @@ async function sendText() {
 }

 async function toggleVoice() {
+  if (handsFreeMode) {
+    stopHandsFree("Arrete");
+    return;
+  }
+  await startHandsFree();
+}
+
+async function startHandsFree() {
  setStatus("Activation du micro", "live");
  await unlockAudio();

@@ -117,52 +140,167 @@ async function toggleVoice() {
    return;
  }

-  if (isRecording) {
-    recorder.stop();
-    return;
-  }
-
  try {
-    const stream = await navigator.mediaDevices.getUserMedia({audio: true});
-    const mimeType = pickMimeType();
-    recorder = new MediaRecorder(stream, mimeType ? {mimeType} : undefined);
-    chunks = [];
-
-    recorder.ondataavailable = (event) => {
-      if (event.data.size) chunks.push(event.data);
-    };
-
-    recorder.onstop = async () => {
-      stream.getTracks().forEach((track) => track.stop());
-      isRecording = false;
-      voiceButton.classList.remove("recording");
-      voiceLabel.textContent = "Demarrer";
-      await sendVoiceBlob(new Blob(chunks, {type: recorder.mimeType || "audio/webm"}));
-    };
-
-    recorder.start();
-    isRecording = true;
+    mediaStream = await navigator.mediaDevices.getUserMedia({
+      audio: {
+        echoCancellation: true,
+        noiseSuppression: true,
+        autoGainControl: true,
+      },
+    });
+    handsFreeMode = true;
    voiceButton.classList.add("recording");
-    voiceLabel.textContent = "Stop";
-    setStatus("Je t'ecoute", "live");
+    voiceLabel.textContent = "Arreter";
+    await startListeningTurn();
  } catch (error) {
    addBubble("Micro indisponible : " + error.message, "error");
    setStatus("Micro indisponible", "error");
  }
 }

+async function startListeningTurn() {
+  if (!handsFreeMode || processingVoice) return;
+
+  cleanupRecorder();
+  chunks = [];
+  speechDetected = false;
+  speechStartedAt = 0;
+  silenceStartedAt = null;
+  utteranceStartedAt = Date.now();
+
+  const mimeType = pickMimeType();
+  recorder = new MediaRecorder(mediaStream, mimeType ? {mimeType} : undefined);
+
+  recorder.ondataavailable = (event) => {
+    if (event.data.size) chunks.push(event.data);
+  };
+
+  recorder.onstop = async () => {
+    stopVolumeMonitor();
+    if (!handsFreeMode) return;
+
+    const spokenLongEnough = speechDetected && Date.now() - speechStartedAt >= MIN_VOICE_MS;
+    if (!spokenLongEnough) {
+      window.setTimeout(startListeningTurn, 120);
+      return;
+    }
+
+    const blob = new Blob(chunks, {type: recorder.mimeType || "audio/webm"});
+    await sendVoiceBlob(blob);
+  };
+
+  recorder.start(250);
+  startVolumeMonitor();
+  setStatus("Je t'ecoute", "live");
+}
+
 function pickMimeType() {
  const types = ["audio/webm;codecs=opus", "audio/webm", "audio/ogg;codecs=opus"];
  return types.find((type) => MediaRecorder.isTypeSupported(type)) || "";
 }

+function startVolumeMonitor() {
+  stopVolumeMonitor();
+
+  const AudioContext = window.AudioContext || window.webkitAudioContext;
+  audioContext = new AudioContext();
+  const source = audioContext.createMediaStreamSource(mediaStream);
+  analyser = audioContext.createAnalyser();
+  analyser.fftSize = 2048;
+  analyserData = new Uint8Array(analyser.fftSize);
+  source.connect(analyser);
+
+  const tick = () => {
+    if (!recorder || recorder.state !== "recording") return;
+
+    const now = Date.now();
+    const volume = getCurrentVolume();
+    if (volume > VOICE_THRESHOLD) {
+      if (!speechDetected) {
+        speechStartedAt = now;
+        setStatus("Je t'entends", "live");
+      }
+      speechDetected = true;
+      silenceStartedAt = null;
+    } else if (speechDetected) {
+      silenceStartedAt = silenceStartedAt || now;
+      if (now - silenceStartedAt >= SILENCE_DELAY_MS) {
+        setStatus("Message recu", "live");
+        recorder.stop();
+        return;
+      }
+    }
+
+    if (now - utteranceStartedAt >= MAX_UTTERANCE_MS) {
+      setStatus("Message recu", "live");
+      recorder.stop();
+      return;
+    }
+
+    monitorFrame = window.requestAnimationFrame(tick);
+  };
+
+  monitorFrame = window.requestAnimationFrame(tick);
+}
+
+function getCurrentVolume() {
+  analyser.getByteTimeDomainData(analyserData);
+  let sum = 0;
+  for (const value of analyserData) {
+    const centered = (value - 128) / 128;
+    sum += centered * centered;
+  }
+  return Math.sqrt(sum / analyserData.length);
+}
+
+function stopVolumeMonitor() {
+  if (monitorFrame) {
+    window.cancelAnimationFrame(monitorFrame);
+    monitorFrame = null;
+  }
+  if (audioContext) {
+    audioContext.close().catch(() => {});
+    audioContext = null;
+  }
+  analyser = null;
+  analyserData = null;
+}
+
+function cleanupRecorder() {
+  stopVolumeMonitor();
+  recorder = null;
+  chunks = [];
+}
+
+function stopHandsFree(label = "Pret") {
+  handsFreeMode = false;
+  processingVoice = false;
+
+  if (recorder && recorder.state !== "inactive") {
+    recorder.onstop = null;
+    recorder.stop();
+  }
+  cleanupRecorder();
+
+  if (mediaStream) {
+    mediaStream.getTracks().forEach((track) => track.stop());
+    mediaStream = null;
+  }
+
+  window.speechSynthesis?.cancel();
+  voiceButton.classList.remove("recording");
+  voiceLabel.textContent = "Demarrer";
+  setStatus(label, "idle");
+}
+
 async function sendVoiceBlob(blob) {
-  addBubble("Message vocal envoye", "user");
+  processingVoice = true;
  setStatus("Transcription", "live");

  const form = new FormData();
  const extension = blob.type.includes("ogg") ? "ogg" : "webm";
  form.append("sessionId", sessionId);
+  form.append("withAudio", "false");
  form.append("audio", blob, `voice.${extension}`);

  try {
@@ -174,23 +312,65 @@ async function sendVoiceBlob(blob) {
  } catch (error) {
    addBubble(error.message, "error");
    setStatus("Erreur", "error");
+  } finally {
+    processingVoice = false;
+    if (handsFreeMode) {
+      window.setTimeout(startListeningTurn, 250);
+    }
  }
 }

 async function handleAssistantResponse(data) {
-  addBubble(data.reply || "", "assistant");
+  const reply = data.reply || "";
+  addBubble(reply, "assistant");
  setStatus(data.emailSent ? "Email envoye" : "Pret", data.emailSent ? "live" : "idle");

-  if (audioToggle.checked && data.audioUrl) {
+  if (!audioToggle.checked || !reply) return;
+
+  if (data.audioUrl) {
    try {
      const audio = new Audio(data.audioUrl);
      await audio.play();
+      return;
    } catch {
-      addBubble("Audio pret, mais le navigateur a bloque la lecture automatique.", "error");
+      // Fall through to the local French browser voice.
    }
  }
+
+  await speakInFrench(reply);
 }

+function speakInFrench(text) {
+  return new Promise((resolve) => {
+    if (!("speechSynthesis" in window) || !("SpeechSynthesisUtterance" in window)) {
+      resolve();
+      return;
+    }
+
+    window.speechSynthesis.cancel();
+    const utterance = new SpeechSynthesisUtterance(text);
+    utterance.lang = "fr-FR";
+    utterance.rate = 0.98;
+    utterance.pitch = 1;
+    utterance.voice = pickFrenchVoice();
+    utterance.onend = resolve;
+    utterance.onerror = resolve;
+    window.speechSynthesis.speak(utterance);
+  });
+}
+
+function pickFrenchVoice() {
+  const voices = window.speechSynthesis.getVoices();
+  return (
+    voices.find((voice) => voice.lang === "fr-FR" && /google|microsoft/i.test(voice.name)) ||
+    voices.find((voice) => voice.lang === "fr-FR") ||
+    voices.find((voice) => voice.lang && voice.lang.toLowerCase().startsWith("fr")) ||
+    null
+  );
+}
+
+window.speechSynthesis?.addEventListener?.("voiceschanged", pickFrenchVoice);
+
 async function resetConversation() {
  await fetch("/api/reset", {
    method: "POST",
--- a/static/index.html
+++ b/static/index.html
@@ -30,7 +30,7 @@

      <section class="conversation" id="conversation" aria-live="polite">
        <article class="bubble assistant">
-          Bonjour Laurent. Appuie sur Demarrer, puis parle-moi naturellement.
+          Bonjour Laurent. Appuie sur Demarrer, puis parle-moi naturellement. Je transmets apres deux secondes de silence.
        </article>
      </section>

@@ -50,6 +50,6 @@
      </section>
    </main>

-    <script src="/app.js?v=4" type="module"></script>
+    <script src="/app.js?v=5" type="module"></script>
  </body>
 </html>
--- a/static/sw.js
+++ b/static/sw.js
@@ -1,8 +1,8 @@
-const CACHE_NAME = "opensquared-assistant-v4";
+const CACHE_NAME = "opensquared-assistant-v5";
 const ASSETS = [
  "/",
  "/styles.css",
-  "/app.js?v=4",
+  "/app.js?v=5",
  "/manifest.webmanifest",
  "/icons/icon-v2.svg"
 ];