justlv/onju-voice

A hackable AI home assistant platform

Onju Voice 🍐🔈

💫 DEMO's

A hackable AI home assistant platform using the Google Nest Mini (2nd gen) form factor, consisting of:

• a custom PCB designed to be a drop-in replacement to the original, using the ESP32-S3 for audio processing
• a server for handling the transcription, response generation and Text-to-Speech from multiple devices on the same network

(This repo focuses on the experimental conversational LLM aspect to replicate some functionality shown in the demos, and not as a full fledged replacement to a home assistant. This is not being actively maintained, but I've released all source code and design files for anyone else to pick up from here.)

Overview

This repo contains firmware, server code and some example applications, intended to be as accessible as possible for getting up and running i.e.:

• Firmware for the custom PCB can be programmed using the Arduino IDE and a USB cable (installation of ESP-IDF not required)
• Server code has minimal requirements besides running Whisper locally, and should be able to run on most devices that you can leave plugged in whether MacOS / Linux / Win etc.
• Hardware can be ordered from PCBWay and Altium design files are included

Example applications

• 📩 Querying and replying to messages (using a custom Maubot plugin & Beeper)
• 💡 Light control with Home Assistant
• 📝 Adding and retrieving notes/memos for the LLM to craft a response with

Not included:

• 👥 Multiple voice characters. I’ll leave it to the user to clone voices as they deem fair use. Also from experience LLM’s < GPT4 don’t consistently enough follow instructions to reliably respond in different characters AND perform multiple function calling with complicated prompts.

Current features of the device <> server platform

• Auto-discovery of devices using multicast announcements
• Remembering conversation history and voice settings for each device
• Sending & receiving audio data from the device, packed as 16-bit, 16kHz (UDP sending, TCP receiving partially buffered into PSRAM)
• Speaker and microphone visualization with the LED’s, and custom LED control via the server
• Mute switch functionality, tap-to-wake for enabling the microphone, and setting mic timeout via the server
• Device-level logging to individual files and console output using rich

Limitations of this release:

• The Arduino IDE doesn’t (yet) support the Espressif’s Audio SDK’s, such as ESP-ADF, ESP-Skainet etc. For these demo's it's not absolutely required, but if you use Espressif’s ESP-IDF with these SDK's you'd unlock features such as:
  • VAD (Voice Activity Detection) - in this example VAD is offloaded to the server using webrtcvad, and the listening period is extended by either tapping the device or by the server sending mic keep alive timeouts (network traffic is really minimal at 16-bit, 16kHz)
  • AEC (Acoustic Echo Cancellation) - to allow you to effectively talk over the assistant by removing the speaker output from audio input
  • BSS (Blind Source Separation) - let’s you use both mic’s for isolating speakers based on location, and other noise suppression
  • Wakewords and other on-device commands - I’m not a believer in this given how finicky these can be and don’t think these are and think all command logic should be handled by layers of language models on the server.
• The server currently only does transcription locally and uses:
  • OpenAI for generating responses & functions calls, but if you have the hardware you could run a local LLM, using something like ToolLLM for calling API’s to add almost any capabilities you’d wish.
  • Text-to-speech from Elevenlabs - this is fair to say the easiest to get running, fastest and most expressive option out there but FWIR data policy is a little dubious so careful about sending anything too sensitive. I’d really like to see comparable performing open source options that you can run locally
• Conversation flow is highly serialized, i.e. recording > transcription > LLM > TTS needs to finish each step before moving onto the next. Not included here is feeding incomplete transcriptions to a smaller model, and streaming slower LLM's like GPT4 to Elevenlabs and sending streaming responses back, it's currently a little too hacky to include in this release.
• No wakeword usage, mostly done intentionally as I feel uttering a wake-word before every response is a terrible experience. This currently uses a combo of VAD, mic-timeouts sent from server, tap-to-wake, mute switch usage etc. Not included here is experiments running a smaller, faster LLM for classification with a running transcription before handing off to a larger LLM with specific prompt

Other areas for improvement

These are things I didn't get time to implement but I believe would be invaluable and pretty achievable

• Speaker diarization - know who is saying what, and have the LLM enage in multi-user conversations or infer when it isn't being spoken to
• Interruptions - requires AEC for simultaneous listening and playback
• Smaller local models/LLM's for running classification, detecting intent and routing to larger LLM's

Installation

🖥️ Server

Ensure you can install Whisper and run at least the base model, following any debugging steps they have if not. If you can get past that, it should be as simple as:

cd server
pip install -r requirements.txt

Adjust settings in the config.yaml, and tweak aspects such as how much silence is needed to start processing to trade-off snappiness vs avoiding cutting off the user.

Add your Elevenlabs token to credentials.json and ensure you have a cloned voice in your account that you set in the config.yaml under elevenlabs_default_voice

You'll also need a greeting WAV set in config.yaml under greeting_wav, that will be sent to devices on connecting to the WiFi. This is up to you to record or procure (e.g.)

A small subset of the config parameters can be set as optional arguments when running the script. For e.g. the following will run the server with note-taking, Home Assistant, Maubot, real sending of messages enabled (a safe guard disabled by default), and a smaller English only Whisper model for transcription.

python server.py --n --ha --mb --send --whisper base.en

🏡 Home Assistant

I recommend setting this up on the same server or one that is always plugged in on your network, following the Docker Compose instructions

Then go through the onboarding, setup a user, name your devices and get a Long Lived token to add to credentials.json together with the URL e.g. http://my-local-server:8123/

🤖 Maubot

Follow instructions here to setup Maubot with your Beeper account. Ensure the correct URL is setup in config.yaml, set send_replies to True if your friends are forgiving of the odd mistakes, and set a footer.

Don’t have Beeper yet and can’t wait? Try setup a Matrix bridge yourself and a custom function definition for OpenAI function calling (and share how you did it!)

Following this example you can also integrate e-mail.

📟 Firmware

Irrespective of what you use for development, the quickest & least error prone setup for building & flashing firmware is probably installing the Arduino IDE Software, and then using this IDE or your preference i.e. VSCode for development (Copilot)

• Add the ESP32 boards as detailed here (TL;DR add https://espressif.github.io/arduino-esp32/package_esp32_index.json to Preferences > Additional Boards Manager URL’s)
• Under Boards Manager, install “esp32” by Espressif Systems
• Under Library Manager, install “Adafruit NeoPixel Library”
• Clone this repo to Documents/Arduino for simplicity.
• Add your WiFi credentials to credentials.h
• Run bash setup-git-hash.sh to add a header with the git-hash (optional). This will then automatically update after commits, and help track the firmware that your devices are running from the server side.
• Open File > Sketchbook > onju-home > onjuino
• Select Tools > Board > esp32 > ESP32S3 Dev Module
• Under Tools ensure:
  • USB CDC on Boot set to Enabled
  • PSRAM set to OPI PSRAM
  • Board is plugged in and Port is selected (you may need to install USB bridge drivers as detailed by Espressif, don’t worry if name is incorrect)
• Build and upload
• If not reset, press the reset button. In Serial Monitor you can also send r to reset the device (assuming it is already booted)

🧩 Hardware

Preview schematics & PCB here You should be able to download files, otherwise they are in the folder hardware in Altium format. Feel free to modify & improve this design and share your updates!

You can order PCBA's directly from PCBWay here. I've used a few suppliers and they are of the most reliable I've experienced for turnkey assembly at that pricepoint so I'm happy to point business their way. (Other options of selling single units, with margins, ended up forcing a pricepoint > Google Nest Mini itself, and wouldn't allow shipment into EU/UK without certification so I abandoned this)

I will be sharing more detailed instructions for replacement.

Replacement gaskets for the microphone & LED's can be made using adhesive foam and a punch set) for example

❓Questions

Does this replace my Google Nest Mini?

While this replicates the interfaces of the Google Nest Mini, don’t expect this to be a 1:1 replacement, for e.g. it is not intended to be a music playback device (although there is probably no reason it couldn’t be developed to be used as such). It’s also worth re-iterating that like the Google Nest Mini, this requires a separate server, although this can be in your home running local models instead of in a Google datacenter. The original is well tested, maintained, certified and works out the box, while this is essentially a dev board with some neat examples for you to build on top of

What if I don’t have a Google Nest Mini but still want to use this?

Fortunately they’re still being sold, you may find deals for <$40 which is pretty good for the quality of speaker and form factor. I picked up quite a few from eBay, just make sure you get the 2nd gen.

The adventurous can get try replacement shells from AliExpress for e.g., but you’ll still need a base, power input, mute switch, speaker & mount, capacitive touch panels, and replacement gaskets etc. A hero out there could design a custom enclosure that fits an off-the-shelf speaker.

But I’m really impatient and want to get hacking away! What can I do?

a) if you can commit to making significant contributions to the codebase and/or major contributions to the board design or RF review, we may be able to make early samples available

b) if you don’t need the form factor, don’t mind rolling up my sleeves, and have some HW experience, you can breadboard it out with readily available components until you can get your hands on an order. Here are the components that should be able to get a demo running (🌸 Adafruit link for convenience but shop around wherever you’d like)

• ESP32-S3 devboard, ideally w/ PSRAM (e.g. QT Py S3 or ESP32-S3)
• Microphone (only need 1 for the Arduino implementation, ensure it's a SPH0645 to limit debugging)
• Amplifier
• Speaker
• Neopixel LED strip - just set the firmware to the correct #
• Breadboard & wire kit (you can use protruding pieces of wire for cap touch)

You'll need to update the custom_boards.h with your pin mapping

🍐 PR's, issues, suggestions & general feedback welcome!🏡

onju-voice's Issues

Edge AI model?

Hello, great and well thought project.

I see you used the esp32-s3 which also has some AI capabilities. I was unable to dig deeper but i know that S3 can be used to run very small AI models (possibly for speech recognition).

My previous research showed that they only supported mandarin and english but i was unable to look for a way to build custom models.

Maybe there is room to have the s3 do voice recognition instead of having a srv running and getting the audio stream? This is especially helpful when planning to have several of these devices around the house.

Using a ESP32-C6 instead of S3?

Hi Justin, super-interesting project!!
I'm a complete noob w.r.t. hardware development, so forgive me if my question is plain stupid 😉
Would it be possible to replace the ESP32-S3 with a ESP32-C6?
The reason being that the C6 supports Thread, so this would enable Matter over Thread use cases 😃

Default volume

Is there a solution to Set the Default volume to 50%

Right now in every reboot (Like after an ESPhome Update) the volume is Set to 100%, so i have to Set the volume Manuel everytime Back to 50%

How to Communicate with device

So I am a little lost. I now have two Minis set up. One that I loaded the ESPHome version into and that is working very well with HA.

I just loaded this repo into a second one. It looks like it is working correctly as the server code sees the board when it is powered up. I also see the devices from HA.

But I do not know how to actually use it? Is there a wake word?

description on how to build the board??

I ordered the board(s) and they arrived with no components.
is there any manual on how to add electronics?
or did I order the wrong thing?
how to I get to have an assembled board ready to be programmed and installed?

PCB orders and status

I would love to order a PCB with all the components already included and placed.

1. What is the current design status of PCB?
2. From where can we order these PCB's?
3. Can they get both made and populated?

Copycat PCB seller?

While doing some research on the Nest Mini's PCB, I stumbled across this website, which appears to be selling a branded version of the Onju Voice PCB: https://tritone-echo.com/product/tritone-echo-pcb/

It looks like they're blatantly ripping your (@justLV) work here, so I wanted to bring it to your attention. The license in this Repo is the MIT license, but you have the PCBs labeled as CC-NC, and that would appear to be in direct conflict with what this company is doing.

OpenvoiceOS and hivemind

Great project, is there a chance to use it as a hivemind client?

Unable to boot board

Probably not the best place for this but I have no idea what to try next. I've received my boards from PCBway but for some reason I can't get them to boot. I've plugged them into USB ports on several machines whilst holding the Boot button but none of them are being picked up. They're not even showing in device manager under Windows. Am I missing something or have I received 5 dead boards?

[REQUEST] New PCB with both an ESP32-S3 and an xCORE chip from XMOS for advanced audio processing as a ESPHome compatible voice-kit?

Any chance someone has the skills, time + interest to re-design a new PCB for the Google Nest Mini and Google Home Mini speaker series that also includes an xCORE xCORE DSP chip from XMOS?

Not sure if all of you have followed news about Nabu Casa's upcmoing voice-assistant hardware project, but just heard Paulus Schoutsen reveal on their Home Assistant's ESPHome Summer Release Party on YouTube that Nabu Casa's ESPHome developers are working on a new open-source hardware platform for their voice-assistant products that will based on ESP32-S3 in combination with a very powerful XMOS xCORE DSP chip for advanced audio processing).

The XMOS xCORE chip adds in-line off-loading voice cleanup from the microphone(s), like Interference Cancellation (IC)​, Acoustic Echo Cancellation (AEC), Noise Suppression (NS), and Automatic Gain Control, etc. and/and other audio post-processing algorithms to improve the solution's voice recognition capabilities). (Depending on which XMOS chip they use their XCORE-VOICE framework could technically also allow also for up to 16 PDM microphones to be connected to a single xCORE device with a different PCB design).

UPDATE: The news about that upcoming voice-kit hardware platform by Nabu Casa is also Home Assistant roadmap, and that ESPHome-based voice-kit hardware has since also mentioned by Mike and Kevin during the Voice Chapter 7 livestream:

https://www.home-assistant.io/blog/2024/06/12/roadmap-2024h1/#current-priority-2-make-assist-easier-to-start-with

"Current priority 2: Make Assist easier to start with" ... "we’re exploring building our voice satellite hardware to create a more plug-and-play experience."

https://www.home-assistant.io/blog/2024/06/26/voice-chapter-7/

I think similar chips from XMOS like their XU-316 AI (xCORE XU316) is by the way used in Amazon Alexa Voice Service (AVS) Development Kit(s), and is used in some Amazon Echo products as well as other popular :

• https://www.xmos.com/develop/xcore-voice
• https://www.xmos.com/xk-voice-l71
• https://www.xmos.com/xmos-delivers-first-amazon-alexa-voice-service-development-kit-with-linear-mic-array-for-far-field-voice-capture/
• https://www.xmos.com/fully-offloaded-giving-smart-tvs-the-voice-power-they-deserve/
• https://www.xmos.com/making-smart-speakers-feel-at-home/
• https://developer.amazon.com/en-US/blogs/alexa/post/edde861e-65a4-4319-b5f1-422b0c626673/xmos-expands-device-types-supported-by-dev-kits-for-avs-with-a-far-field-linear-mic-array-based-reference-solutio
• https://developer.amazon.com/en-US/alexa/devices/alexa-built-in/development-resources/sdk

As far as I can tell the complete source code for XMOS's xcode-voice firmware is available on Github under sln_voice repo:

https://github.com/xmos/sln_voice

More information about that in their user-guide for their XK-VOICE-L71 Evaluatuion board:

https://www.xmos.com/download/XVF3610-User-Guide(v5_7_3).pdf

Since Nabu Casa's designs it said to be open-source hardware and XMOS integration will probably be added to the ESPHome's Media Player Components (and Microphone Components) I for one am hoping that it could and will be extended to different types of speakerless solutions with appliance solutions with AUX-output/audio-output and AUX-input/audio-input port and not only for voice-assistant.

Personally I would also love to see inexpensive speakerless network-streamer player/receiver hardware without microphones but only with with AUX-out that can connect to any of your existing amplifiers or speakers with built-in amplifiers in order to replace products like Chromecast Audio and Amazon Echo Input / Echo Link Amp, (e.i. devices with no on-board speakers that must be connected to external speakers for audio output (AUX-output).

That is, I am sure that not everyone only wants "smart speakers" with voice-assistant and that instead many would be also happy to have network streamers/players without microphone which only purpose is to receive and output highest quality audio possible from Music Assistant to your "dumb" speakers.

I for one still have loads of Chromecast Audio audio-only receivers connected to various models and brands of different speaker/reciever systems in each room used to achieve multi-room music playback on a budget (because could not afford Sonos speakers in all rooms).

So even if though Nabu Casa's hardware will initially primarly be designed for "Home Assistant Satellite" (also known as "Wyoming Satellite") for voice-assistant appliances, such open-source hardware it just like the ESPHome firmware does have a lot of potential for different use cases.

Also on my wishlist if a network streamer receiver hardware with AUX-input and ADC to get music from analog audio source. As an easy way to achieve a remote AUX input into Music Assistant from an external analog audio source like a vinyl record player (LP turntable) or cassette player.

What I want to achieve is a solution that is easy to install/maintain and use that allow my wife to stream music from a vinyl record player (LP turntable) to any speaker or group of speakers in our home. The vinyl record player (turntable) setup she has a pre-amp with phono (RCA) output ports for analog audio in stereo.

• Architecture example: Analog audio source with preamp -> ADC network appliance -> music stream -> Music Assistant -> Any speakers

I would therefore prefer if we could buy some kind of networked (Wi-Fi) enabled appliance like a music streamer with stereo AUX input port that it will use for on-the-fly perform analog-to-digital conversion (ADC) + encoding for streaming to a Music Provider inside Music Assistant.

I do however think that both such a solution does need its own non-propriatory audio-only streaming protocol for high-quality music streams?

Antenna choice?

How's the antenna performance on this design?
I'm asking because it seem a bit strange choice for using such a low gain antenna.

The one specified in your current design datasheet say that the gain is: -0.5 dBi, which is very bad.
I would rather suggest something using uFL connector with a gain of at least a 2.0 dBi.
Maybe something like:

• this (2.0 dBi)
• or Ignion NANO mXTEND™ (NN02-101) (2.4 dBi)

(Request) Please Consider Onju Voice PCB for Echo Dot (5) Kids

Hey,
First of all thank you, I bought already 5 PCBs of your development.
The way you think is a great example of Local helpers and self controlled devices.

I have not the knowledge to recreate the PCB for my kids Echo Dots but I would like to switch them to locally, so if you could spare the time I would appreciate it.

Thank you for your work!

How to flash

Hi! I've just received my PCBs and I thought I could flash them using the USB port, but after checking out the schematic, it turns out i can only do it with the debug header.

1. Is that correct? Is the USB port just for charging?
2. What type of flashing device do I need?

PCB download

Hi,
I would like to download the PCB files, but following this link it says that I don't have permission. Could you please give download permission or maybe upload the files to another service?
Thanks!

How to help?

Hi,
Super cool project, I would love to contribute and that is not just because I am impatient and want access to the early PCB 😃 . In all seriousness, I am an electrical engineer with some circuit design experience and a lot of software experience. Is there any specific tasks you want to tackle first?

[REQUEST] Onju Voice PCB variant for the Google Home Mini (1st gen) speaker with Micro-USB charging port

Please consider designing/making an Onju Voice PCB variant for the Google Home Mini (1st gen) with a Micro-USB charging port.

https://www.ifixit.com/Device/Google_Home_Mini

https://www.ifixit.com/Guide/Google+Home+Mini+Disassembly/109922

https://www.ifixit.com/Teardown/Google+Home+Mini+Teardown/102264

FCC ID A4R-H0A

https://fccid.io/A4RH0A

https://fcc.report/FCC-ID/A4RH0A

Bet quite a few people like myself also own many of the first-generation Google Nest Mini and are interested in converting them.

https://www.dimensions.com/element/google-home-mini

The downsides to the first-generation Google Home Mini are a worse speaker and worse speaker array compared to the newer generation, (as well as that Micro-USB charging port however you would probably want to still keep it a Micro-USB charging port since all of them shipped with a Micro-USB charger power-adapter).

https://en.wikipedia.org/wiki/Google_Nest_(smart_speakers)

https://support.google.com/googlenest/answer/7072284

[REQUEST] 3.5mm audio jack/plug (phone connector) as an line input auxiliary port (AUX) for music

Please consider adding an optional feature with 3.5mm audio jack/plug (phone connector) as an auxiliary port (AUX) for music input.

The main reason for the request is would love to be able to route and cast that I2S audio stream from that line input with retro analoge music stereo players that do not have built-in audio casting/streaming capabilities, like for example LP record players (turntables), cassette players, Music CD players, Mini-Disc Player, (which for semi-nostalgic and tactile reasons I do not want to just want to record and digitize to stream as FLAC files which many suggests doing instead as a workaround solition).

However not sure if this is technically possible for music input with the built-in ESP32 DAC features without also adding other I2S chip?

e.g. LP record player (turntable) with a preamp -> Onju-Voice with line input AUX port -> ESPHome -> Home Assistant -> Google Cast.

The idea comes from the "Google Home Max" speaker which difference itself by featuring a 3.5mm audio jack (phone connector) as as a line input auxiliary port (which is not the same as a microphone) and that enables the "Google Home Max" speaker to also cast Chromecast audio, which uniquely makes it the only commercial hardware appliance that features Chromecast audio casting.

ESPHome i2s_audio component for ESP32 allows for receiving and sending audio via I²S but not sure how well works for line input?

https://esphome.io/components/i2s_audio

This could perhaps be used with an audio casting add-on in Home Assistant like AirCast or similar to cast Chromecast audio or AirPlay?

https://github.com/hassio-addons/addon-aircast

https://github.com/hassio-addons/addon-airsonos

PS: My ultimate goal would be to achieve an open-source multi-room sound system with casting done over some open audio protocol.

Let Elevenlabs use new multilangual v2 model

At default, the Elevenlabs API uses the old model. The multilangual model v2 is much better and sounds more natural in other languages (Dutch for me)

Here is the code to change the model. For the file elevenlabs.py around line 61

Before:

 def text_to_speech(self, device, text, path_name="data"):
        now = datetime.now()
        now_str = now.strftime("%Y-%m-%d_%H-%M-%S")
        payload = json.dumps({
            "text": text,
        })

After:

 def text_to_speech(self, device, text, path_name="data"):
        now = datetime.now()
        now_str = now.strftime("%Y-%m-%d_%H-%M-%S")
        payload = json.dumps({
            "text": text,
            "model_id": "eleven_multilingual_v2",
        })

[REQUEST] Onju Voice PCB variant for the "Google Home Max" speaker

Please consider designing/making an Onju Voice PCB variant for the "Google Home Max" speaker.

https://en.wikipedia.org/wiki/Google_Nest_(smart_speakers)

https://support.google.com/googlenest/answer/7072284

Bet quite a few people like myself also own a few of these original Google Home speakers and are interested in converting them.

https://www.dimensions.com/element/google-home-max

Feature-wise the main difference is that it features a 3.5-mm audio connector (phone connector) for audio input which enables it to cast Chromecast audio, which uniquely makes it the only commercial hardware appliance that features Chromecast audio casting.

FCC ID A4R-H0B

https://fccid.io/A4R-H0B

https://fcc.report/FCC-ID/A4R-H0B/3627625

https://apps.fcc.gov/oetcf/eas/reports/ViewExhibitReport.cfm?mode=Exhibits&RequestTimeout=500&calledFromFrame=N&application_id=ZJOgPO3nv280csenI2Dd0Q%3D%3D&fcc_id=A4R-H0B

FYI, other than on FCC, I could not find any free teardown information about the "Google Home Max" speaker model.

There is a paid version of a teardown report though:

https://www.reverse-costing.com/teardowns/pe_googleteardown-home-max-h0b-0/#features

QQ - where are the mic's?

Just putting some onjus together right now and I have one unit where I can't get the mic working. Where are the mics anyway? Is it on the board or in the speaker housing?

Order with JLCPCB?

@justLV this project is very exciting, thank you for sharing your work.

Is it possible that you could setup your design for purchase with JLCPCB as you have with PCBWay? I see that you get paid/credit if ordered via PCBWay, and I want that to continue with JLCPCB. I'm trying to find a cheaper supplier, but not cut you out of the deal!

Custom PBC support for Google Home Mini 1st gen?

Hello,

I'm interested in using a custom PBC with my Google Home Mini 1st gen device. I know the documentation recommends using 2nd gen devices, but since I already have a 1st gen unit, I wanted to check if custom PBC would work on it before purchasing an upgrade.

Does anyone have experience getting custom PBC working on the original Home Mini? Or are there technical limitations that prevent it from being supported?

Any insight would be appreciated. Thanks!

project future plans?

Hey Justin what are you future plans for this project?

I am asking cause folks like to have it available for different devices.
Could you think of Onju being a repo collecting ESP-S3 pcbs for different devices?

---

Also @tetele made an awesome ESPhome config for all HomeAssistant folks.
https://github.com/tetele/onju-voice-satellite
What are you opinions regarding referring in the readme to his documentation repo?

Greetings 👋

esphome compatibility?

esphome is a project that generates firmware for various esp32 microcontroller powered devices from configuration files. This project has recently added Voice Assistant support that uses I²S Audio.

How possible would it be to make esphome firmware work on onju-voice board?

Media Player does not work

I am having problem playing music through the device in Home Assistant and Music Assistant, the device does not respond at all after commands been sent, does the Media Player allow to play streaming music?

[REQUEST] Onju Voice PCB variants for the original Google Home speaker (a.k.a. Google Home Pineapple)

Please consider designing/making an Onju Voice PCB variants for the original Google Home speaker (a.k.a. Google Home Pineapple).

https://www.edn.com/teardown-google-home-smart-speaker/

https://www.ifixit.com/Teardown/Google+Home+Teardown/72684

https://www.theverge.com/2016/11/7/13549848/ifixit-teardown-google-home-same-chromecast

Bet quite a few people like myself also own a few of these original Google Home speakers and are interested in converting them.

https://en.wikipedia.org/wiki/Google_Nest_(smart_speakers)

https://support.google.com/googlenest/answer/7072284

How to connect the device to server?

Stupid question I know. But what am I missing here?

I have the device with custom PCB and firmware loaded.

I have the server running in the same WiFi network.

What next?

(I don't have homeassistant running)

Text to speech errors

I have put "Lily" in my voice lab on Elevenlabs.com.
I have put my elevenLabs token in cedentials.json
I have put "Lily" as my eleven_labs_default_voice in in config.yaml

I have NOT put anything in voices.json

I am getting the following errors:

WARNING onju-coral-V3: Voice 'Lily' not found, using default Lily elevenlabs.py:51 Exception in thread Thread-2 (transcribe_respond): Traceback (most recent call last): File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner self.run() File "/usr/lib/python3.10/threading.py", line 953, in run self._target(*self._args, **self._kwargs) File "/home/joe/onju-voice/server/server.py", line 156, in transcribe_respond wav_fname = tts.text_to_speech( File "/home/joe/onju-voice/server/elevenlabs.py", line 71, in text_to_speech voice_id = self.get_voice_id(device) File "/home/joe/onju-voice/server/elevenlabs.py", line 52, in get_voice_id return self.voices[self.default_voice]['voice_id'] KeyError: 'Lily'

On device wake word (microWakeWord) support?

Hi
The guys from Nabu Casa just announced that they managed to add on-device wake word detection (they called it microWakeWord) on the ESP S3-Box3 device.
Is there a chance we can benefit from this new feature on the "Onju-voice devices" in the future?

https://www.home-assistant.io/blog/2024/02/21/voice-chapter-6/

PCB/components Issue (Was a supplier issue)

Hi,

Ordered the PCB + Assembly with the provided link to PCB way.

Received feedback on the assembly with pictures :

For C28,C29 part package is 0603. Pad is 0402, we can not solder, please confirm what should we do?

Extract from the bill of materials :

C28, C29 2
Murata Electronics
GCM1555C1H160FA16D CAP CER 16PF 50V C0G/NP0 0402 CAP 0402_1005

Defect PCB?

I think my PCB IS defect, since a while i only have one LED working, in espHome there are 6 LEDs Set, but only one is working.

Onju for Echo dot ?

Hi @justLV ,
Is there a PCB version for the Amazon echo dot 5 ?
If not, cloud it be even possible to make one ?
Thanks

CPL?

I can't find the CPL file. Is it missing?

LED1 and LED6 orientation

I have received an engineering question from PCBWay, who are currently fabricating a set of replacement PBCs.

Which is pin 1 for the 2 LEDs?

PCB Conponent issue: BOM in PCBWay

Quotation and problem T-2D6W460623A-10sets-onju-voice-bom(2023-12-11).xls

I got quotation that there was problem in BOM in PCBway.

Value listed as [0.1W], but part value is [200MW], please confirm which one should we follow?

Which one is correct? Can I use 200MW ohm?

[REQUEST] Temperature Sensor (BME280)?

Hi JustLV and congratulations for the amazing job!
Looking at the pcb it seems like there is space to add something usefull, do you think a sensor for temperature (like the BME280) could fit? It would maybe need some adjustment due to the enclosured space of course, but add a thermometer could be very usefull in the home assistant world

Esp board compatibility

Could this board be used for this project?

ESP32-S3-Korvo-1 Development Board https://a.co/d/e4cm7tp

A way to add a display maybe?

I love my first tinkerings with Onju and must say it's the best solution for Home Assistant right now if you want some useful audio quality. But one thing would still be great to add... a screen.

These S3 Boxes have screens attached to the ESP32 and adding one of these in front or 3D print supported on top would be awesome. But I have no idea where to go to with such a screen if I even find one or steal it off such a S3 Box.

Ah btw, PCBWay has some problems with LEDs as it seems. 5 pcbs ordered and on all 5 at least on LED is semi dead. In this case it always means dead red color. I moved one working LED on one PCB to one PCB where only one LED is dead and now at least have one working PCB. Rest will be fixed when my LED order is here. Maybe I will sell 2-3 or them then.

P.S. When I say "I" on anything soldering realted I always mean MY BROTHER. ^^

PSRAM not available

Hi,

I ordered 5 pcbs with already soldered components. Out of these 5 pcbs one reports 0B PSRAM available. I'm using ESPHome with @tetele's MicroWakeWord configuration.

Configuration related to the board itself and PSRAM:

esp32:
  board: esp32-s3-devkitc-1
  framework:
    type: esp-idf
    version: recommended
    sdkconfig_options:
      CONFIG_ESP32_S3_BOX_BOARD: "y"

psram:
  mode: octal
  speed: 80MHz

After upload not much RAM is used:

RAM:   [=         ]  11.3% (used 36960 bytes from 327680 bytes)
Flash: [========  ]  82.9% (used 1521433 bytes from 1835008 bytes)

But when enabling micro wake word I receive an error:

[C][psram:020]: PSRAM:
[C][psram:021]:   Available: NO
...
...
[E][esp-idf:000]: E (20485) AUDIO_PIPELINE: audio_pipeline.c:255 (audio_pipeline_init): Memory exhausted

When I enabled debug I can see that there is 0B PSRAM available:

This is the problematic board:

Other PCB reports much more available PSRAM and even 2 times more free heap:

[C][psram:020]: PSRAM:
[C][psram:021]:   Available: YES
[C][psram:024]:   Size: 8191 KB

8 291 859 bytes is ~8MB which is accurate for ESP32-S3R8 board.

ESPHome config for both boards is exactly the same. What can be a reason for this issue?

Home Assistant Media Playback functionality.

Might be a duplicate of "#14"

Being able to use these board replacements with Home Assistant media playback would also be dope.

Using Home Assistant smart home software.
https://www.home-assistant.io/

Might fit in well with their "Year of the Voice".

I have to run a keep alive script to keep my nest speakers awake and semi synced which mutes, plays 1 second of silence, then unmutes, every 3 minutes.
They always get desynced and they also play the chime whenever an action is done to them other then mute if my script fails. (such as setting volume, play, pause)

Having a way to have all these synced and also not have the activation noise would be so nice and I would replace all the nest boards around my house.

Fingers crossed for boards for the larger nest audio as well.

Ready to use PCB

Is there anyone Out who has an spare PCB ready to use and is able to send it to Germany? For testing this i dont want to Order 5 PCBs

Cannot get the speaker to work in ESP-IDF

I am trying to port the provided firmware to ESP-IDF but it seems like I am not able to get the speaker to work.

First off, I wanted to make sure if the code is alright, so I recreated the circuit (or let's say the important parts of the circuit) on a breadboard with ESP32-S3-DevkitC at its core, and seems like the speaker works just fine when using the wired up circuit. But when I flash the same binary onto the custom PC, for some reason the speaker does not play the sound. I noticed that there is a variable defined without any documentation in the original firmware called SPEAKER_EN, and it is set to output high, but never used. Does this have anything to do with the speaker not playing what's been written on I2S TX channel when the PCB is used?

Out of 5 boards, one seems numb, potential PCB issue debugging

Hi everyone,

First of all AWESOME PROJECT. I got me boards from pcbway and installing the first device was just pure satisfaction, so easy to get it setup and it instantly worked nicely on my homeassistant setup (using the esphome config)

Then I installed a second one, also working great, but the third one I tried seems numb. I can make it work when I am really close to it and speak loudly, but else it is strange. Do you have any points what to look for on the PCB when debugging ? I would love to not have to throw one out...

So if there is any voltages I can check, components that might be misplaced or anything else you can point me to please let me know. I am kinda used to do PCB debugging and changing out smd components, so that wont be an issue, but any tipps will save me time ;-)

I will still keep posting here in case anyone runs into the same issue as well

PCB check

PCBway asked me to verify the component orientation is the attached pictures. Since I have no clue, I'd appreciate help from someone who has working boards. Thanks!
pleasecheck.zip