This is one of the most discussed areas on our Discord channel so I thought I would dedicate a page to explaining and helping find the best working solution.

One of the big differences between VACA with View Assist and commercial smart home assistants, is that you can install VACA on any android device with Android 8+ installed. This allows great flexibility to find the perfect device you your environment and use cases. However, it can bring its own challenges. The main being the varying quality of the microphone and speaker hardware and the Android builds' echo and noise cancelling capabilities. As such, it can be difficult and frustrating to get good wake word and command detection on all devices.

We have invested much time to find the best answer to this and this is discussed further down. But first, a little background on how all this works.

There are 2 key elements to the audio for voice assistants...

Wake Word Detection

Built into the VACA app is an android version of OpenWakeWord by @dscripka. This uses a machine learning engine to detect the specific wake word when heard by the device microphone. OpenWakeWord uses a model for each wake word and these have been trained to different degrees and therefore work at varying levels depending on mic noise, background noise, accent etc.

We find Hey Jarvis to be the best overall but others also work well for some and less well for others. Do experiment to find the one that best suits you and works.

Note

The wake word detection engine is always listening to the audio from the mic, listening for that wake word. No audio is sent outside the device during this stage.

Voice Command Interpretation

Once the wake work has been detected, VACA starts a voice pipeline within Home Assistant and streams the mic audio to your Home Assitant server. This utilises the configuration you have in the Voice Assistant config in HA. Captured audio is passed to the STT (Speech To Text) engine, which in turn converts this to text to pass to your Conversation Engine (Home Assistant, AI LLM etc).

The different options available here for the STT engine, will produce different results. Currently we feel that HA Could is the best option (but it is subscription based) but local options like Whisper or Speech-To-Phrase do work but require more setup.

Any issues in this area, you can look at the STT sensor in the VACA device in Home Assistant to see what was heard and interpeted. Other than VACA providing the audio stream, this is all a Home Assistant action.

Getting the best setup

In VACA, we try to make the setup experience quick and simple to lower the barrier to entry that previously existed in creating you own voice assistant. We have spent a considerable amount of time experimenting with code and settings to get the best wake word detection and voice command interpretation we can.

In v0.8.x and prior, we added no gain to the wake word detection audio but then when streaming the audio to HA, we added an auto gain to improve translation. This generally works well but has caused a lot of confussion when enabling the diagnostic overlay and seeing virtually no audio response when trying to get the wake word to detect but then a good audio level when streaming. This configuration works well on most devices but not all.

In v0.9.0, we have added a couple of experimental options (enabled in the diagnostic section in the HA device settings). This allows you to play around with some config options to improve detection and interpretation for your specific devices and environment.

Advanced gain control

Contrary to the described way of working above, this applies the same gain to all audio, whether listening for wake word or streaming for the command. Now, when looking at the diagnostic overlay, you will see a consistent audio level. This gain level can also be adjusted by the gain slider.

Voice enhancer

In Android, there are different mic stream inputs that are tuned for different purposes. In general we use the standard raw mic input. However, on many devices there is an option to choose an input that is specifically tuned for Voice Recognition. The caveat here is that the benefit of this is completely dependant on the OS build and can have no effect or can have a good effect.

Important

Please note, turning these on can reduce the performance on some devices, so you will need to experiment. We would recommend enabling Voice Enhancer first and then the Advanced gain to see if that improves detection for you.