Text-to-Speech Audio Issue in Python

General Help
1

I have installed mycroft-mimic3-tts (the python module) to my system, the purpose being to convert text reponses from a python AI bot into audio. Although the program functions and returns text, no audio is heard. Does anyone know mycroft-mimic3-tts works in relation to Zorin OS 16? My friend who runs Fedora 38 has it working fine. When I run the program it does return an error, I will post it:

Reading text from stdin...
INFO:mimic3_tts.tts:Loaded voice from /usr/share/mycroft/mimic3/voices/en_UK/apope_low
ALSA lib conf.c:3725:(snd_config_hooks_call) Cannot open shared library libasound_module_conf_pulse.so (libnsl.so.2: cannot open shared object file: No such file or directory)
ALSA lib pcm.c:2660:(snd_pcm_open_noupdate) Unknown PCM default
aplay: main:852: audio open error: No such file or directory
Traceback (most recent call last):
  File "mimic3.py", line 40, in <module>
  File "mimic3_tts/__main__.py", line 129, in main
  File "mimic3_tts/__main__.py", line 481, in process_lines
  File "mimic3_tts/__main__.py", line 512, in play_wav_bytes
  File "subprocess.py", line 424, in check_output
  File "subprocess.py", line 528, in run
subprocess.CalledProcessError: Command '['aplay', '-q', '/tmp/tmp1hfvmzu5.wav']' returned non-zero exit status 1.
[161555] Failed to execute script 'mimic3' due to unhandled exception!

This is the error that appears when I navigate to /usr/bin and type the following into my terminal:
sudo mimic3 "hello world"
If working properly, a voice should have repeated "hello world" in an AI voice, but it did not.

Any assistance is appreciated!
Thanks in advance.
-Joshua

2

Can you try installing the libasound2-plugins:
http://archive.ubuntu.com/ubuntu/pool/main/a/alsa-plugins/libasound2-plugins_1.2.2-1ubuntu1_amd64.deb

Then test?

3

Thanks for the recommendation! I downloaded the deb package and opened it with software. It looked like it was already installed. Should I uninstall and reinstall it from the deb package?

4

If it is already installed, then reinstall is probably not necessary.

Maybe you need to redirect a symlink to the shared object file:

ln -s /usr/lib/libnsl.so.3 /usr/lib/libnsl.so.2

5

Okay, I typed the following into my terminal window:
sudo ln -s /usr/lib/libnsl.so.3 /usr/lib/libnsl.so.2
No errors were returned to the terminal window, so I assume it successfully redirected the symlink. When I tried the program again, there was still no voice, so unfortunately it's seemingly another dead end.

6

Have you installed all dependencies?

7

I found a list of dependencies for mycroft-mimic3-tts, and I was able to install most of them, but the following are so-called dependencies that my terminal did not recognize as being valid packages:

  • kaldi
  • slt_dynamic_augmentation
  • slt_interactive_shell
  • slt_knowledge_intensive
  • slt_lexical_semantics
  • slt_phonetic_phenomena
  • slt_pronunciation
  • slt_speech_acts
  • slt_speech_generation
  • slt_speech_recognition
  • slt_speech_synthesis
  • slt_text_analysis
  • slt_text_generation
  • slt_text_processing
  • slt_text_summarization
  • slt_topic_modeling
  • slt_utterance_generation

Is it likely that the above are already installed? Seems like this sort of thing should be installed already with the python module.

8

Unfortunately no
Could you provide the terminal output when you try to install them

Also apologies as I won't be replying for sometime, It's 1:30 here, we can work on this tomorrow

9

No problem. Here is the output:

<username>@<username/system>:~$ sudo apt install slt_text_processing
[sudo] password for <username>:       
Reading package lists... Done
Building dependency tree       
Reading state information... Done
E: Unable to locate package slt_text_processing

Just the average 'package not found' error that comes up when it cannot find the software in the repository.

10

I followed the documentation here (I suggest you go through it once for hardware/software requirements, If you haven't already)and was able to get it working on Zorin, so the good news is it works.

I used the pip install method (verified working), but I believe
both .deb method or building from source should also work (avoid the first (plugin method)).

Can you uninstall everything mimic related (just to be safe) and then reinstall using either the

pip install mycroft-mimic3-tts[all]

Or any other method and relay any issues that occurs

[again not the plugin mycroft-pip install mycroft-plugin-tts-mimic3[all] ]

also you neither need sudo nor you need to navigate to usr/bin (always avoid using terminal in root directory if you can). You can run

mimic3 "Hello World"

perfectly from home directory.(Tested)
I also tested some additional flags and everything seems to be working fine

1 Like
11

This is insightful. When I get a chance I'll give it a try and post the results. Just to be clear, you are suggesting that I install the following using pip3?
pip3 install mycroft-mimic3-tts[all]
and uninstall the plugins deb package, depending on the above command instead to install them, or were you suggesting to leave the plugins installed but get rid of other package dependencies?
Also, the link you gave for the documentation isn't valid, it returns a 'Page not found' error.
Thanks!

12

Sorry, here is the corrected one

As for your question, I am sorry but it's a bit unclear to me,
Here

What I am suggesting is that you remove everything (just in case) that you previously installed(basically get the system to a state,that you had, previousto trying to install mimic3), while trying to install mimic3 then run the pip3 install...

1 Like
13

Awesome thanks!

OK, I looked in by bash_history file to see what commands I'd installed, then did my best to remove all packages I installed that were dependencies for mimic3. Then I did

pip3 install mycroft-mimic3-tts[all]

It came up with an error that is different than what I got before, I'll post it:

Reading text from stdin...
Traceback (most recent call last):
  File "/home/username/.local/bin/mimic3", line 8, in <module>
    sys.exit(main())
  File "/home/username/.local/lib/python3.8/site-packages/mimic3_tts/__main__.py", line 129, in main
    process_lines(state)
  File "/home/username/.local/lib/python3.8/site-packages/mimic3_tts/__main__.py", line 450, in process_lines
    process_line(line, state, line_id=line_id, line_voice=line_voice)
  File "/home/username/.local/lib/python3.8/site-packages/mimic3_tts/__main__.py", line 379, in process_line
    state.tts.speak_text(line)
  File "/home/username/.local/lib/python3.8/site-packages/mimic3_tts/tts.py", line 368, in speak_text
    voice = self._get_or_load_voice(self.voice)
  File "/home/username/.local/lib/python3.8/site-packages/mimic3_tts/tts.py", line 595, in _get_or_load_voice
    voice = Mimic3Voice.load_from_directory(
  File "/home/username/.local/lib/python3.8/site-packages/mimic3_tts/voice.py", line 283, in load_from_directory
    onnx_model = Mimic3Voice._load_model(
  File "/home/username/.local/lib/python3.8/site-packages/mimic3_tts/voice.py", line 403, in _load_model
    onnx_model = onnxruntime.InferenceSession(
  File "/home/username/.local/lib/python3.8/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 432, in __init__
    raise e
  File "/home/username/.local/lib/python3.8/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 419, in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
  File "/home/username/.local/lib/python3.8/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 451, in _create_inference_session
    raise ValueError(
ValueError: This ORT build has ['AzureExecutionProvider', 'CPUExecutionProvider'] enabled. Since ORT 1.9, you are required to explicitly set the providers parameter when instantiating InferenceSession. For example, onnxruntime.InferenceSession(..., providers=['AzureExecutionProvider', 'CPUExecutionProvider'], ...)
14

Does This mean you were not able to remove all of the things ?
If yes That would be the most likely culprit.
I suspect(and correct me if I am wrong)
But you previously installed the plugin for mimic3 which needs the complete mycroft ai package to work
My speculation is based on this thread where a user using the complete AI package reports a similar issue on Ubuntu 20.04 (This is what Zorin16.2 is based on)
And since I have tested the mimic3 independent package on zorin, It's likely you still have mycroftai leftovers creating issues.

You don't need to install anything as mycroft has packaged mimic as a complete package with all dependencies.
To summarize
Uninstall everything mimic related (if there were any leftover previously)
Uninstall the dependencies you installed later,before running the pip command
then run

pip3 install mycroft-mimic3-tts[all]

Additionally
If you haven't already I suggest asking this question on Mycroft forum as well, Where people more experienced in with mycroft's system might provider quicker solutions

15

Thanks for the info. I'm actually logged in now on a different Zorin OS 16.3 that I just installed. All I have done is installed python3-pip then I ran:

pip3 install mycroft-mimic3-tts[all]

It went through a lengthy installation that included some warnings and errors, and when it was done I ran

mimic3 "Hello World"

and it simply said that the command 'mimic3' was not found! This no longer makes any sense to me. Unless anyone else has an idea, I'll probably try and make do with a different TTS python module :wink:
Anyway, thanks @Aravisian and @Kayjzjzk for your help!

16

Agreed.

I just tested this using:

The only dpendency I needed out of:

sudo apt-get install gcc make pkg-config automake libtool libasound2-dev

was libasound2-dev

I then scrolled down to the Build Section and installed. Everything Itested out worked.
@Kayjzjzk says that it also worked in testing.
I think that from this, we can establish that Zorin OS does not stand out in some way.
It's usually something simple that makes you facepalm when you find it.

Given that two participants had a successful install on the first try; Yet, you ran into trouble on two separate machines, I wonder if you have installed or configured something along the way that is interfering somehow.

If this was me...
I would (tedious, I know) start with

sudo apt clean && sudo apt autoremove

Then from the github link I just posted, examine each step as you apply them.
By mulling over each step which we know works, it may provide that "aha" moment. I cannot count how many times I tried a software option, then decided I preferred a different software... But ran into trouble on the different software installation due to what I had done trying out the first software option.
And completely forgot about since it wasn't really memorable. At the time, I did not think it would be an issue later...

17

Thank you for the advice! This gives me hope :smile: Just out of curiosity, what exactly does "sudo apt clean" and "sudo apt autoremove" do exactly? Does it uninstall all installed applications, including python modules?

18

No, not at all.
sudo apt clean clears out the apt Cache folder, removing any cached files for install. Doing this helps prevent an install command from installing what is cached, rather than the specific version you targeted. sudo apt autoremove removes (Uninstalls) the automatically installed dependencies that were installed along with other or previous software but no longer needed due to that software also being removed or having already been removed.
I couldn't figure out how to make that sentence shorter.

19

I tried what you suggested above, and it does work on my other machine!
Installing mycroft-mimic3-tts[all] then running:

sudo apt-get install gcc make pkg-config automake libtool libasound2-dev

Works just fine, no errors! Awesome, I have a handle on things now :smiley:
I did not actually have to walk through the setup on the github link, but I will do that on my other machine if it's not as simple to get mimic3 working there.
I really appreciate all of your help, thanks again!

2 Likes
20

Transkriptor's API can also be useful for audio to text conversion. You can also try text to voice.