Simple Client Recording Attempt #39

justinlevi · 2023-08-21T16:28:11Z

I start up the server via $ python ./run_server.py

(whisper_live)  whisperlive git:(main)✗  🚀 python ./run_server.py
Downloading: "https://github.com/snakers4/silero-vad/archive/master.zip" to /Users/justinwinter/.cache/torch/hub/master.zip
2023-08-21 12:14:34.119619 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '628'. It is not used by any node and should be removed from the model.
2023-08-21 12:14:34.119647 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '629'. It is not used by any node and should be removed from the model.
2023-08-21 12:14:34.119652 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '623'. It is not used by any node and should be removed from the model.
2023-08-21 12:14:34.119655 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '625'. It is not used by any node and should be removed from the model.
2023-08-21 12:14:34.119659 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '620'. It is not used by any node and should be removed from the model.
2023-08-21 12:14:34.119696 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '139'. It is not used by any node and should be removed from the model.
2023-08-21 12:14:34.119701 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '131'. It is not used by any node and should be removed from the model.
2023-08-21 12:14:34.119704 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '140'. It is not used by any node and should be removed from the model.
2023-08-21 12:14:34.119708 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '134'. It is not used by any node and should be removed from the model.
2023-08-21 12:14:34.119711 [W:onnxruntime:, graph.cc:3543 CleanUnusedInitializersAndNodeArgs] Removing initializer '136'. It is not used by any node and should be removed from the model.
ERROR:root:no close frame received or sent

Then start up the client via:

(whisper_live)  whisperlive git:(main)✗  🚀 python ./run_client.py
[INFO]: * recording
[INFO]: Waiting for server ready ...
False en transcribe
[INFO]: Opened connection
[INFO]: Server Ready!
Traceback (most recent call last):
  File "/Users/justinwinter/projects/whisperlive/./run_client.py", line 3, in <module>
    client()
  File "/Users/justinwinter/projects/whisperlive/whisper_live/client.py", line 298, in __call__
    self.client.record()
  File "/Users/justinwinter/projects/whisperlive/whisper_live/client.py", line 234, in record
    data = self.stream.read(self.CHUNK)
  File "/opt/homebrew/Caskroom/miniconda/base/envs/whisper_live/lib/python3.9/site-packages/pyaudio/__init__.py", line 570, in read
    return pa.read_stream(self._stream, num_frames,
OSError: [Errno -9981] Input overflowed

// run_client.py

from whisper_live.client import TranscriptionClient
client = TranscriptionClient("0.0.0.0", "8080", is_multilingual=False, lang="en", translate=False)
client()

The text was updated successfully, but these errors were encountered:

zoq · 2023-08-22T16:35:40Z

Are you running this on a mac?

zoq · 2023-08-22T16:49:10Z

A temporary workaround is to set exception_on_overflow=False in https://github.com/collabora/WhisperLive/blob/main/whisper_live/client.py#L234.

This might cause that we skip some frames, we are looking into updating the frame rate for the different platforms.

justinlevi · 2023-08-22T18:13:19Z

@zoq Thanks for the idea. Yes, I am on a mac. I tried setting the exception_on_overflow=False but still getting the same error:

(whisper_live)  whisperlive git:(main)✗  🚀 python ./run_client.py
[INFO]: * recording
[INFO]: Waiting for server ready ...
False en transcribe
[INFO]: Opened connection
[INFO]: Server Ready!
Traceback (most recent call last):
  File "/Users/justinwinter/projects/whisperlive/./run_client.py", line 3, in <module>
    client()
  File "/Users/justinwinter/projects/whisperlive/whisper_live/client.py", line 299, in __call__
    self.client.record()
  File "/Users/justinwinter/projects/whisperlive/whisper_live/client.py", line 235, in record
    data = self.stream.read(self.CHUNK)
  File "/opt/homebrew/Caskroom/miniconda/base/envs/whisper_live/lib/python3.9/site-packages/pyaudio/__init__.py", line 570, in read
    return pa.read_stream(self._stream, num_frames,
OSError: [Errno -9981] Input overflowed


    def record(self, out_file="output_recording.wav"):
        n_audio_file = 0
        # create dir for saving audio chunks
        if not os.path.exists("chunks"):
            os.makedirs("chunks", exist_ok=True)
        try:
            for _ in range(0, int(self.RATE / self.CHUNK * self.RECORD_SECONDS)):
                if not Client.RECORDING: break
                self.exception_on_overflow=False
                data = self.stream.read(self.CHUNK)
                self.frames += data

                audio_array = Client.bytes_to_float_array(data)
                
                self.send_packet_to_server(audio_array.tobytes())

                # save frames if more than a minute
                if len(self.frames) > 60*self.RATE:
                    t = threading.Thread(
                        target=self.write_audio_frames_to_file,
                        args=(self.frames[:], f"chunks/{n_audio_file}.wav", )
                    )
                    t.start()
                    n_audio_file += 1
                    self.frames = b""

zoq · 2023-08-22T20:50:53Z

Okay, I can reproduce the issue on a mac, I'll come up with a fix.

aavetis · 2023-11-09T21:33:03Z

welp

Geczy · 2023-11-12T07:58:41Z

rip?

zoq · 2023-11-12T18:02:38Z

welp

Do you still have this problem with the latest release?

sbrnaderi · 2023-11-18T11:41:35Z

I have pulled the latest changes from the git repo and I still have this issue on my mac. Have you made changes regarding this issue?
Thanks.

sbrnaderi · 2023-11-18T11:58:53Z

OK, I just tried to increase the self.chuck value to 1024 * 4 and I don't get the error anymore and the transcription works fine.
This is in client.py file.

zoq · 2023-11-18T19:37:16Z

This is on a mac?

sbrnaderi · 2023-11-18T20:59:25Z

This is on a mac?

Yes, this is on MacBook Pro (intel based). I increased the chunk size and I could get the code to work. I also noticed that if I use the bigger Whisper model (medium), then I have to increase this further to 1024 * 8.

arbianqx · 2023-12-14T15:43:07Z

This issue still persists on macbook. Tried sending an audio file, is working just fine.

zoq · 2023-12-14T15:48:12Z

Looking into it today.

niderhoff · 2023-12-18T13:25:58Z

OK, I just tried to increase the self.chuck value to 1024 * 4 and I don't get the error anymore and the transcription works fine. This is in client.py file.

tried that but for me it still crashed with the original error.

zoq · 2023-12-18T20:01:05Z

I can confirm if I update the chunk size it works. @niderhoff let try to figure out why it's not working on your system. Just to make sure our setup is the same. You are using the pip package, and not the docker container, or do you run the scripts without the pip package?

asadal · 2023-12-24T22:06:26Z

I have the same problem.

[INFO]: * recording
[INFO]: Waiting for server ready ...
True ko transcribe
[INFO]: Opened connection
[INFO]: Server Ready!
Traceback (most recent call last):
  File "/Users/asadal/Documents/Dev/Hani/WhisperLive_streamlit.py", line 13, in <module>
    client()
  File "/Users/asadal/miniconda3/lib/python3.10/site-packages/whisper_live/client.py", line 490, in __call__
    self.client.record()
  File "/Users/asadal/miniconda3/lib/python3.10/site-packages/whisper_live/client.py", line 371, in record
    data = self.stream.read(self.chunk)
  File "/Users/asadal/miniconda3/lib/python3.10/site-packages/pyaudio/__init__.py", line 570, in read
    return pa.read_stream(self._stream, num_frames,
OSError: [Errno -9981] Input overflowed

Macbook Pro 14 M1 Pro. Simple client Recording.
I used pip package.

# Run the client
from whisper_live.client import TranscriptionClient
client = TranscriptionClient(
  "localhost",
  9090,
  is_multilingual=True,
  lang="ko",
  translate=False,
  model_size="small"
)
client()

Then, I encountered an error,

TypeError: TranscriptionClient.__init__() got an unexpected keyword argument 'model_size'

So, I disabled the option, model_size="small" and ran again.
But an error occured, OSError: [Errno -9981] Input overflowed

I changed self.chunk = 1024 to self.chunk = 1024 * 4. But encounterd same error.

zoq · 2023-12-24T23:41:14Z

We have to release the latest pip package, in the meantime you can remove model_size="small" from the TranscriptionClient call. For the overflow issue, can you try stream.read(self.chunk, exception_on_overflow=False) in https://github.com/collabora/WhisperLive/blob/main/whisper_live/client.py#L415

asadal · 2023-12-24T23:56:20Z

Thanks zoq,
I'll wait for update pip.
Thank you for creating such a great application.

Best Regards.

makaveli10 · 2024-01-02T05:11:01Z

@asadal pip package is updated. Let us know if you are still facing the issue.

kjyv · 2024-01-02T12:37:31Z

I have had success with the newest version and setting exception_on_overflow to False. Can this be set by default?

zoq · 2024-01-02T13:30:39Z

#83 does that, we will merge it and release a new version

JonathanLehner · 2024-01-12T22:00:20Z

I got segmentation fault with the latest version...

makaveli10 · 2024-01-19T05:37:06Z

@JonathanLehner can you share more details when do you see the segfault?and does it happen always on the latest version?

JonathanLehner · 2024-01-20T01:15:07Z

I just tried the demo from the Readme:

llm_server.py
from whisper_live.server import TranscriptionServer
server = TranscriptionServer()
server.run("0.0.0.0", 8080)

llm_client.py
from whisper_live.client import TranscriptionClient
client = TranscriptionClient(
"localhost",
8080,
is_multilingual=True,
lang="en",
translate=False,
#model_size="tiny"
)

client()
#client("audio_test.wav")

python llm_server.py
zsh: segmentation fault python llm_server.py
(physiotherapy) jonathan@Jonathans-MBP physiotherapy % /usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Simple Client Recording Attempt #39

Simple Client Recording Attempt #39

justinlevi commented Aug 21, 2023

zoq commented Aug 22, 2023

zoq commented Aug 22, 2023

justinlevi commented Aug 22, 2023

zoq commented Aug 22, 2023

aavetis commented Nov 9, 2023

Geczy commented Nov 12, 2023

zoq commented Nov 12, 2023

sbrnaderi commented Nov 18, 2023

sbrnaderi commented Nov 18, 2023

zoq commented Nov 18, 2023

sbrnaderi commented Nov 18, 2023

arbianqx commented Dec 14, 2023

zoq commented Dec 14, 2023

niderhoff commented Dec 18, 2023

zoq commented Dec 18, 2023

asadal commented Dec 24, 2023 •

edited

Loading

zoq commented Dec 24, 2023

asadal commented Dec 24, 2023

makaveli10 commented Jan 2, 2024

kjyv commented Jan 2, 2024 •

edited

Loading

zoq commented Jan 2, 2024

JonathanLehner commented Jan 12, 2024

makaveli10 commented Jan 19, 2024

JonathanLehner commented Jan 20, 2024

Simple Client Recording Attempt #39

Simple Client Recording Attempt #39

Comments

justinlevi commented Aug 21, 2023

zoq commented Aug 22, 2023

zoq commented Aug 22, 2023

justinlevi commented Aug 22, 2023

zoq commented Aug 22, 2023

aavetis commented Nov 9, 2023

Geczy commented Nov 12, 2023

zoq commented Nov 12, 2023

sbrnaderi commented Nov 18, 2023

sbrnaderi commented Nov 18, 2023

zoq commented Nov 18, 2023

sbrnaderi commented Nov 18, 2023

arbianqx commented Dec 14, 2023

zoq commented Dec 14, 2023

niderhoff commented Dec 18, 2023

zoq commented Dec 18, 2023

asadal commented Dec 24, 2023 • edited Loading

zoq commented Dec 24, 2023

asadal commented Dec 24, 2023

makaveli10 commented Jan 2, 2024

kjyv commented Jan 2, 2024 • edited Loading

zoq commented Jan 2, 2024

JonathanLehner commented Jan 12, 2024

makaveli10 commented Jan 19, 2024

JonathanLehner commented Jan 20, 2024

asadal commented Dec 24, 2023 •

edited

Loading

kjyv commented Jan 2, 2024 •

edited

Loading