-
-
Notifications
You must be signed in to change notification settings - Fork 10.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Forward audio #14
Comments
Adding this would really making streaming games would so much better. So far all the other existing solutions kinda suck. |
Is it even possible without a custom rom? |
I just implemented a PoC (separated of scrcpy) to enable audio forwarding from the device to the computer: aoa-audio. On Debian stable, it "works": the device sound is played on the computer. This is quite manual though, and several issues must be solved before something similar could be implemented in scrcpy. Any help is welcome 😉 |
May it be possible to use AudioRecord api server side? |
|
I may be wrong but here is an example of what I thought: Sorry I should have added this on my previous comment. |
Oh, you're right, I missed the |
Arf, just calling Note that |
Yes... :( Audio support is really tricky :/ |
Even from an app, the source
(while it works with |
Hi, I would like to share with you my work-in-progress branch: I implemented it, and it works fine for me! 🎉 I would like early feedbacks so that any major issue can be fixed. Requirements
How toInstall libusb:
Checkout the
Use a debug build to report problems:
Run with audio forwarding enabled:
The audio should be forwarded. When closed, the audio forwarding may not be totally disabled. It may require to restart the current audio track (in VLC for example). If it is not sufficient, unplug the device. At worst (but it should not be necessary), disable/enable USB debugging. How it worksScrcpy connects to the device over USB to send AOA requests so that the computer becomes an "USB audio accessory" for the device. This creates a new audio input source on the computer (which can be seen in PulseAudio for example). Scrcpy records this input source and immediately plays it to the default output source. Thank you for your feedbacks. |
In theory, it should now also work on Windows (in MSYS2/mingw64), with the
However, when it try to communicate to open the device over USB on Windows,
I have no idea how to get the permissions granted. Any ideas? |
On macOS you can install libusb with the following command: On a MBP 15, macOS 10.13.3 it is not working. I start the app while a podcast is playing on the device. This behaviour is 100% reproducible on my side. Here are the command line output:
|
Thank you for the test 👍
It did not select the right input source, either because none were created, either because for now it just selects the last one, assuming it's the most recent (so it should probably be the input source from the device). I'll check that. Thank you. |
For your information I've done 2 tests one with a headset connected to the computer and another one without any headset plugged in. Good luck |
@eyal-lezmy I investigated on Mac, and it appears that enabling "audio accessory" on the device does not create a new audio input source on the computer. So unfortunately, I think that the feature will be Linux-only… I updated the branch to reject the input source if its name does not contain the device model (so that it does not open the built-in microphone for example). |
Ok so heres a wholly stupid question. Would it work via the Linux environment in windows 10? |
You tell me 😉 |
What is the plan of merging this excellent feature into the main branch? |
Did you test it? Does it work correctly for you? There are several problems:
So for now, I don't merge it. Maybe I could, mentionning that it is very experimental... |
Hi, thanks for the package! I've been enjoying it a lot On my Archlinux machine, I have both libusb and libusb-compat installed besides the dependencies already listed in the AUR package for scrcpy. The testing scrcpy pacakge was compiled with the pre-built server.jar. When attempting to start scrcpy audio forwarding with my Nokia6.1 (Android 8.1, July 1st 2018 patch), the program exits with the following error message:
Without the --forward-audio flag, the program runs fine. |
@yimuchen Could you test manually with pactl and ffplay:
|
@rom1v I'm a bit new to writing this sort of code, Can you help me with how I should get the string for my device, the The output of pactl is:
On a side note, as a test, I edited the |
Sorry, I copied incomplete instructions I kept in some textfile. Instead, follow the steps I wrote in the README of aoa-audio. |
@rom1v Thanks for instructions! I think the issue that I have is that the udev rules for my phone isn't available yet. The vendor/product ID under |
for Bluetooth audio from android to windows pc, unfortunately its not available outside the MS store |
@sapkalrohan Huh Is that not built-in? Last I checked pairing a phone to Windows Bluetooth would automatically connect the A2DP sink. |
Hi, I have a solution there's an app available on windows store as Bluetooth audio receiver @parkerlreed This app uses A2DP protocol as used in Wireless buds to transfer audio to pc and it's seamless have a try |
Similar to #1887 (comment), by creating a fake (Most code is from Sndcpy, I only did two small modifications) As mentioned above, this API requires I still prefer integrating audio forwarding in Scrcpy, it requires no software setup, no Bluetooth hardware, and is portable on all platforms. The usage is same as Sndcpy, VLC player is required to play the stream. Integration with Scrcpy client to be done (I'm not familiar with C and FFMpeg). adb forward tcp:28200 localabstract:sndcpy
# start Scrcpy here
vlc -Idummy --demux rawaud --network-caching=0 --play-and-exit tcp://localhost:28200 Pre-compiled server binary: |
Absolutely awesome. 🎉
Oh, I didn't know that it worked since Android 11. I was seriously considering adding an option to install the app (#3517), and investigated some possible architecture to make audio (from the app) and video+controls (from the server) work correctly, but it adds a lot of complexity. It it can work that way, I'll definitely add it in scrcpy (when I have more time though). I have already worked on some code to properly capture and encode to OPUS (rather than using raw audio), I will try to use your magic-Context stuff :) |
@yume-chan How do I use this? I downloaded the scrcpy-server file and replaced it in the scrcpy 1.25 folder.
|
If you are on Android 11, I checked Android source code, there is no this |
Yes, I'm on android 11, |
The screen sharing part is working fine, but the audio is not being transmitted for some reason on the android 13 device.
|
nvm figured it out:
That's quite a lot of commands though, Thanks |
In Android 11, only the foreground App can start recording audio. This check was removed in Android 12, so my method works. (Or maybe Android 12 didn't check App Ops permission at all) The only way I found to work around this foreground check, is by displaying this "Share heap dump" dialog. This is actually an activity in The complete process is:
(These can all be done in Scrcpy client) I don't really like it. EDIT: Anyway, there is another pre-compiled server binary including the fake Context patch for Android 11. To use it, execute EDIT 2: Forgot to mention, another method to bypass the foreground App check is by running ADB as root. |
Please test #3757 🚀 |
I added binaries for Windows if you want to test: #3757 |
Tested binary on 64bit Windows. Worked flawlessly on Android 12 Samsung device! There's slight delay in audio and video (video lags slightly which kind of irks for sometime when you observe lip-sync) but good nevertheless. It probably is because I was on wifi adb. |
Implemented in scrcpy v2.0 🚀 |
How should I make audio forwarding disabled by default? |
Run |
How do I play audio on both the headset and the receiver? I want the beatsaber audio to be perfectly lined up when playing, but also to be able to stream it. |
Not possible (for now?), see #3875. |
It must be possible via some set of configurations on the host device; the oculus does this by default for all of it's streaming. Is there a way to "record" device audio and play it back on the second device, maybe if there's something additional installed on the Android side? |
Sorry, but how does it work on scrcpy 2.0. I tried connecting via USB, the phone shows, visually speaking, but any sound from the phone, happens on the phone, not on the PC. The PC is Ubuntu 23.04 |
What is the console output when you run scrcpy? What is your Android version? |
scrcpy 2.0 https://github.com/Genymobile/scrcpy @rom1v Sorry forgot about that. Thank you. |
What is the result of
For example, if you play a video on your phone with vlc (or any other player), the sound is played on the phone? Note that notifications or phone calls are not forwarded. |
scrcpy 2.0 https://github.com/Genymobile/scrcpy You helped me solve the mystery. I was trying to get phone calls on the PC. For videos I can literally hear them on the PC which IS AMAZING!! This is like a new level for me. I am guessing on version 3.0 they might be able to include phone calls so I can do full productivity on the PC without the phone. |
Directing audio to the client would be pretty neat.
Currently you can accomplish the same thing by plugging a 3.5mm cable into the mic or line-in on the computer. Some manufacturers are dropping headphone ports though.
Suggestion by @rom1v: The best solution currently is to connect the device via bluetooth to the computer. It works for all apps, it does not suffer from additional limitations and works for basically all devices.
--> There is work in progress for Android >= 11. Please test #3757 🚀
The text was updated successfully, but these errors were encountered: