This is way better than it used to be (I measured half a second latency in 2012: https://www.jefftk.com/p/android-sound-synthesis-and-latency) but it is still too high to let you use an Android device as a musical instrument. This is something iPhones have always been able to do, and one of my biggest disappointments with Android.
(Disclosure: I work for Google, speaking only for myself)
I agree audio at minimum should have been done differently on Android.
The latency(different type I believe) of Spotify applications and even YouTube applications is pretty bad compared to an Apple device. Sometimes the audio just fails to respond for half a second or so. This is not an interface bug from my experience.
What are the equivalent numbers for iPhone? Touch to audio and round-trip times?
This data [0] doesn't have the latest models, and it's meassuring latency internaly, not "touch-to-sound". The ones they tested average around 7-8ms. I've heard of <10ms as the benchmark for "instant" in audio.
Edit: I've been thinking about this and a touch-to-tone latency below 10ms seems imposible, even with the 120hz digitizer on newer iPhones. I would be very interested in seeing some experimental data as well.
[0] https://juce.com/discover/stories/Mobile%20performance%20ind...
The best I could find was https://developer.apple.com/forums/thread/71301 which suggests a 32ms touch latency. I think that might include some processing, but it doesn't seem much better than the 20ms latency in Android mentioned in the article.
The audio processing latency seems to be single digit on both systems.
In any case, I don't think these differences are big enough to argue that "Android can't be used for instruments".
Xiaomi has hit 480hz touch sampling with the Mi 11. It looks possible in hardware and in software, someone just has to manage to perfect both in the same phone
https://superpowered.com/latency has messy data, but it looks like typically single-digit latency.
I think typical newish (2014+) devices have between 5-9ms latency. Having core audio from the start was a huge leg up for iOS for sure. I am mostly platform agnostic unless I am working with audio where I pretty much only use Apple products.
The article and all the responses here express "latency" as a simple scalar number. Is it actually more like a histogram of values than a scalar, and the marketing number is the 90th %ile of that histogram?
My impression is that Android has more latency variance than iOS. If that's true then it's important to be clear what the reported numbers represent.
If you open the audio context you should see consistent latency as long as you keep it open, but each time you open a new audio context you might end up with different latency. You should not see jitter within a single context.
Latency variance is known as jitter. What gives you the impression that it would be high on Android?
Having used nearly every flagship android device from the HTC Dream up through the Nexus 6p, and then switching to an iphone 6 and never looking back, except watching acquaintances use their android phones.
Also from talking to an engineer who worked on earlier generations of iphones, who said that every team had a latency budget that they could not exceed.
On the 3D side it is similarly disappointing, as Google never put the effort of proper frameworks similar to Apple's kits, and SceneForm on Google's tradition is now deprecated after VR on Android lost its marketing drive.
If android can get its audio latency act together there is some real opportunity here to steal market share from Apple. Since the release of the 2nd iPad Pro Apple has forced audio processing onto the slower low power CPU core, this means that for the last three years I can run all my iOS synth applications at 96kHZ with a low buffer sample size without any drop outs on my first gen iPad Pro and an iPad mini 3. What I cannot do is get similar performance on last year's iPad Pro. It is absolute insanity that Apple has no work around to direct audio processing to the high powered cores for apps made for musicians. Presumably at some point they will allow you to do this with a documented programing call but it has been three years of bad performance on synth applications due to the necessity to have high sample buffers (ie high latency).
I've noticed an increase in measured latencies, but didn't know about that slower CPU core thing.
Sad to see Apple take such a direction - musicians appreciated having proper hardware/software and Apple definitely made money on all the paid content. Why let this arrangement fall by the wayside?
The 10x engineers have either left Apple or were reorganized to gimmick projects like SwiftUI.
Is there more details about this published somewhere? Iām very curious to learn more
> Since the release of the 2nd iPad Pro Apple has forced audio processing onto the slower low power CPU core
What makes you believe this?
On Android is even worse, specially if one gets an average model.
Responsiveness and low input-to-result latency was why I chose iPhone over android 10 years ago even though I prefer a more open environment like android. Sounds like things havenāt changed a whole lot on that front.
Sometimes I wonder if these increased levels of abstraction and even digitization itself (or at least the Internet-ization of things) is a mistake for human-machine interface. Watching drone racing pilots use analog controls and analog video headsets in spite of the potato resolution because WiFi/digital/etc just add too much latency is kind of eye opening and makes you sort of reconsider the last couple decades. (Improvements have been made so digital isnāt so high latency and VR has served to address a lot of these latency concerns, but itās still kind of interesting...)
What's funny is that this āopenā environment of android now is nearly the opposite.
With any android device openly spying on you while locking down all the open source apps you might want to install.
What does this mean? I can choose not to install google services in my phone, and use an open app store like F-Droid.
Not that it counts for much in the grand scheme of things, seeing as 99% of users will never install a custom rom.
Can you really install a custom rom, when it flushes proprietary firmware for eg the camera? There are only a handful of devices that doesnāt do that afaik.
It used to be possible to dip your toes in with light modifications while still keeping everything working. Now with verified boot chains, safety net and hardware backed verification, you either have everything fully locked down or you go full Richard Stallman and have everything foss.
I like how the second line of your comment answers the first.
It's spying on me, but I can still install whatever i want. Same as Windows 10.
It never became worse than iOS on "open" perspective.
Yes, I'm all against data collection and I'm pro open source, but let's be honest, Android, despite its flaws (of which it has many), has always been the best MAJOR choice in that regard.
Of course you can always pick up a Librem 5 or a Pinephone, but I'd argue it comes with a lot of drawbacks.
For comparison, the Nintendo Switch cardboard piano works by having you press a cardboard key with a highly reflective sticker on the other end, to lift that sticker above a barrier, so it can be seen by the game controller's integrated IR camera, which streams the image over to the switch over bluetooth, which then runs some image analysis to figure out which key sticker just became visible and THEN plays a note and the latency is STILL so much lower than on Android, that you can actually play it like a musical instrument.
Do you have any figures to corroborate that? Because it sounds incredibly unlikely.
The wiimote has built in IR blob extraction from the sensor on device, so I think he is wrong unless Switch/joycon changed things. In some modes, which piano may use, it basically is only sending sparse location data of 4 brightest detected light blobs.
It'd be nice if something was done about the extra 100-200ms (!!) of latency that Bluetooth headsets can add[1]. I understand this is largely a problem from the manufacturer problem, but I'd love to see Google lead the way with some sort of certification program, or even make their own low cost BT chipset for 3rd party headphones to use, to improve the situation.
Of course Android is already world's better than desktop, where 200ms is seemingly closer to average than an outlier.
It is also kind of cool that perusing that chart, it seems headsets connected to Android have, on average, lower latency than when connected to iOS!
https://www.rtings.com/headphones/tests/connectivity/bluetoo...
There are low latency bluetooth protocols, if you sort by latency on rtings for aptx-LL you can see some headsets are down below 40ms. Of course, this relies on both the host and the headset having support for the protocol and being engineered well. SBC and aptX just aren't designed for low latency, and low latency is something that can't be hacked into every codec after the fact.
I'll argue that an device meant for phone calls (which is where bluetooth audio started out!) should aim for low latency by default. I also understand that the CPU/Latency/Memory trade offs made over a decade ago are not the same one's we'd make now.
Doesn't change the fact that the same device can have a 3x latency difference between platforms with the same codec.
Google went and developed VP9 as a royalty free codec in the AV space, they should do something similar for Bluetooth. Right now if I'm on a video call with someone and we both have BT headsets, it is very possible that there is a 500ms audio latency!
Latency on calls over cellular networks is already very high (my subjective experience). But we humans are good at working around it when it's just two people - wait for the other person to finish speaking before you speak, and the other person might have a little longer delay before hearing your response.
If there are multiple parties on the call, this breaks down, because if two people want to speak, they both start talking and don't realize it for a little while.
On video calls this also breaks down, because either video and audio get out of sync (like with zoom) or you get odd artifacts in either the video or audio as they are kept in sync.
> I'll argue that an device meant for phone calls (which is where bluetooth audio started out!) should aim for low latency by default. I also understand that the CPU/Latency/Memory trade offs made over a decade ago are not the same one's we'd make now.
It turns out that the device that's meant for phone calls has to, first and foremost, make sure that there are no dropouts, crackles and other buffer underrun articles. Latency comes afterwards.
What about LC3? It was announced a year ago?
apt-x is designed for ransom extraction via "intellectual property" licensing.
How is this different from any other licensing?
I would be happy if something could be done to prevent audio latency for all devices on my system from permanently increasing if I ever connect a bluetooth audio device.
On Linux, I have to restart pulseaudio after using Bluetooth audio, otherwise retroarch is unplayable.
BTW where are you getting numbers for 200ms latency being average on desktop? My understanding was that latency is typically around 40-50ms on desktop if you are using analog speakers.
I've never understood why Wi-Fi latency is only 2-3ms while Bluetooth is 100-200ms when they use the same frequency band.
Is it intentional to ensure there's plenty of buffer so audio packets can be re-sent when they fail, and the audio is seamless? In other words, is 100-200ms just inherent at that frequency with expected interference?
Or is it actually just terribly designed?
> I've never understood why Wi-Fi latency is only 2-3ms while Bluetooth is 100-200ms when they use the same frequency band.
You're mixing up two different things. Transport latency of WiFi is 2-3ms, but for bluetooth you're suddently talking about a transport, codec, receive buffer and playback code latency. That's not an apple-to-apple comparison.
If you want comparable numbers, measure the latency of audio being streamed from one computer to another via PulseAudio and you'll quickly see that it's more than 2-3ms. And there's a reason for that: most people strongly prefer their music to not stutter and work properly with all kinds of headphones instead of having absolutely no latency. Most people prefer to shove their phone in a separate room, back pocket and walk around RF saturated areas and still listen to music without interruption.
Hence the manufacturers err on the side of buffering some audio so it's more resilient to connection dropouts. You wifi, after all, doesn't need to move around busy highways and public transport systems.
I'd assume a combination of larger buffers for seamless error recovery and also for lower power consumption. The receiving side of that bluetooth connection is also typically extremely battery constrained.
Oh I'd never considered lower power consumption as a reason for increased latency.
I don't immediately see the relationship however -- is there anything you could point me to, to explain the mechanism?
There is AptX LL (low latency). I have no idea how widespread support is, but you can buy BT dongles, eg to connect video equipment.
The figure of most interest to someone playing an electronic instrument is "tap-to-touch latency". The article indicates a minimum of 43 ms for that (28 round trip latency - 5 audio input latency + 20 touch latency). That's like ten times what you'd hope for.
Emulators of old retro games also care about audio latency, as the emulated audio is generated on the fly by a turning machine, which makes it very hard to match visuals to audio if there is inherent audio latency.
If just matching is desired then you can just buffer video frames; all audio frameworks provide means to synchronize video and audio.
Of course, it probably makes the experience just worse if the delay is silly long..
By doing that you create input lag for the game.
I was very sad when I got a Galaxy Note (R.I.P. Note line) and Elite Beat Agents was unplayable due to audio lag. It makes sense, but it is a shame.
Damn, Elite Beat Agents was such a good game, I cried like a baby in the stage with You're My Inspiration.
The 20 ms tap-to-touch reference is from 2017, maybe this has gotten better in the past 4 years? But maybe not. And per the source, it's possible to get down to 10 ms round trip latency if you buy the right phone.
60Hz touch scan rate (16.67ms) was common in 2017, but now we can easily buy a phone with 240Hz(4.17ms) touch scan rate. It should be much improvement.
The article specifically says that tap to tone latency is typically around 20ms.
deleted
We're talking about tap-to-audio-output latency though, so it has nothing to do with display part of the screen.
And there is no "touch" on PC generally speaking.. input latency would be on mouse or keyboard.
I think you're talking different thing from the topic.
In case of audio it doesn't really matter how fast the display updates though? Unless your numbers are for the digitizer and not the screen.
Andrew Huang recently made a YouTube video [0] that provides some insight from the perspective of a music producer and someone who recently created a music production app. It's not overly technical, but it highlights some of the challenges one might face.
As the article says, there is room for improvement, so I guess it's good to see that the long term goal is 10ms round trip.
That was a strange video. It seemed like he started talking about latency between Android and IOS devices, which makes sense given that this article talks about response time of just under 40 ms like you said. The article says this is "well within the range required for real-time applications" if you define real-time applications as non-pro-audio stuff. If you're playing an instrument and it takes 40 ms for you to hear back the sound, that's Not Good.
But then the video takes a strange turn and starts talking about "Stability" of mac vs PC, and I had to turn it off. It made the argument that "I haven't seen any professional producers use anything except macs" as its main argument.
It would have been nice to hear some more technical information like mentioning core-audio, or something else.
Regarding the article itself. I'm excited that Android is target less than 20 ms round-trip latency. I'm hoping that this work can be brought back to the Linux Desktop. Messing around with Jack and dealing with xruns is something I wish I could stop doing.
To be a little pedantic: He is not claiming that producers are never using PCs, he is claiming that live audio never runs on PC, because stability is the top priority there. Later he says in the studio you can tolerate a little instability if the other benefits of windows are worth it.
That no producers are using windows PCs is an obviously false claim. There are DAWs that don't even run on mac. There are also live production tools that don't, like Notch, so there must be someone running windows on stage, anyway.
That macs are more stable in day-to-day operation is not really a bold claim though.
> That macs are more stable in day-to-day operation is not really a bold claim though.
It absolutely is. Do you have any data to back up the claim? Anecdotally windows 10's stability seems vastly superior, especially with every release of MacOS seemingly regressing further and further.
Linux audio can be real bad but the one thing that could possibly make it worse is taking anything from Android. Android is the only computing platform I have ever used that was unable to play a simple audio file through without the audio dropping out at least once (I had the same experience on multiple devices, 5+ years apart).
Have you looked at Pipewire?
I've started to. But I haven't done enough research on how all the other applications I use ( Carla, Ardour, soft synths, etc ) will play along with Pipewire instead of jack.
>"I haven't seen any professional producers use anything except macs" as its main argument.
Yeah that is both totally irrelevant to the topic at hand and also a bit strange. I have only seen PCs in the pro studios I have used but that doesn't make me believe nothing else is used.
Same here. I've even seen an increased number of Linux-based studios, oftentimes running Bitwig or Reaper. I don't doubt that audio latency can be a little high on some workflows, but I've honestly had a less laggy experience with Pulse than Coreaudio. Out of the box, Bitwig has a latency of ~8ms on my (Linux) machine, as opposed to 20ms on my Macbook.
Usually, "an update on [blank]" is euphemism for said product getting cancelled.
More to the point, one of PulseAudio's claims was that they consumed less CPU than Android's audio pipeline. With the recent hyped PipeWire, I wonder how that compares to Android's Oboe.
That's actually the joke here, which got a chuckle out of me. The idea is that excessively large audio latency itself has gotten cancelled.
I haven't played with it myself (it's been a few years since I was actively involved in Android audio latency), but from what I've seen, AAudio is pretty good, probably close to what the hardware is capable of. I'd very much encourage people to do empirical measurements against, for example, PipeWire (which also looks good). Do keep in mind that constraints are different on mobile, and things like power management can and do get in the way of down-to-the-wire latency.
> I'd very much encourage people to do empirical measurements against, for example, PipeWire (which also looks good).
A rule of thumb from reading linux audio mailing lists but which probably works generally-- completely disregard any claim of measurement of latency that isn't prefixed.
For example, the Google blog labels the Y-axis with "round-trip time in milliseconds." They're clearly and explicitly measuring round-trip latency. They then piggy-back on that well-understood concept to introduce the concept of "tap-to-tone latency" and show measurements for that.
Almost to the case, the times I've read a back-and-forth conversation where the word "latency" is used unprefixed, the commenters might as well be describing the warmth of vinyl recordings. Often they are simply describing an integer they typed into a widget or config file.
On several occasions I've witnessed users insert Jack between a single application and ALSA because "that's what the professionals use." These are knowledgeable users who want as little latency between the system and their ear as possible.
Are you saying there is still a use case for going straight to ALSA without something in the middle?
At Google it typically manifests itself as "Advancing our amazing bet" https://news.ycombinator.com/item?id=12792928
Where was that claim made (for pulseaudio vs Android)?
Hearing any claim that pulseaudio uses less cpu than some alternative makes me blink. For years I could not use pulseaudio while playing games, because it would eat 99% cpu and I only had one cpu core. Things are better now that I have eight cores, but I don't know if that's due to the extra cores or if pulseaudio has improved.
I tried jackd, but the documentation is overwhelming.
I remember ESD and OSS working just fine, but I was young and maybe I was just happy to get audio at all. I grew up with blips and bleeps coming out of the PC speaker.
Get a daily email with the the top stories from Hacker News. No spam, unsubscribe at any time.