Hacker News
ā† Back

An update on Android's audio latency

3 years ago/170 comments/android-developers.googleblog.com
3 years ago by jefftk

This is way better than it used to be (I measured half a second latency in 2012: https://www.jefftk.com/p/android-sound-synthesis-and-latency) but it is still too high to let you use an Android device as a musical instrument. This is something iPhones have always been able to do, and one of my biggest disappointments with Android.

(Disclosure: I work for Google, speaking only for myself)

3 years ago by limeblack

I agree audio at minimum should have been done differently on Android.

The latency(different type I believe) of Spotify applications and even YouTube applications is pretty bad compared to an Apple device. Sometimes the audio just fails to respond for half a second or so. This is not an interface bug from my experience.

3 years ago by thomasahle

What are the equivalent numbers for iPhone? Touch to audio and round-trip times?

3 years ago by kall

This data [0] doesn't have the latest models, and it's meassuring latency internaly, not "touch-to-sound". The ones they tested average around 7-8ms. I've heard of <10ms as the benchmark for "instant" in audio.

Edit: I've been thinking about this and a touch-to-tone latency below 10ms seems imposible, even with the 120hz digitizer on newer iPhones. I would be very interested in seeing some experimental data as well.

[0] https://juce.com/discover/stories/Mobile%20performance%20ind...

3 years ago by thomasahle

The best I could find was https://developer.apple.com/forums/thread/71301 which suggests a 32ms touch latency. I think that might include some processing, but it doesn't seem much better than the 20ms latency in Android mentioned in the article.

The audio processing latency seems to be single digit on both systems.

In any case, I don't think these differences are big enough to argue that "Android can't be used for instruments".

3 years ago by bentpins

Xiaomi has hit 480hz touch sampling with the Mi 11. It looks possible in hardware and in software, someone just has to manage to perfect both in the same phone

3 years ago by jefftk

https://superpowered.com/latency has messy data, but it looks like typically single-digit latency.

3 years ago by S_A_P

I think typical newish (2014+) devices have between 5-9ms latency. Having core audio from the start was a huge leg up for iOS for sure. I am mostly platform agnostic unless I am working with audio where I pretty much only use Apple products.

3 years ago by philsnow

The article and all the responses here express "latency" as a simple scalar number. Is it actually more like a histogram of values than a scalar, and the marketing number is the 90th %ile of that histogram?

My impression is that Android has more latency variance than iOS. If that's true then it's important to be clear what the reported numbers represent.

3 years ago by jefftk

If you open the audio context you should see consistent latency as long as you keep it open, but each time you open a new audio context you might end up with different latency. You should not see jitter within a single context.

3 years ago by Godel_unicode

Latency variance is known as jitter. What gives you the impression that it would be high on Android?

3 years ago by philsnow

Having used nearly every flagship android device from the HTC Dream up through the Nexus 6p, and then switching to an iphone 6 and never looking back, except watching acquaintances use their android phones.

Also from talking to an engineer who worked on earlier generations of iphones, who said that every team had a latency budget that they could not exceed.

3 years ago by pjmlp

On the 3D side it is similarly disappointing, as Google never put the effort of proper frameworks similar to Apple's kits, and SceneForm on Google's tradition is now deprecated after VR on Android lost its marketing drive.

3 years ago by Blueskytech

If android can get its audio latency act together there is some real opportunity here to steal market share from Apple. Since the release of the 2nd iPad Pro Apple has forced audio processing onto the slower low power CPU core, this means that for the last three years I can run all my iOS synth applications at 96kHZ with a low buffer sample size without any drop outs on my first gen iPad Pro and an iPad mini 3. What I cannot do is get similar performance on last year's iPad Pro. It is absolute insanity that Apple has no work around to direct audio processing to the high powered cores for apps made for musicians. Presumably at some point they will allow you to do this with a documented programing call but it has been three years of bad performance on synth applications due to the necessity to have high sample buffers (ie high latency).

3 years ago by Tade0

I've noticed an increase in measured latencies, but didn't know about that slower CPU core thing.

Sad to see Apple take such a direction - musicians appreciated having proper hardware/software and Apple definitely made money on all the paid content. Why let this arrangement fall by the wayside?

3 years ago by layoutIfNeeded

The 10x engineers have either left Apple or were reorganized to gimmick projects like SwiftUI.

3 years ago by aeontech

Is there more details about this published somewhere? Iā€™m very curious to learn more

3 years ago by xenadu02

> Since the release of the 2nd iPad Pro Apple has forced audio processing onto the slower low power CPU core

What makes you believe this?

3 years ago by pjmlp

On Android is even worse, specially if one gets an average model.

3 years ago by Robotbeat

Responsiveness and low input-to-result latency was why I chose iPhone over android 10 years ago even though I prefer a more open environment like android. Sounds like things havenā€™t changed a whole lot on that front.

Sometimes I wonder if these increased levels of abstraction and even digitization itself (or at least the Internet-ization of things) is a mistake for human-machine interface. Watching drone racing pilots use analog controls and analog video headsets in spite of the potato resolution because WiFi/digital/etc just add too much latency is kind of eye opening and makes you sort of reconsider the last couple decades. (Improvements have been made so digital isnā€™t so high latency and VR has served to address a lot of these latency concerns, but itā€™s still kind of interesting...)

3 years ago by thefourthchime

What's funny is that this ā€œopenā€ environment of android now is nearly the opposite.

With any android device openly spying on you while locking down all the open source apps you might want to install.

3 years ago by andrepd

What does this mean? I can choose not to install google services in my phone, and use an open app store like F-Droid.

Not that it counts for much in the grand scheme of things, seeing as 99% of users will never install a custom rom.

3 years ago by kaba0

Can you really install a custom rom, when it flushes proprietary firmware for eg the camera? There are only a handful of devices that doesnā€™t do that afaik.

3 years ago by foobar33333

It used to be possible to dip your toes in with light modifications while still keeping everything working. Now with verified boot chains, safety net and hardware backed verification, you either have everything fully locked down or you go full Richard Stallman and have everything foss.

3 years ago by BoorishBears

I like how the second line of your comment answers the first.

3 years ago by idonotknowwhy

It's spying on me, but I can still install whatever i want. Same as Windows 10.

3 years ago by fomine3

It never became worse than iOS on "open" perspective.

3 years ago by ladyanita22

Yes, I'm all against data collection and I'm pro open source, but let's be honest, Android, despite its flaws (of which it has many), has always been the best MAJOR choice in that regard.

Of course you can always pick up a Librem 5 or a Pinephone, but I'd argue it comes with a lot of drawbacks.

3 years ago by RedNifre

For comparison, the Nintendo Switch cardboard piano works by having you press a cardboard key with a highly reflective sticker on the other end, to lift that sticker above a barrier, so it can be seen by the game controller's integrated IR camera, which streams the image over to the switch over bluetooth, which then runs some image analysis to figure out which key sticker just became visible and THEN plays a note and the latency is STILL so much lower than on Android, that you can actually play it like a musical instrument.

3 years ago by fistynuts

Do you have any figures to corroborate that? Because it sounds incredibly unlikely.

3 years ago by cma

The wiimote has built in IR blob extraction from the sensor on device, so I think he is wrong unless Switch/joycon changed things. In some modes, which piano may use, it basically is only sending sparse location data of 4 brightest detected light blobs.

3 years ago by com2kid

It'd be nice if something was done about the extra 100-200ms (!!) of latency that Bluetooth headsets can add[1]. I understand this is largely a problem from the manufacturer problem, but I'd love to see Google lead the way with some sort of certification program, or even make their own low cost BT chipset for 3rd party headphones to use, to improve the situation.

Of course Android is already world's better than desktop, where 200ms is seemingly closer to average than an outlier.

It is also kind of cool that perusing that chart, it seems headsets connected to Android have, on average, lower latency than when connected to iOS!

https://www.rtings.com/headphones/tests/connectivity/bluetoo...

3 years ago by kevingadd

There are low latency bluetooth protocols, if you sort by latency on rtings for aptx-LL you can see some headsets are down below 40ms. Of course, this relies on both the host and the headset having support for the protocol and being engineered well. SBC and aptX just aren't designed for low latency, and low latency is something that can't be hacked into every codec after the fact.

3 years ago by com2kid

I'll argue that an device meant for phone calls (which is where bluetooth audio started out!) should aim for low latency by default. I also understand that the CPU/Latency/Memory trade offs made over a decade ago are not the same one's we'd make now.

Doesn't change the fact that the same device can have a 3x latency difference between platforms with the same codec.

Google went and developed VP9 as a royalty free codec in the AV space, they should do something similar for Bluetooth. Right now if I'm on a video call with someone and we both have BT headsets, it is very possible that there is a 500ms audio latency!

3 years ago by cout

Latency on calls over cellular networks is already very high (my subjective experience). But we humans are good at working around it when it's just two people - wait for the other person to finish speaking before you speak, and the other person might have a little longer delay before hearing your response.

If there are multiple parties on the call, this breaks down, because if two people want to speak, they both start talking and don't realize it for a little while.

On video calls this also breaks down, because either video and audio get out of sync (like with zoom) or you get odd artifacts in either the video or audio as they are kept in sync.

3 years ago by izacus

> I'll argue that an device meant for phone calls (which is where bluetooth audio started out!) should aim for low latency by default. I also understand that the CPU/Latency/Memory trade offs made over a decade ago are not the same one's we'd make now.

It turns out that the device that's meant for phone calls has to, first and foremost, make sure that there are no dropouts, crackles and other buffer underrun articles. Latency comes afterwards.

3 years ago by mrwoggle

What about LC3? It was announced a year ago?

3 years ago by summm

apt-x is designed for ransom extraction via "intellectual property" licensing.

3 years ago by AndriyKunitsyn

How is this different from any other licensing?

3 years ago by cout

I would be happy if something could be done to prevent audio latency for all devices on my system from permanently increasing if I ever connect a bluetooth audio device.

On Linux, I have to restart pulseaudio after using Bluetooth audio, otherwise retroarch is unplayable.

BTW where are you getting numbers for 200ms latency being average on desktop? My understanding was that latency is typically around 40-50ms on desktop if you are using analog speakers.

3 years ago by crazygringo

I've never understood why Wi-Fi latency is only 2-3ms while Bluetooth is 100-200ms when they use the same frequency band.

Is it intentional to ensure there's plenty of buffer so audio packets can be re-sent when they fail, and the audio is seamless? In other words, is 100-200ms just inherent at that frequency with expected interference?

Or is it actually just terribly designed?

3 years ago by izacus

> I've never understood why Wi-Fi latency is only 2-3ms while Bluetooth is 100-200ms when they use the same frequency band.

You're mixing up two different things. Transport latency of WiFi is 2-3ms, but for bluetooth you're suddently talking about a transport, codec, receive buffer and playback code latency. That's not an apple-to-apple comparison.

If you want comparable numbers, measure the latency of audio being streamed from one computer to another via PulseAudio and you'll quickly see that it's more than 2-3ms. And there's a reason for that: most people strongly prefer their music to not stutter and work properly with all kinds of headphones instead of having absolutely no latency. Most people prefer to shove their phone in a separate room, back pocket and walk around RF saturated areas and still listen to music without interruption.

Hence the manufacturers err on the side of buffering some audio so it's more resilient to connection dropouts. You wifi, after all, doesn't need to move around busy highways and public transport systems.

3 years ago by kllrnohj

I'd assume a combination of larger buffers for seamless error recovery and also for lower power consumption. The receiving side of that bluetooth connection is also typically extremely battery constrained.

3 years ago by crazygringo

Oh I'd never considered lower power consumption as a reason for increased latency.

I don't immediately see the relationship however -- is there anything you could point me to, to explain the mechanism?

3 years ago by MrBuddyCasino

There is AptX LL (low latency). I have no idea how widespread support is, but you can buy BT dongles, eg to connect video equipment.

3 years ago by Jeff_Brown

The figure of most interest to someone playing an electronic instrument is "tap-to-touch latency". The article indicates a minimum of 43 ms for that (28 round trip latency - 5 audio input latency + 20 touch latency). That's like ten times what you'd hope for.

3 years ago by Drakim

Emulators of old retro games also care about audio latency, as the emulated audio is generated on the fly by a turning machine, which makes it very hard to match visuals to audio if there is inherent audio latency.

3 years ago by _flux

If just matching is desired then you can just buffer video frames; all audio frameworks provide means to synchronize video and audio.

Of course, it probably makes the experience just worse if the delay is silly long..

3 years ago by Drakim

By doing that you create input lag for the game.

3 years ago by TheRealSteel

I was very sad when I got a Galaxy Note (R.I.P. Note line) and Elite Beat Agents was unplayable due to audio lag. It makes sense, but it is a shame.

3 years ago by Lucasoato

Damn, Elite Beat Agents was such a good game, I cried like a baby in the stage with You're My Inspiration.

3 years ago by morsch

The 20 ms tap-to-touch reference is from 2017, maybe this has gotten better in the past 4 years? But maybe not. And per the source, it's possible to get down to 10 ms round trip latency if you buy the right phone.

3 years ago by fomine3

60Hz touch scan rate (16.67ms) was common in 2017, but now we can easily buy a phone with 240Hz(4.17ms) touch scan rate. It should be much improvement.

3 years ago by fistynuts

The article specifically says that tap to tone latency is typically around 20ms.

3 years ago by hnuser123456

deleted

3 years ago by fireattack

We're talking about tap-to-audio-output latency though, so it has nothing to do with display part of the screen.

And there is no "touch" on PC generally speaking.. input latency would be on mouse or keyboard.

I think you're talking different thing from the topic.

3 years ago by thomasahle

In case of audio it doesn't really matter how fast the display updates though? Unless your numbers are for the digitizer and not the screen.

3 years ago by karlding

Andrew Huang recently made a YouTube video [0] that provides some insight from the perspective of a music producer and someone who recently created a music production app. It's not overly technical, but it highlights some of the challenges one might face.

As the article says, there is room for improvement, so I guess it's good to see that the long term goal is 10ms round trip.

[0] https://youtube.com/watch?v=-sPbcTcUmcI

3 years ago by Naac

That was a strange video. It seemed like he started talking about latency between Android and IOS devices, which makes sense given that this article talks about response time of just under 40 ms like you said. The article says this is "well within the range required for real-time applications" if you define real-time applications as non-pro-audio stuff. If you're playing an instrument and it takes 40 ms for you to hear back the sound, that's Not Good.

But then the video takes a strange turn and starts talking about "Stability" of mac vs PC, and I had to turn it off. It made the argument that "I haven't seen any professional producers use anything except macs" as its main argument.

It would have been nice to hear some more technical information like mentioning core-audio, or something else.

Regarding the article itself. I'm excited that Android is target less than 20 ms round-trip latency. I'm hoping that this work can be brought back to the Linux Desktop. Messing around with Jack and dealing with xruns is something I wish I could stop doing.

3 years ago by kall

To be a little pedantic: He is not claiming that producers are never using PCs, he is claiming that live audio never runs on PC, because stability is the top priority there. Later he says in the studio you can tolerate a little instability if the other benefits of windows are worth it.

That no producers are using windows PCs is an obviously false claim. There are DAWs that don't even run on mac. There are also live production tools that don't, like Notch, so there must be someone running windows on stage, anyway.

That macs are more stable in day-to-day operation is not really a bold claim though.

3 years ago by kllrnohj

> That macs are more stable in day-to-day operation is not really a bold claim though.

It absolutely is. Do you have any data to back up the claim? Anecdotally windows 10's stability seems vastly superior, especially with every release of MacOS seemingly regressing further and further.

3 years ago by ubercow13

Linux audio can be real bad but the one thing that could possibly make it worse is taking anything from Android. Android is the only computing platform I have ever used that was unable to play a simple audio file through without the audio dropping out at least once (I had the same experience on multiple devices, 5+ years apart).

Have you looked at Pipewire?

3 years ago by Naac

I've started to. But I haven't done enough research on how all the other applications I use ( Carla, Ardour, soft synths, etc ) will play along with Pipewire instead of jack.

3 years ago by Daho0n

>"I haven't seen any professional producers use anything except macs" as its main argument.

Yeah that is both totally irrelevant to the topic at hand and also a bit strange. I have only seen PCs in the pro studios I have used but that doesn't make me believe nothing else is used.

3 years ago by smoldesu

Same here. I've even seen an increased number of Linux-based studios, oftentimes running Bitwig or Reaper. I don't doubt that audio latency can be a little high on some workflows, but I've honestly had a less laggy experience with Pulse than Coreaudio. Out of the box, Bitwig has a latency of ~8ms on my (Linux) machine, as opposed to 20ms on my Macbook.

3 years ago by AceJohnny2

Usually, "an update on [blank]" is euphemism for said product getting cancelled.

More to the point, one of PulseAudio's claims was that they consumed less CPU than Android's audio pipeline. With the recent hyped PipeWire, I wonder how that compares to Android's Oboe.

3 years ago by raphlinus

That's actually the joke here, which got a chuckle out of me. The idea is that excessively large audio latency itself has gotten cancelled.

I haven't played with it myself (it's been a few years since I was actively involved in Android audio latency), but from what I've seen, AAudio is pretty good, probably close to what the hardware is capable of. I'd very much encourage people to do empirical measurements against, for example, PipeWire (which also looks good). Do keep in mind that constraints are different on mobile, and things like power management can and do get in the way of down-to-the-wire latency.

3 years ago by jancsika

> I'd very much encourage people to do empirical measurements against, for example, PipeWire (which also looks good).

A rule of thumb from reading linux audio mailing lists but which probably works generally-- completely disregard any claim of measurement of latency that isn't prefixed.

For example, the Google blog labels the Y-axis with "round-trip time in milliseconds." They're clearly and explicitly measuring round-trip latency. They then piggy-back on that well-understood concept to introduce the concept of "tap-to-tone latency" and show measurements for that.

Almost to the case, the times I've read a back-and-forth conversation where the word "latency" is used unprefixed, the commenters might as well be describing the warmth of vinyl recordings. Often they are simply describing an integer they typed into a widget or config file.

On several occasions I've witnessed users insert Jack between a single application and ALSA because "that's what the professionals use." These are knowledgeable users who want as little latency between the system and their ear as possible.

3 years ago by cout

Are you saying there is still a use case for going straight to ALSA without something in the middle?

3 years ago by alpb

At Google it typically manifests itself as "Advancing our amazing bet" https://news.ycombinator.com/item?id=12792928

3 years ago by cout

Where was that claim made (for pulseaudio vs Android)?

Hearing any claim that pulseaudio uses less cpu than some alternative makes me blink. For years I could not use pulseaudio while playing games, because it would eat 99% cpu and I only had one cpu core. Things are better now that I have eight cores, but I don't know if that's due to the extra cores or if pulseaudio has improved.

I tried jackd, but the documentation is overwhelming.

I remember ESD and OSS working just fine, but I was young and maybe I was just happy to get audio at all. I grew up with blips and bleeps coming out of the PC speaker.

Daily Digest

Get a daily email with the the top stories from Hacker News. No spam, unsubscribe at any time.