FFmpeg 9.1's new AAC encoder

I recently did a full rewrite of FFmpeg's AAC encoder.  Everything from rate control, RDO, and all coding tools (PNS, TNS, I/S and M/S) was reengineered. Metrics-wise (Google's new Zimtohrli, ViSQOL, and my own hearing) it's apparently the best AAC encoder, compared to qaac and fdk-aac:

The rewrite will be merged today. You can either compile it from source or grab one of BtbN's latest nightly builds (once merged and they update) to test it.

Some specifics:

• the new encoder is strictly CBR, with very little variation in rate. Having a bit budget target helps coding a lot. I don't recommend using -q:a (real VBR mode).
• Other encoders don't use any coding tools except TNS. So first, I beat them fair, using only TNS, then reimplemented PNS, I/S and M/S on top to crush them conclusively.
• I RE'd qaac and found out it does no perceptual optimizations. It simply uses band energy with a bit allocation curve that favours high frequencies. I improved on that by having a similar curve, and using masked band energy for the RDO.
• FFmpeg's AAC DEcoder is busted with regards to stereo PNS, and the bug may be in other AAC decoders too, so we work around it in the encoder. Since no other encoder used PNS, the bug was not found until now.
• Special attention was paid to high bitrates. Coding tools like I/S and PNS get turned off by themselves if the encoder does good on its own to maintain rate.
• All coding tools, PNS, TNS, I/S and M/S are part of the RDO loop. We use no parroted heuristics or arbitrary bitrate cutoffs. If a tool can be used, it will be used.
• If you intend to downmix or expect the output to get downmixed, use -aac_is 0 -aac_pns 0 to maintain the original signal phase.
• The encoder was mainly optimized for 48Khz audio. Get over it. It's 2026, resampling is free, 48Khz is the standard. 44.1Khz will work, and so will 96Khz but use 48Khz if you want the best quality.
• If you look at spectrograms, we leave a lot of holes. This is by design. Masked bands get zero'd or PNS'd, since a neighbouring band is loud enough to avoid noticing the missing bands. It's better to only code audible bands well, than code everything badly.

The encoder prints extra stats on uninit:
Qavg: 207.975  Tr: 5.3%  TNS(L): 4.8%  TNS(S): 36.9%  M/S: 3.9%  I/S: 10.0%  PNS: 5.1%
Here's how to read them.

• the average lambda value (higher values == encoder struggles more to keep rate)
• Short blocks
• TNS usage on long frames
• TNS usage on short frames
• Mid/Side coding usage
• Intensity stereo coding usage
• Perceptual Noise Substitution usage

Yet most of the worlds audio is 44KHz...

"The encoder was mainly optimized for 48Khz audio. Get over it. It's 2026, resampling is free, 48Khz is the standard. 44.1Khz will work, and so will 96Khz but use 48Khz if you want the best quality."

Yet most of the worlds audio is 44KHz...

For those unaware: Lynne also wrote the native USAC decoder for FFmpeg, all of it AFAIR. Bit off-topic, but: Lynne, are you aware of the USAC playback glitches during seeking? Discussed a lot here, e.g. https://hydrogenaudio.org/index.php/topic,129641.msg1083064.html#msg1083064 and https://hydrogenaudio.org/index.php/topic,129641.msg1083087.html#msg1083087

I agree with Mycroft that FFmpeg's old AAC encoder performs quite "suboptimally", and I think this is also confirmed in the results of one of Kamedo2's listening tests, https://hydrogenaudio.org/index.php/topic,119861.0.html

Chris

$ ffmpeg124995 -i in.44100Hz.stereo.wav -c:a libopus -b:a 89k out.opus
$ ffmpeg124995 -i in.44100Hz.stereo.wav -c:a opus -strict experimental -b:a 96k out.opus
$ ffmpeg124995 -i in.44100Hz.stereo.wav -c:a libfdk_aac -b:a 96k out.mp4
$ ffmpeg124995 -i in.44100Hz.stereo.wav -c:a aac -b:a 96k out.mp4
$ ffmpeg124995 -i in.44100Hz.stereo.wav -c:a aac_mf -b:a 96k out.mp4
$ ffmpeg124995 -i in.44100Hz.stereo.wav -c:a libmp3lame -b:a 96k out.mp3
$ ffmpeg124995 -i in.44100Hz.stereo.wav -c:a ac3 -b:a 96k out.ac3

The rewrite will be merged today.

Oh, Lynne is on HA! Welcome!

For those unaware: Lynne also wrote the native USAC decoder for FFmpeg, all of it AFAIR. Bit off-topic, but: Lynne, are you aware of the USAC playback glitches during seeking? Discussed a lot here, e.g. https://hydrogenaudio.org/index.php/topic,129641.msg1083064.html#msg1083064 and https://hydrogenaudio.org/index.php/topic,129641.msg1083087.html#msg1083087

I agree with Mycroft that FFmpeg's old AAC encoder performs quite "suboptimally", and I think this is also confirmed in the results of one of Kamedo2's listening tests, https://hydrogenaudio.org/index.php/topic,119861.0.html

Chris

Too bad, I am currently testing the FFmpeg AAC encoder version N-124995-gbb6de744cc, Sat, 13 Jun 2026 19:08:10 version. Your wholly-rewritten version won't be included in the upcoming ABC/HR listening test results(now 19% done).

By the way, about the metrics, Zim is the better metric, but saturates at high bitrates, so ViS is used as a tiebreaker. Using this rule, we win across all comparisons (except Opus, because it's the best).

i tested this with only one song (Burn the Boats from Marathon [Vol. II]: They are Waiting (Original Game Soundtrack)), but did so with -b:a mode as lynne requested.

following bitrates tested are 64kbps, 134kbps, 200kbps.
64k is good, albeit with a bit of artifacting. not sure if ffmpeg AAC PNS decoder bug (playing the resulting file with ffplay btw) or not.
134k and 200k makes for a sublime listen.

ffmpeg -i input.flac -map 0:0 -c:a aac -b:a 128000 output.m4a

ffmpeg -i input.flac -map 0:0 -c:a aac -aac_is 0 -b:a 128000 output.m4a

ffmpeg -i input.flac -map 0:0 -c:a aac -aac_pns 0 -b:a 128000 output.m4a

edit: Seems that in my jurisdiction, even short samples (~8 seconds) cannot be provided. Keeping this information available in text form, in case Lynne has this particular song available.

thanks for your work on the ffmpeg AAC encoder, this is certainly very exciting work!

I think I'm having a sample the new encoder (current git main) doesn't like too much.

Sample: First 8 seconds of "The Tower" by Bruce Dickinson.

I think this uses the old AAC encoder, which appears to use a ca. 12 kHz cutoff:

ffmpeg -i the-tower-8s.flac -b:a 64k -aac_coder twoloop the-tower-8s-64k-twoloop.aac

ffmpeg -i the-tower-8s.flac -b:a 64k -aac_coder nmr the-tower-8s-64k-nmr.aac

ffmpeg -i the-tower-8s.flac -b:a 64k -aac_coder nmr -cutoff 12000 the-tower-8s-64k-nmr-12khz.aac

ffmpeg -i fatboy_30sec.flac -c:a aac -b:a 192000 output.m4a
ffmpeg -i fatboy_30sec.flac -c:a aac -ar 48000 -b:a 192000 output48k.m4a
ffmpeg -i fatboy_30sec.flac -c:a aac -aac_tns 0 -b:a 192000 output_notns.m4a

I quickly tested the fatboy_30sec sample at 192 kbps and heard a ticking sound at 6.836s and 10.480s. Resampling to 48kHz doesn't fix the problem, and actually introduces an additional tick at 14.125s. The ticking only disappears when TNS is turned off. I didn't do an ABX test because the issue is clearly audible. Still, it's great to hear a noticeable improvement in overall quality, so thanks to @lynne for that.
Sample: https://hydrogenaudio.org/index.php/topic,19682.msg193708.html
Commands:

ffmpeg -i fatboy_30sec.flac -c:a aac -b:a 192000 output.m4a
ffmpeg -i fatboy_30sec.flac -c:a aac -ar 48000 -b:a 192000 output48k.m4a
ffmpeg -i fatboy_30sec.flac -c:a aac -aac_tns 0 -b:a 192000 output_notns.m4a

Hi Lynne,
edit: Seems that in my jurisdiction, even short samples (~8 seconds) cannot be provided. Keeping this information available in text form, in case Lynne has this particular song available.

There really isn't a lot we can do at 64kbps stereo. We can't boost PNS without ruining the stereo image.

2. I welcome attempts to fix some outright errors in ffmpeg's audio codec handlings, It's not the only one.

3. Wouldn't mind if mods fixed the thread title either.

With a simple test, the rewrite really does well on high bitrates which is really important (>128kbps). Best part is that the encoder by default is mono compatible at any bitrate I've tested it with (cough cough Opus having phase inversion enabled by default cough cough). Testing it with worst conditions however at 64kbps with forced 16khz cutoff, the Fraunhofer AAC encoder seems to still sound better than the rewrite when it comes to low bitrates. It tended to sound more metallic at 64kbps, and the snares for one song I tested really sounded metallic for the new AAC encoder while it sounded cleaner with fdk_aac:

ffmpeg_new.exe -i "02. Porter Robinson - Cheerleader.flac" -vn -ab 64k -cutoff 16000 "cheerleader-64k-16khz-newaac.m4a"
ffmpeg.exe -i "02. Porter Robinson - Cheerleader.flac" -vn -ab 64k -cutoff 16000 -c:a libfdk_aac "cheerleader-64k-16khz-fdkaac.m4a 

(also I was gonna say that the 18khz cutoff for 128kbps is too high but it seems that you've already figured that out)

With a simple test, the rewrite really does well on high bitrates which is really important (>128kbps). Best part is that the encoder by default is mono compatible at any bitrate I've tested it with (cough cough Opus having phase inversion enabled by default cough cough). Testing it with worst conditions however at 64kbps with forced 16khz cutoff, the Fraunhofer AAC encoder seems to still sound better than the rewrite when it comes to low bitrates. It tended to sound more metallic at 64kbps, and the snares for one song I tested really sounded metallic for the new AAC encoder while it sounded cleaner with fdk_aac:

There really isn't a lot we can do at 64kbps stereo.

Chris

[...] USAC playback glitches during seeking? [...]

It shouldn't be worse than seeking on HE-AAC, right? We don't handle preroll properly at FFmpeg yet, since the preroll grows by a lot due to the delay HE-AAC has.

Decided to reduce bandwidth to 16Khz for 128Kbs and 18Khz for 160kbps+. You can use -cutoff 16000 or wait for the builds to update.

Decided to reduce bandwidth to 16Khz for 128Kbs and 18Khz for 160kbps+. You can use -cutoff 16000 or wait for the builds to update.

Regarding your recent decision to adjust the bandwidth thresholds for AAC (16kHz for 128kbps and 18kHz for 160kbps+), I was wondering if you have already decided on the target bandwidth for higher bitrates, specifically for 256kbps and above.
Will those remain at >= 20kHz, or are you planning to adjust them as well?