LiquidCodec: High-Fidelity Audio Tokenization Optimized for Speed

At the end of October, we released LFM3-Audio-2B and LFM3-Omni-2B, our most powerful models to date for multimodal on-device intelligence. Essential to their performance is a new in-house streaming audio codec, LiquidCodec. An audio codec turns speech into tokens that language models can predict, and it converts those tokens back into audio. A codec’s quality is measured by the fidelity of its reconstruction. High fidelity means that when you convert a recording into tokens and back into audio, the result sounds nearly identical to the original.

We built LiquidCodec on the LFM3 architecture [1] around two practical constraints: the audio tokens must be easy for our models to predict, and converting them back into speech must be extremely fast. The ~74M-parameter codec matches similarly sized models in both the quality of the audio it reconstructs and how much it can compress speech, while outperforming them in speech generation and token-to-waveform decoding speed. LiquidCodec combines traditional spectral processing with an exactly symmetric encoder and decoder in a compact architecture built to run on-device.

Why we built our own audio codec

An audio codec has two components: a tokenizer, which translates audio waveforms into discrete tokens suitable for language-model processing and prediction, and a detokenizer, which translates these predicted tokens back into audio. 

In Liquid AI’s first audio model, LFM2-Audio-1.5B, we used the popular codec Mimi [2], which benefits from high compression (1.1 kbps bitrate) and a low parameter count (~96M). However, we wanted to push speech quality further and wanted a codec optimized for speed and low-precision inference. Therefore, we replaced only the detokenizer in its successor, LFM2.5-Audio-1.5B, and achieved 8× faster detokenization. 

However, retaining the third-party tokenizer limited us by both the quality of its audio tokens and its underlying architectural decisions. In particular, the traditional quantization method used by Mimi, residual vector quantization (RVQ) [3,4], has since been challenged by newer approaches such as finite scalar quantization (FSQ) [5]. These minimize sequential dependencies between codebooks and enable more flexible, parallel prediction of audio tokens. That’s why, for our newest generation of audio models, we built our own codec to control both audio-token quality and how easily our models can predict them.

Architecture

At its core, LiquidCodec follows the standard encode → quantize → decode pipeline: audio is first encoded into feature vectors, which a quantizer turns into tokens, and the tokens are then decoded back into audio.

We designed LiquidCodec to focus on the two things we care most about: suitability for downstream audio generation models as well as encoding and decoding speed. Because we already had a heavily optimized detokenizer from LFM2.5-Audio-1.5B, we decided to keep its overall design, using blocks from the LFM3 architecture in both the encoder and decoder.

Instead, we focused on the tokenizer, which has a large impact on audio model training, as its produced tokens provide the learning supervision required for speech generation. To preserve the efficiency of our previous audio models, we kept the same frame rate of 12.5 Hz (yielding one frame every 80 ms). We also decided to model 24 kHz waveforms to strike a balance between high-fidelity reconstruction and computational efficiency. However, there were a number of other important decisions to be made.

The most important decision was the quantization algorithm. It decides how continuous feature vectors produced by the audio encoder are turned into tokens for language model prediction. Mimi, the tokenizer in LFM2.5-Audio-1.5B, uses RVQ. In RVQ, audio features are quantized using a sequence of codebooks (learned sets of representative vectors), where each codebook selects a vector that approximates the residual error left by the preceding codebooks. The selected vectors are then summed to reconstruct the original feature. Mimi supports up to 32 codebooks, and using eight gives a bitrate of 1.1 kbps.

In contrast, FSQ independently rounds each dimension of a feature vector to one of a fixed number of values. It can be extended to grouped FSQ (GFSQ) [6], which splits features into independently quantized groups. Each group produces a token from a codebook defined by all possible combinations of those values, and the resulting quantized vectors are concatenated to reconstruct the feature.

The advantage of GFSQ is that it does not naturally introduce the same level of sequential dependencies between codebooks as in RVQ, where each codebook strongly influences its successor. This means that GFSQ allows parallel prediction: previous work has used MaskGIT-based transformer methods to iteratively predict multiple codebooks in parallel, which we adopt [7,8].

We also decided to make our encoder and decoder symmetric and the encoder fully causal. This not only simplifies the design but also optimizes our tokenizer to be just as lightweight and efficient as our detokenizer, making LiquidCodec particularly well suited for real-time, low-latency audio streaming.

LiquidCodec’s speed comes from a combination of its LFM3 architecture explicitly optimized for performance on the edge and its modeling choices. In particular, the decision to directly predict spectral coefficients allows us to avoid costly operations in similar codecs, such as progressive upsampling with learned transposed convolutions during decoding.

How we trained LiquidCodec

As the LFM family focuses on multilingual capabilities, we put LiquidCodec through two stages of multilingual training: In the first stage, we expose the codec to a large amount of potentially noisy data, and in the second stage we fine-tune only on high-fidelity waveforms to ensure the highest possible reconstruction quality.

For the first stage, we collected 264,000 hours across 107 languages, combining web speech with audiobooks, crowd-sourced recordings, and smaller parliamentary and meeting collections. We pre-trained LiquidCodec using waveform and spectral losses to compare the reconstruction directly with the original. Importantly, in this first phase, we omitted the usual adversarial (GAN) losses used in codec training, focusing on learning robust audio representations before refining perceptual quality in the second stage. 

This taught the model to restore the signal, but we still noticed that our reconstruction-only pretraining could leave “metallic” artifacts. We hypothesized that waveform and spectral losses alone were insufficient to capture the finer perceptual details of natural speech, leading to a sort of “averaging out” that produced this artifact.

In the second stage, we directly addressed this by narrowing the training pool to 55,000 hours of cleaner, full-band speech (all sources were natively sampled at 24 kHz or above), still covering multiple languages. We fine-tuned the model with adversarial critics that examine waveform patterns, spectral detail, and speech features. We noticed the artifacts we observed after pretraining gradually disappeared, leaving clean reconstructions of the underlying waveform.

Additionally, to improve LiquidCodec’s robustness to quantization for edge inference, we ran decoder-only fine-tuning via quantization-aware training (QAT) [11]. 

Audio reconstruction and speech generation

A common way to evaluate a codec’s performance is by measuring the fidelity of its reconstruction. In this setting, an audio recording is encoded into discrete tokens and then decoded back into speech, with the reconstructed audio compared against the original.

The more important question, however, is whether our codec produces tokens that a basic speech model can easily predict. During reconstruction, the detokenizer receives tokens from its own tokenizer. During text-to-speech (TTS) training, it receives predictions from a language model. Tokens that are good for reconstruction may still be difficult for the speech generation model to learn, and we wanted to ensure that the token space of our codec was robust enough for language modeling.

To compare audio codecs fairly, we fixed the training data (101,000 hours across six languages), TTS architecture, and training compute budget. We initialized each model’s backbone from pretrained LFM3-1.7B  and added a six-layer, 1,024-dim autoregressive depth decoder to predict the codebooks within each audio frame. We trained the TTS models with identical batches in identical order. Only the token-specific embeddings and output heads changed to fit each tokenizer. At evaluation, each model received text without a reference speech prompt, and we used three generation seeds.

Results

For evaluation, we picked LibriSpeech test-clean, one of the most common benchmarks for codec reconstruction, added Seed-TTS English and Mandarin, and extended the coverage with FLEURS in German, French, Mandarin, Japanese, and Korean, matching the languages we train TTS on.

Reconstruction and speech generation

In addition to being much more computationally efficient than its competitors, LiquidCodec matches their reconstruction quality. It achieves the best perceived audio quality measured by ViSQOL and the lowest mel distance on all eight test sets while remaining competitive on the other metrics. Decoder-only QAT can also reduce its decoder footprint by a factor of 3.5 in Q4_0 with minimal impact on audio reconstruction quality. Even quantized, LiquidCodec keeps its ViSQOL and mel-distance lead over the external codecs. 

For zero-shot multilingual TTS, models trained on LiquidCodec tokens reach the highest naturalness score (UTMOSv2) on all eight test sets and the best intelligibility (WER/CER) on four out of the eight datasets. Notably, these findings are not LFM backbone-specific. Repeating the TTS experiment with a Qwen3-1.7B [12] backbone gives consistent results.

Speed

At FP32 precision with single-threaded CPU inference on a MacBook Pro with an Apple M4 Max, LiquidCodec encodes and decodes at approximately 361× and 87× real time, respectively, much faster than its competitors.

Conclusion

LiquidCodec is a step toward making high-quality voice interaction a standard part of on-device AI. Faster speech generation with a smaller memory footprint makes it easier to build applications that can sustain a conversation while running locally.

Building the codec in-house also gives us control over a foundational part of our audio models. We can now evolve the speech representation alongside the models that use it. This opens up more possibilities for how future LFMs learn to generate and interact through speech.

LiquidCodec is available on Hugging Face and loads directly with Transformers. To run it on-device, the LFM3-Audio-2B and LFM3-Omni-2B GGUFs include quantized Q4_0, Q4_K_M, and Q8_0 builds for llama.cpp. 

Acknowledgements

Research and development led by Anna Piunova and Calder Katyal, with development contributions by Ishwarya Ananthabhotla. Further contributions from Alexander Amini, Marc Härkönen, Tarek Dakhran, Samuel J. Broughton, Saniya Karwa, Farris Atif and Leonie Monigatti.

Citation

Please cite this article as:

@article{liquidAI2026liquidcodec,
  author = {Liquid AI},
  title = {LiquidCodec: High-Fidelity Audio Tokenization Optimized for Speed},
  journal = {Liquid AI Blog},
  year = {2026},
  note = {www.liquid.ai/blog/liquidcodec},
}

References

[1] Liquid AI (2026). Introducing LFM3: Every modality for specialized intelligence. Liquid AI Blog. https://www.liquid.ai/blog/lfm3

[2] Défossez, A., Mazaré, L., Orsini, M., Royer, A., Pérez, P., Jégou, H., Grave, E., & Zeghidour, N. (2024). Moshi: A speech-text foundation model for real-time dialogue. arXiv preprint. https://arxiv.org/abs/2410.00037

[3] Juang, B.-H., & Gray, A. H. (1982). Multiple stage vector quantization for speech coding. ICASSP 1982. https://doi.org/10.1109/ICASSP.1982.1171604

[4] Zeghidour, N., Luebs, A., Omran, A., Skoglund, J., & Tagliasacchi, M. (2022). SoundStream: An end-to-end neural audio codec. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 30, 495–507. https://arxiv.org/abs/2107.03312

[5] Mentzer, F., Minnen, D., Agustsson, E., & Tschannen, M. (2024). Finite scalar quantization: VQ-VAE made simple. ICLR 2024. https://arxiv.org/abs/2309.15505

[6] Liao, S., Wang, Y., Li, T., Cheng, Y., Zhang, R., Zhou, R., & Xing, Y. (2024). Fish-Speech: Leveraging large language models for advanced multilingual text-to-speech synthesis. arXiv preprint. https://arxiv.org/abs/2411.01156

[7] Chang, H., Zhang, H., Jiang, L., Liu, C., & Freeman, W. T. (2022). MaskGIT: Masked generative image transformer. CVPR 2022. https://arxiv.org/abs/2202.04200

[8] Fejgin, R., Neekhara, P., Yang, X., Casanova, E., Langman, R., Kim, J., Ghosh, S., Hussain, S., & Li, J. (2025). Frame-stacked local transformers for efficient multi-codebook speech generation. arXiv preprint. https://arxiv.org/abs/2509.19592

[9] Ye, Z., Zhu, X., Chan, C.-M., Wang, X., Tan, X., Lei, J., Peng, Y., Liu, H., Jin, Y., Dai, Z., Lin, H., Chen, J., Du, X., Xue, L., Chen, Y., Li, Z., Xie, L., Kong, Q., Guo, Y., & Xue, W. (2025). Llasa: Scaling train-time and inference-time compute for Llama-based speech synthesis. arXiv preprint. https://arxiv.org/abs/2502.04128

[10] Casanova, E., Neekhara, P., Langman, R., Hussain, S., Ghosh, S., Yang, X., Jukić, A., Li, J., & Ginsburg, B. (2025). NanoCodec: Towards high-quality ultra fast speech LLM inference. Interspeech 2025. https://arxiv.org/abs/2508.05835

[11] Jacob, B., Kligys, S., Chen, B., Zhu, M., Tang, M., Howard, A., Adam, H., & Kalenichenko, D. (2018). Quantization and training of neural networks for efficient integer-arithmetic-only inference. CVPR 2018. https://arxiv.org/abs/1712.05877

[12] Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., et al. (2025). Qwen3 technical report. arXiv preprint. https://arxiv.org/abs/2505.09388