Run a Hugging Face model on your Mac with MLX

Create a Python virtual environment, run pip install mlx-lm, then call mlx_lm.generate --model mlx-community/<model> --prompt "...". The first run downloads the model from Hugging Face; later runs reuse the local copy. MLX is Apple's array framework for machine learning on Apple silicon, and the mlx-community organization on Hugging Face hosts thousands of models already converted for it. Pick a 4-bit file when memory is tight and an 8-bit file when you want more headroom for quality. Everything below was run on a Mac with an Apple M4 and 16 GB of memory (sysctl reported Apple M4 and 17179869184 bytes) on macOS 27.0.1, and the numbers are what the tools printed.

What MLX and mlx-lm are

The MLX README describes MLX as "an array framework for machine learning on Apple silicon, brought to you by Apple machine learning research". Two properties matter here. It has a Python API that closely follows NumPy, plus C++, C and Swift APIs. And arrays live in shared memory, so operations can run on the CPU or the GPU without copying data between them.

mlx-lm is a separate Python package built on MLX. Its README says it is for "generating text and fine-tuning large language models on Apple silicon with MLX", with Hugging Face Hub integration and support for quantizing and uploading models. It gives you the command line tools used below: mlx_lm.generate, mlx_lm.chat and mlx_lm.convert.

Install it in a virtual environment

A venv keeps the packages out of your system Python. I used Python 3.14.7 from Homebrew (the macOS system Python here is 3.9.6, which I did not try):

python3.14 -m venv mlx-venv
./mlx-venv/bin/pip install mlx-lm
./mlx-venv/bin/pip list | grep -i -E "^(mlx|mlx-lm|transformers|huggingface)"

The install pulled in these versions:

huggingface_hub   2.4.0
mlx               0.32.4
mlx-lm            0.32.2
mlx-metal         0.32.4
transformers      5.22.1

Downloads go to the Hugging Face cache, ~/.cache/huggingface by default. To keep a test run self-contained, set HF_HOME to a folder you can delete afterwards; I did this, and the model files landed under it instead of in the home folder:

export HF_HOME="$PWD/hf-home"

Generate text with a model from mlx-community

The --model flag takes a local folder or a Hugging Face repo name. If you leave it out, mlx-lm uses mlx-community/Llama-3.2-3B-Instruct-4bit, according to its README and --help. I used a small one so the download is quick. Before writing this I checked that mlx-community/Qwen2.5-0.5B-Instruct-4bit and -8bit exist in the Hugging Face model list (the Qwen2.5-0.5B-Instruct card lists the Apache 2.0 license; read the card of any model you ship):

./mlx-venv/bin/mlx_lm.generate \
  --model mlx-community/Qwen2.5-0.5B-Instruct-4bit \
  --prompt "Explain in two sentences what an Apple silicon Mac is." \
  --max-tokens 100 --seed 0

Printed (excerpt):

==========
An Apple silicon Mac is a powerful and versatile computer that combines the best features of Apple hardware with the latest software and software development tools to create a cutting-edge computing experience. It's a powerful and versatile computer ...
==========
Prompt: 41 tokens, 435.835 tokens-per-sec
Generation: 66 tokens, 225.996 tokens-per-sec
Peak memory: 0.361 GB

The summary lines are the useful part: prompt speed, generation speed and peak memory. A model this small repeats itself, which is normal. For an interactive session, mlx_lm.chat keeps the conversation context for as long as it runs:

./mlx-venv/bin/mlx_lm.chat --model mlx-community/Qwen2.5-0.5B-Instruct-4bit --max-tokens 60

I fed it one question and it answered at 224.9 tokens per second with a 0.35 GB peak. It also got the answer wrong: asked "What is MLX?", the 0.5B model described MLflow and TensorFlow. Tiny models are good for checking your setup and speed, not for facts, so test a size you actually plan to use. mlx_lm.generate -h lists the rest of the options, including sampling flags such as --temp and --top-p.

Choose 4-bit or 8-bit for your Mac's memory

Quantizing stores each weight in fewer bits, so the file and the memory it needs shrink. The mlx-lm README says only two things about fit: models that are large relative to your RAM can be slow, and when you see the warning "Generating with a model that requires ...", you can raise the system wired memory limit with sudo sysctl iogpu.wired_limit_mb=N, where N should be larger than the model's size in megabytes but smaller than the machine's memory. That wiring feature needs macOS 15 or newer. The README gives no table of which model fits which Mac, so measure, as below.

Same prompt, same seed, same Mac (Apple M4, 16 GB), each run twice; both runs printed the same peak memory and similar speeds:

On this Mac and this model, 4-bit used about 40 percent less peak memory than 8-bit and generated faster. Those are measurements of one 0.5B model, not a promise for other sizes. The 8-bit and 4-bit runs also stopped after different numbers of tokens (36 and 66), so the speeds compare roughly, not exactly. Practical steps:

• Look at the model's file size on its Hugging Face page. Peak memory in my runs was larger than the weights file (0.361 GB for a 265 MB file), so leave room for more than the file size.
• Start with 4-bit, check the output quality on your own prompts, and move up to 8-bit only if 4-bit is not good enough.
• Remember the rest of the system needs memory too. A model that nearly fills your RAM is the case the README warns will be slow.

Convert a model yourself with mlx_lm.convert

If a model has no MLX version, convert it. The README shows mlx_lm.convert --model <repo> -q to quantize from the command line, and --upload-repo to push the result to Hugging Face. I converted the original Qwen/Qwen2.5-0.5B-Instruct, saving to a folder I chose with --mlx-path:

./mlx-venv/bin/mlx_lm.convert \
  --hf-path Qwen/Qwen2.5-0.5B-Instruct \
  -q \
  --mlx-path ./qwen-4bit-mine

It printed Using dtype: bfloat16, Quantizing and Quantized model with 4.502 bits per weight. The folder held a 278,065,225-byte model.safetensors (290 MB for the whole folder) next to the config and tokenizer files. The output is above 4 bits per weight because quantization also stores per-group scale values. Running it by path works the same way:

./mlx-venv/bin/mlx_lm.generate --model ./qwen-4bit-mine --prompt "Say hello in five words." --max-tokens 30 --seed 0

It answered Hello, how may I assist you today? at 182.8 tokens per second with a 0.350 GB peak. mlx_lm.convert --help also lists --q-bits and --q-group-size; I did not change them. Keep the original model's license with the converted files, because a conversion is a derived copy.

Where MLX Swift fits for apps

mlx-lm is Python, so it suits your Mac and scripts, not an iOS or macOS app you ship. For apps, the MLX README lists a Swift API, and the MLX Swift README calls MLX Swift "a Swift API for MLX". You add https://github.com/ml-explore/mlx-swift.git as a package dependency in Xcode or SwiftPM. The README notes that SwiftPM on the command line cannot build the Metal shaders, so the final build has to go through Xcode.

Language model code lives in a separate package. The mlx-swift-examples README says MLXLMCommon, MLXLLM, MLXVLM and MLXEmbedders moved to mlx-swift-lm, which describes itself as a Swift package for building tools and applications with large language models and vision language models. Its README shows a Package.swift dependency and products to link (MLXLLM, MLXLMCommon, MLXHuggingFace, plus the HuggingFace and Tokenizers products from two Hugging Face packages). The examples repo has an app that runs on both iOS and macOS and downloads a model from Hugging Face (LLMEval) and a small chat app (MLXChatExample). I did not build any of these for this guide, so check the current README versions before copying a dependency line. For the Core ML alternative, see Using a Hugging Face model in an iOS app.

Clean up

A test run is easy to remove. I deleted only the folders under my scratch directory: the hf-home cache (1.7 GB for the original and the two quantized Qwen files), the converted folder and the venv (474 MB). Nothing was left in ~/.cache. If you did not set HF_HOME, the models are in ~/.cache/huggingface/hub. To look for more models to try, see the models page.

Sources

All read on 2 November 2026, and every command above was run on the Mac described at the top.

• ml-explore/mlx README, what MLX is, the NumPy-like Python API and unified memory
• ml-explore/mlx-lm README, install, mlx_lm.generate, mlx_lm.chat, mlx_lm.convert -q, the default model and the Large Models note on the wired memory limit
• mlx-community on Hugging Face, the organization hosting converted models
• ml-explore/mlx-swift README, the Swift API, Xcode and SwiftPM installation, and the note on Metal shaders
• ml-explore/mlx-swift-examples README, LLMEval, MLXChatExample and the move of the language model libraries
• ml-explore/mlx-swift-lm README, the package for language models in Swift
• Qwen/Qwen2.5-0.5B-Instruct, the model converted above and its license metadata