# Run a Hugging Face model on your Mac with MLX > Install mlx-lm in a Python venv, run a model from the mlx-community org on Apple silicon, pick 4-bit or 8-bit for your Mac's memory, and convert your own model, with real speed and memory numbers from an M4. - Canonical page: https://obelisk.club/blog/run-hugging-face-models-mac-mlx - Author: Peter Zhang - Published: 2026-11-02 - Tags: mlx, hugging-face, apple-silicon, mac, language-model, indie Create a Python virtual environment, run `pip install mlx-lm`, then call `mlx_lm.generate --model mlx-community/ --prompt "..."`. The first run downloads the model from Hugging Face; later runs reuse the local copy. MLX is Apple's array framework for machine learning on Apple silicon, and the `mlx-community` organization on Hugging Face hosts thousands of models already converted for it. Pick a 4-bit file when memory is tight and an 8-bit file when you want more headroom for quality. Everything below was run on a Mac with an Apple M4 and 16 GB of memory (`sysctl` reported `Apple M4` and `17179869184` bytes) on macOS 27.0.1, and the numbers are what the tools printed. ## What MLX and mlx-lm are The MLX README describes MLX as "an array framework for machine learning on Apple silicon, brought to you by Apple machine learning research". Two properties matter here. It has a Python API that closely follows NumPy, plus C++, C and Swift APIs. And arrays live in shared memory, so operations can run on the CPU or the GPU without copying data between them. `mlx-lm` is a separate Python package built on MLX. Its README says it is for "generating text and fine-tuning large language models on Apple silicon with MLX", with Hugging Face Hub integration and support for quantizing and uploading models. It gives you the command line tools used below: `mlx_lm.generate`, `mlx_lm.chat` and `mlx_lm.convert`. ## Install it in a virtual environment A venv keeps the packages out of your system Python. I used Python 3.14.7 from Homebrew (the macOS system Python here is 3.9.6, which I did not try): ```bash python3.14 -m venv mlx-venv ./mlx-venv/bin/pip install mlx-lm ./mlx-venv/bin/pip list | grep -i -E "^(mlx|mlx-lm|transformers|huggingface)" ``` The install pulled in these versions: ``` huggingface_hub 2.4.0 mlx 0.32.4 mlx-lm 0.32.2 mlx-metal 0.32.4 transformers 5.22.1 ``` Downloads go to the Hugging Face cache, `~/.cache/huggingface` by default. To keep a test run self-contained, set `HF_HOME` to a folder you can delete afterwards; I did this, and the model files landed under it instead of in the home folder: ```bash export HF_HOME="$PWD/hf-home" ``` ## Generate text with a model from mlx-community The `--model` flag takes a local folder or a Hugging Face repo name. If you leave it out, `mlx-lm` uses `mlx-community/Llama-3.2-3B-Instruct-4bit`, according to its README and `--help`. I used a small one so the download is quick. Before writing this I checked that `mlx-community/Qwen2.5-0.5B-Instruct-4bit` and `-8bit` exist in the Hugging Face model list (the Qwen2.5-0.5B-Instruct card lists the Apache 2.0 license; read the card of any model you ship): ```bash ./mlx-venv/bin/mlx_lm.generate \ --model mlx-community/Qwen2.5-0.5B-Instruct-4bit \ --prompt "Explain in two sentences what an Apple silicon Mac is." \ --max-tokens 100 --seed 0 ``` Printed (excerpt): ``` ========== An Apple silicon Mac is a powerful and versatile computer that combines the best features of Apple hardware with the latest software and software development tools to create a cutting-edge computing experience. It's a powerful and versatile computer ... ========== Prompt: 41 tokens, 435.835 tokens-per-sec Generation: 66 tokens, 225.996 tokens-per-sec Peak memory: 0.361 GB ``` The summary lines are the useful part: prompt speed, generation speed and peak memory. A model this small repeats itself, which is normal. For an interactive session, `mlx_lm.chat` keeps the conversation context for as long as it runs: ```bash ./mlx-venv/bin/mlx_lm.chat --model mlx-community/Qwen2.5-0.5B-Instruct-4bit --max-tokens 60 ``` I fed it one question and it answered at 224.9 tokens per second with a 0.35 GB peak. It also got the answer wrong: asked "What is MLX?", the 0.5B model described MLflow and TensorFlow. Tiny models are good for checking your setup and speed, not for facts, so test a size you actually plan to use. `mlx_lm.generate -h` lists the rest of the options, including sampling flags such as `--temp` and `--top-p`. ## Choose 4-bit or 8-bit for your Mac's memory Quantizing stores each weight in fewer bits, so the file and the memory it needs shrink. The `mlx-lm` README says only two things about fit: models that are large relative to your RAM can be slow, and when you see the warning "Generating with a model that requires ...", you can raise the system wired memory limit with `sudo sysctl iogpu.wired_limit_mb=N`, where N should be larger than the model's size in megabytes but smaller than the machine's memory. That wiring feature needs macOS 15 or newer. The README gives no table of which model fits which Mac, so measure, as below. Same prompt, same seed, same Mac (Apple M4, 16 GB), each run twice; both runs printed the same peak memory and similar speeds: | Model | Weights file | Peak memory | Generation speed | |---|---|---|---| | Qwen2.5-0.5B-Instruct, original (bf16) | 942 MB | not run | not run | | mlx-community/Qwen2.5-0.5B-Instruct-8bit | 501 MB | 0.609 GB | 147 to 153 tokens/sec | | mlx-community/Qwen2.5-0.5B-Instruct-4bit | 265 MB | 0.361 GB | 226 to 240 tokens/sec | On this Mac and this model, 4-bit used about 40 percent less peak memory than 8-bit and generated faster. Those are measurements of one 0.5B model, not a promise for other sizes. The 8-bit and 4-bit runs also stopped after different numbers of tokens (36 and 66), so the speeds compare roughly, not exactly. Practical steps: - Look at the model's file size on its Hugging Face page. Peak memory in my runs was larger than the weights file (0.361 GB for a 265 MB file), so leave room for more than the file size. - Start with 4-bit, check the output quality on your own prompts, and move up to 8-bit only if 4-bit is not good enough. - Remember the rest of the system needs memory too. A model that nearly fills your RAM is the case the README warns will be slow. ## Convert a model yourself with mlx_lm.convert If a model has no MLX version, convert it. The README shows `mlx_lm.convert --model -q` to quantize from the command line, and `--upload-repo` to push the result to Hugging Face. I converted the original `Qwen/Qwen2.5-0.5B-Instruct`, saving to a folder I chose with `--mlx-path`: ```bash ./mlx-venv/bin/mlx_lm.convert \ --hf-path Qwen/Qwen2.5-0.5B-Instruct \ -q \ --mlx-path ./qwen-4bit-mine ``` It printed `Using dtype: bfloat16`, `Quantizing` and `Quantized model with 4.502 bits per weight.` The folder held a 278,065,225-byte `model.safetensors` (290 MB for the whole folder) next to the config and tokenizer files. The output is above 4 bits per weight because quantization also stores per-group scale values. Running it by path works the same way: ```bash ./mlx-venv/bin/mlx_lm.generate --model ./qwen-4bit-mine --prompt "Say hello in five words." --max-tokens 30 --seed 0 ``` It answered `Hello, how may I assist you today?` at 182.8 tokens per second with a 0.350 GB peak. `mlx_lm.convert --help` also lists `--q-bits` and `--q-group-size`; I did not change them. Keep the original model's license with the converted files, because a conversion is a derived copy. ## Where MLX Swift fits for apps `mlx-lm` is Python, so it suits your Mac and scripts, not an iOS or macOS app you ship. For apps, the MLX README lists a Swift API, and the MLX Swift README calls MLX Swift "a Swift API for MLX". You add `https://github.com/ml-explore/mlx-swift.git` as a package dependency in Xcode or SwiftPM. The README notes that SwiftPM on the command line cannot build the Metal shaders, so the final build has to go through Xcode. Language model code lives in a separate package. The `mlx-swift-examples` README says `MLXLMCommon`, `MLXLLM`, `MLXVLM` and `MLXEmbedders` moved to [mlx-swift-lm](https://github.com/ml-explore/mlx-swift-lm), which describes itself as a Swift package for building tools and applications with large language models and vision language models. Its README shows a `Package.swift` dependency and products to link (`MLXLLM`, `MLXLMCommon`, `MLXHuggingFace`, plus the `HuggingFace` and `Tokenizers` products from two Hugging Face packages). The examples repo has an app that runs on both iOS and macOS and downloads a model from Hugging Face (`LLMEval`) and a small chat app (`MLXChatExample`). I did not build any of these for this guide, so check the current README versions before copying a dependency line. For the Core ML alternative, see [Using a Hugging Face model in an iOS app](/blog/hugging-face-models-in-ios-apps-core-ml). ## Clean up A test run is easy to remove. I deleted only the folders under my scratch directory: the `hf-home` cache (1.7 GB for the original and the two quantized Qwen files), the converted folder and the venv (474 MB). Nothing was left in `~/.cache`. If you did not set `HF_HOME`, the models are in `~/.cache/huggingface/hub`. To look for more models to try, see the [models page](/models). ## Sources All read on 2 November 2026, and every command above was run on the Mac described at the top. - [ml-explore/mlx README](https://github.com/ml-explore/mlx), what MLX is, the NumPy-like Python API and unified memory - [ml-explore/mlx-lm README](https://github.com/ml-explore/mlx-lm), install, `mlx_lm.generate`, `mlx_lm.chat`, `mlx_lm.convert -q`, the default model and the Large Models note on the wired memory limit - [mlx-community on Hugging Face](https://huggingface.co/mlx-community), the organization hosting converted models - [ml-explore/mlx-swift README](https://github.com/ml-explore/mlx-swift), the Swift API, Xcode and SwiftPM installation, and the note on Metal shaders - [ml-explore/mlx-swift-examples README](https://github.com/ml-explore/mlx-swift-examples), `LLMEval`, `MLXChatExample` and the move of the language model libraries - [ml-explore/mlx-swift-lm README](https://github.com/ml-explore/mlx-swift-lm), the package for language models in Swift - [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct), the model converted above and its license metadata