Running Llama 4 Locally on an M4 Mac: A Practical Guide for Bootstrappers

You are a solo developer or a small team building an AI-powered product. You are spending $200-$500/month on API calls to OpenAI, Anthropic, or Google. Your margins are thin. Your runway is finite. And a significant portion of those API calls are for tasks that do not require frontier model intelligence — code completion, text classification, data extraction, draft generation, summarization.

What if you could run those tasks locally, at zero marginal cost, with data that never leaves your machine?

With Apple's M4 chip family and Meta's Llama 4 models, that is now not just possible — it is practical. This guide walks you through everything you need to know: hardware requirements, software setup, which tasks work well locally, which still need cloud APIs, and the real-world cost savings you can expect.

No hype. No "local AI will replace the cloud" nonsense. Just a practical, honest guide for bootstrappers who want to cut costs without cutting corners.

Why Run Local: The Four Advantages

1. Zero Marginal Cost After Hardware

Once you own the hardware, every inference is free. No per-token billing. No surprise charges when a user triggers a long conversation. No anxiety about cost spikes during peak usage. For a bootstrapped company, this predictability is worth its weight in gold.

The math is simple: if you are spending $300/month on API calls and can offload 60% of those calls to local inference, you save $180/month — $2,160/year. That is meaningful for a company counting every dollar. And the savings scale: as your usage grows, local inference costs stay flat while API costs scale linearly.

2. Data Never Leaves Your Machine

Data privacy is not just a compliance checkbox — it is a competitive advantage. When you run inference locally, sensitive data (customer information, proprietary code, financial documents, medical records) never hits a third-party API. This eliminates:

• Data processing agreements with AI providers
• Compliance concerns around data residency and sovereignty
• The risk of training data being used to improve third-party models
• Customer objections to their data being sent to external AI services

For developers building products in regulated industries (healthcare, finance, legal), local inference can be the difference between "possible" and "not possible" for certain features.

3. No Rate Limits

Cloud API rate limits are the silent killer of AI-powered features. You build a great feature, users love it, adoption spikes — and suddenly you are hitting 429 errors because your usage exceeded the provider's rate limits. Local inference has no rate limits beyond your hardware's throughput.

4. Offline Capability

Local models work without an internet connection. Build AI features that work on airplanes, in areas with poor connectivity, or in secure environments where internet access is restricted. This is a niche advantage, but when you need it, nothing else will do.

Hardware Requirements: What You Actually Need

The M4 Lineup for Local Inference

Apple's M4 chip family is uniquely suited for local LLM inference because of its unified memory architecture. Unlike discrete GPU setups where data must be copied between CPU and GPU memory, M4 chips share memory between CPU and GPU cores, eliminating the biggest bottleneck in local inference.

Here is what each tier gives you:

M4 (base) — 16GB RAM minimum

• Usable for small models (7B-8B parameter quantized)
• Inference speed: 10-15 tokens/second for 8B Q4 models
• Verdict: Functional for experimentation, too slow for production use

M4 Pro — 24GB or 36GB RAM

• The minimum viable option for practical local inference
• 24GB: Run 8B-13B models comfortably, squeeze in small 30B quantized models
• 36GB: Run 30B+ models at usable speeds
• Inference speed: 20-35 tokens/second for 8B Q4 models, 8-15 tokens/second for 30B Q4 models
• Verdict: Good for solo developers and small teams. The sweet spot for cost-conscious bootstrappers

M4 Max — 48GB, 64GB, or 128GB RAM

• The recommended option for serious local inference workloads
• 48GB: Run 30B-70B quantized models comfortably
• 64GB-128GB: Run larger models, multiple models simultaneously, or higher quantization levels
• Inference speed: 30-50 tokens/second for 8B Q4, 15-25 tokens/second for 30B Q4, 5-10 tokens/second for 70B Q4
• Verdict: The best option if you can afford it. Makes local inference feel like a real development tool, not a toy

Key insight: RAM is the bottleneck, not GPU. Every parameter in the model needs to fit in memory. A 7B parameter model at Q4 quantization requires approximately 4GB of RAM. A 70B model at Q4 requires approximately 40GB. Buy the most RAM you can afford.

Setup Guide: From Zero to Local Inference

Step 1: Choose Your Runtime

Two main options for running LLMs locally on Mac:

Ollama (Recommended for Most Users)

Ollama is the easiest way to run local models. It handles model downloading, quantization selection, and Metal GPU acceleration automatically.

• Install: brew install ollama
• Start the server: ollama serve
• Pull a model: ollama pull llama4 (pulls the default quantization)
• Run interactively: ollama run llama4
• API endpoint: http://localhost:11434/api/generate (OpenAI-compatible API available)

Ollama's advantage is simplicity. It abstracts away the complexity of model formats, quantization levels, and GPU configuration. For most developers, this is the right choice.

llama.cpp (Recommended for Power Users)

llama.cpp gives you more control over inference parameters, quantization, and performance tuning.

• Clone the repo: git clone https://github.com/ggerganov/llama.cpp
• Build with Metal support: cd llama.cpp && make LLAMA_METAL=1
• Download a GGUF model from Hugging Face
• Run: ./llama-cli -m model.gguf -p "Your prompt here" -n 512 --gpu-layers 99

llama.cpp's advantage is performance. Direct control over GPU layer offloading, context size, batch processing, and memory management lets you squeeze more performance out of your hardware.

Step 2: Choose Your Model and Quantization

Llama 4 comes in several sizes. Here is how to choose:

Llama 4 Scout (smallest) — Best for: code completion, classification, simple extraction. Fits easily on M4 Pro with room to spare.

Llama 4 Maverick (medium) — Best for: summarization, draft generation, conversational AI. Requires M4 Pro 36GB or M4 Max.

Llama 4 Behemoth (largest) — Best for: complex reasoning, multi-step analysis. Requires M4 Max 128GB or will not fit in memory at usable quantization levels.

For quantization levels, Q4_K_M is the sweet spot for most use cases. It provides 95%+ of full-precision quality at roughly 25% of the memory footprint. Here is the quantization spectrum:

• Q2_K: Smallest, lowest quality. Only use if you absolutely cannot fit a larger quantization.
• Q4_K_M: The sweet spot. Best balance of quality and size for most applications.
• Q5_K_M: Slightly better quality, 25% more memory. Use if you have the RAM to spare.
• Q6_K: Near-full quality, significantly more memory. Only on M4 Max with ample RAM.
• Q8_0: Highest practical quantization. Minimal quality loss but requires double the memory of Q4.

Step 3: Configure for Optimal M4 Metal Performance

Apple's Metal framework is the key to getting good performance on M4 chips. Both Ollama and llama.cpp support Metal acceleration, but there are configuration tweaks that make a significant difference:

GPU Layer Offloading: Set the number of GPU layers to the maximum your memory allows. Every layer that runs on the GPU (via Metal) is dramatically faster than CPU inference. In Ollama, this is handled automatically. In llama.cpp, use --gpu-layers 99 (it will use as many as memory allows).

Context Size: Larger context windows require more memory. If you are doing simple tasks (classification, extraction), reduce context size to 2048 or 4096 tokens. Reserve larger contexts (8192+) for tasks that genuinely need them (long document summarization, multi-turn conversation).

Batch Size: For throughput-sensitive applications, increase the batch size. This allows the model to process multiple tokens in parallel, improving tokens/second at the cost of slightly higher latency for individual requests.

Memory Locking: On macOS, use --mlock in llama.cpp to prevent the model from being swapped to disk. Disk swapping destroys inference performance.

Which Tasks Work Well Locally

Local inference is not a replacement for cloud APIs — it is a complement. Here are the tasks where local models excel:

Tasks That Shine Locally

• Code completion and suggestion: Low-latency, high-frequency task. Local inference eliminates the API round-trip and provides instant suggestions.
• Text classification: Categorizing support tickets, tagging content, sentiment analysis. These tasks do not require frontier intelligence.
• Data extraction: Pulling structured data from unstructured text. Names, dates, amounts, addresses — local models handle these well.
• Summarization: Generating summaries of documents, emails, meeting notes. Quality is very good for extractive and mildly abstractive summarization.
• Draft generation: First drafts of emails, documentation, reports. The output needs human editing anyway, so frontier quality is unnecessary.
• Embeddings: Running a local embedding model for RAG applications. Zero marginal cost and complete data privacy.
• JSON/structured output: Generating structured responses for data pipelines. Local models handle constrained output formats reliably.

Tasks That Still Need Cloud APIs

• Complex multi-step reasoning: Tasks requiring chain-of-thought across many steps still benefit from frontier models.
• Very long context: Processing 100K+ token inputs exceeds local memory for most configurations.
• Multimodal tasks: Image understanding, audio processing, and video analysis are better handled by cloud APIs with specialized infrastructure.
• Real-time applications with strict latency requirements: If you need sub-100ms response times for user-facing features, cloud APIs with dedicated infrastructure are more reliable.
• Tasks requiring the absolute latest knowledge: Local models have a knowledge cutoff. Tasks requiring current information need cloud APIs or RAG.

Cost Comparison: Real Numbers

Let us run the numbers for a solo developer making 100 queries per day:

Cloud API costs (estimated at GPT-4-class pricing):

• 100 queries/day x 30 days = 3,000 queries/month
• Average 500 input tokens + 300 output tokens per query
• Estimated cost: $15-$25/month for a mix of simple and complex tasks

Local inference costs:

• Hardware: MacBook Pro M4 Pro 36GB — $2,799 (one-time, and you need a laptop anyway)
• Electricity: Approximately $3-5/month for inference workloads
• Marginal cost per query: Effectively $0

Blended approach (recommended):

• Run 60% of queries locally (simple tasks): $0/month
• Run 40% of queries via cloud API (complex tasks): $8-$12/month
• Total: $8-$12/month versus $15-$25/month pure cloud
• Savings: 40-60% on API costs

For teams with higher volume (1,000+ queries/day) or using more expensive models, the savings are proportionally larger. Teams spending $500/month on API calls can realistically cut that to $150-$200 with a local/cloud hybrid approach.

Common Pitfalls and How to Avoid Them

Pitfall 1: Expecting Cloud-Quality Responses for Every Task

The biggest mistake bootstrappers make is running a local model, comparing its output to GPT-4 or Claude Opus on a complex reasoning task, and concluding that local models are not ready. Local models are not meant to replace frontier models for every task. They are meant to handle the 50-70% of your workload that does not require frontier intelligence. Set expectations appropriately, and local inference becomes a valuable tool rather than a disappointment.

Pitfall 2: Not Monitoring Memory Pressure

macOS will happily swap model weights to disk if you run out of physical RAM, and your inference speed will drop from 30 tokens/second to 2 tokens/second. Use Activity Monitor or memory_pressure in the terminal to verify your system has enough free memory. If you are running other memory-intensive applications (Chrome with 50 tabs, Docker containers, Xcode), you may need to close them before running inference.

Pitfall 3: Ignoring Model Selection

Not all tasks need the same model. Running Llama 4 Maverick for text classification is overkill — Scout handles it faster with similar accuracy. Build a routing layer that sends simple tasks to smaller models and complex tasks to larger ones (or to cloud APIs). This maximizes both speed and quality.

Integration with Development Workflows

VS Code Integration

Ollama's OpenAI-compatible API makes it a drop-in replacement for tools that support custom API endpoints. Popular VS Code extensions like Continue and Cody can be configured to use a local Ollama endpoint for code completion and chat, giving you an AI-assisted coding experience with zero API costs.

Terminal Workflows

Build shell aliases and scripts that pipe data through local models:

• classify — pipe text to a local model for categorization
• summarize — generate summaries of files or clipboard content
• extract — pull structured data from unstructured input
• translate — translate text between languages locally

These one-liners become incredibly powerful when you do not have to worry about API costs or rate limits. You can run them hundreds of times a day without thinking about billing.

Local RAG Setup

Combine a local LLM with a local vector database (Chroma, LanceDB) and a local embedding model for a complete RAG stack that runs entirely on your machine. This is especially valuable for developers working with sensitive data that cannot be sent to third-party APIs.

The local RAG stack: local embedding model → LanceDB/Chroma → Llama 4 via Ollama → your application. Total cloud cost: $0. Total data exposure: none.

For more deep dives on AI engineering topics like this, tune into the TBPN daily show (11 AM - 2 PM PT on YouTube and X). And if you are the kind of developer who runs models locally to save money, you will appreciate the TBPN sticker pack for your laptop — because even bootstrappers need a little flair. Check out the full collection of drinkware and t-shirts at merchtbpn.com.

Performance Benchmarks: M4 Pro vs M4 Max

Real-world benchmarks for common tasks (using Llama 4 Scout Q4_K_M):

Code completion (short prompt, short output):

• M4 Pro 36GB: 28 tokens/second, 1.2s to first token
• M4 Max 64GB: 42 tokens/second, 0.7s to first token

Text summarization (1000-token input, 200-token output):

• M4 Pro 36GB: 24 tokens/second, 1.8s to first token
• M4 Max 64GB: 38 tokens/second, 1.0s to first token

Data extraction (500-token input, structured JSON output):

• M4 Pro 36GB: 26 tokens/second, 1.4s to first token
• M4 Max 64GB: 40 tokens/second, 0.8s to first token

The M4 Max is roughly 50% faster than the M4 Pro for most inference tasks, primarily due to higher memory bandwidth (which is the bottleneck for LLM inference on Apple Silicon). Whether that premium is worth it depends on how much you value inference speed for your specific workflow.

Frequently Asked Questions

Can I run Llama 4 on an older M1 or M2 Mac?

Yes, but with significantly reduced performance. M1 and M2 chips have lower memory bandwidth, which is the primary bottleneck for LLM inference. Expect roughly 40-60% of the tokens/second compared to equivalent M4 configurations. The M1 Max and M2 Max with 64GB+ RAM are still viable options — just slower. The base M1 and M2 with 8-16GB RAM are not practical for anything beyond tiny models.

Is the quality of local Llama 4 good enough to replace cloud APIs?

For the tasks listed in the "works well locally" section, yes — Llama 4's quality is competitive with cloud API outputs. For complex reasoning, creative writing, and tasks requiring the absolute best quality, cloud frontier models still have an edge. The practical approach is a hybrid: route simple tasks locally and complex tasks to cloud APIs. Most developers find that 50-70% of their API calls can be handled locally without noticeable quality degradation.

How much disk space do the models require?

Llama 4 Scout Q4_K_M requires approximately 5-6GB of disk space. Llama 4 Maverick Q4_K_M requires approximately 20-25GB. Llama 4 Behemoth Q4_K_M requires approximately 60-80GB. Ollama stores models in its cache directory, and you can have multiple models downloaded simultaneously. Plan for 50-100GB of disk space if you want to experiment with several model sizes and quantization levels.

Can I use local models for a production application serving users?

For low-traffic applications (under 10 concurrent users), a single M4 Max can serve as a viable inference server. For higher traffic, you would need multiple machines or a load-balanced setup, at which point cloud APIs often become more cost-effective due to their elastic scaling. The sweet spot for local production inference is internal tools, developer productivity applications, and low-traffic B2B products where data privacy is the primary concern.