How to Run a Large Language Model on Your Mac Effortlessly

In the era of tools like ChatGPT, Claude, and Gemini, wouldn’t it be great to have something similar running on your own computer? While those models are incredibly large and require significant computing power, there are smaller models that can run locally. They might not be as advanced, but they’re still quite useful for many tasks.

In this article, I’ll guide you through setting up a large language model on your macOS system. To follow along, you’ll need a Mac with an M1 chip or newer. The more RAM your system has, the smoother the experience will be.

One of the tools that makes this incredibly easy is Ollama—it simplifies the process to just one command.

The best part? All your data stays local. No cloud dependency, no external servers—just your machine and your data.

Let’s get started by installing it.

---

Install Ollama

Visit Ollama’s official site and download the macOS installen. Open the .dmg file and follow the on-screen instructions to complete the installation.

After the installation, let’s verify that everything is set up correctly. Open your terminal and run the following command:

ollama --version

If the installation was successful, you should see an output like this:

ollama version is 0.5.4

---

Using Ollama Models

Once you’ve installed a model, you can start interacting with it directly through the terminal. For example:

ollama run llama3.2

If the model isn’t already installed, this command will first download Llama3.2. Keep in mind that the model is approximately 2GB in size, so the download might take a few moments.

Once the download is complete, you can have a little chat with the model. Let’s start with an easy question:

>>> What's the capital of Germany?
The capital of Germany is Berlin.

Not too difficult, but still pretty cool, right?

After writing this article, I’m starting to feel a little hungry. Let’s see if it can help with that by providing a recipe for Spaghetti Carbonara:

>>> What ingredients are in Spaghetti Carbonara, please provide a detailed repice.
Spaghetti Carbonara is a classic Italian dish made with just a few high-quality ingredients. Here's a detailed recipe:

Ingredients:

* 12 oz (340g) spaghetti
* 4 oz (115g) guanciale or pancetta, diced (you can also use bacon as a substitute)
* 3 large eggs
* 1 cup (250ml) grated Parmigiano-Reggiano cheese
* Salt and black pepper, to taste
* Freshly ground nutmeg, to taste

Optional ingredients:

* Fresh parsley, chopped (for garnish)
* Extra virgin olive oil (for serving)

Recipe:

1. Bring a large pot of salted water to a boil and cook the spaghetti according to package instructions until al dente. Reserve 1 cup of pasta water before draining.
2. In a large skillet, cook the guanciale or pancetta over medium heat until crispy. Remove from heat and let cool slightly.
3. In a medium bowl, whisk together the eggs, Parmigiano-Reggiano cheese, and a pinch of salt and pepper until well combined.
4. Add the cooked spaghetti to the bowl with the egg mixture and toss everything together until the spaghetti is well coated.
5. Add the reserved pasta water to the bowl in small increments and continue to toss until the spaghetti is coated in a creamy sauce.
6. Add the cooled guanciale or pancetta to the bowl and toss everything together until the guanciale is evenly distributed throughout the spaghetti.
7. Season with salt, black pepper, and nutmeg to taste.
8. Serve immediately, garnished with chopped parsley if desired.

 Notes:

* Carbonara should never contain any cream, as it's not a traditional ingredient in Italian cuisine.
* The key to a creamy Carbonara is using high-quality Parmigiano-Reggiano cheese and whisking the eggs until they're well combined with the cheese.
* Guanciale or pancetta adds a rich, savory flavor to the dish, but you can substitute bacon if you prefer.
* Reserve pasta water before draining the spaghetti to help create a creamy sauce.

So, there you have it—a quick introduction to running AI models locally on your Mac. Give it a try, and you might be surprised at just how useful (and entertaining) these tools can be.

Meanwhile, I’m heading to my kitchen. If my Carbonara turns out great, I’ll credit the model. If not, well, I’ll blame my cooking skills. 😅👨‍🍳👋