I feel like I have written this post three times already, but this one is a different flavor of that same post.
Good news, everyone! Apple has announced its new desktop computers. On August 25th, the Mac Mini got the M6, which is Apple’s first 2 nm chip. It has a 12-core CPU, a 12-core GPU, and a dual 16-core neural engine. If those specifications don’t mean anything to you, that is perfectly fine. You probably have no idea what any of that means.
The Mac Studio is getting the M5 Ultra, not an M6 chip, which is a quad-die monster with 36 CPU cores, 80 GPU cores, and over 4.4 TB/s of bandwidth between its dies. It is also configurable up to 512 GB of unified memory, which is going to be (allegedly) available in October, but I suspect that this timeline will likely be pushed for most users, as memory availability is famously finicky at the moment.
Here’s what I want to say before I dive in on all the LinkedIn thought leaders: The Apple Silicon is legitimately fucking awesome. I’m thinking more in line with somebody who creates content and renders video. The speed at which these things can render video is crazy. It makes video editing and video rendering a breeze when you have that much unified memory and that much computing power. If you’re somebody who is a YouTuber and you make money making YouTube videos, this is gonna make your workflow crazy fast.
Additionally, I like Apple products. Yes, I have an Ubuntu daily driver and server that multitasks, and I like it, but I’m currently writing this on my MacBook Air, which is the best laptop I have ever owned. I write code on it every single day. I’m not going to sit here and dunk on Macs, because that’s not what I’m trying to do. I will continue owning and purchasing Apple computers because they are good. But I want a caveat that by saying they are good at personal computing. They are, first and foremost, and were engineered around the idea of you using this machine for yourself.
Here is what I do not like about the current Apple ecosystem, and it has almost nothing to do with Apple at all. Approximately 30 minutes after Apple announced their new hardware specifications, about a hundred thousand people with “AI thought leader” in their LinkedIn headline began trying to explain to us what hardware we need to purchase.
512GB of unified memory only matters in crazy edge cases.
I’m going to attempt to write something here that closely approximates the way that LinkedIn thought leaders like to format their posts about AI. Know that this is me trying to mimic AI. We’ll see how close I can get. Remember the rules:
Be as hyperbolic as possible while saying as little as possible.
Use emojis.
Don’t use paragraphs.
🚨 GAME CHANGER 🚨
Apple just quietly killed the AI datacenter.
512GB of unified memory.
You can now run a 400B parameter model. On your DESK. For less than a used Corolla.
Most companies aren’t ready for what this means. 👇
(And then they continue for another 25 or 30 single-line sentences that are spaced out because apparently paragraphs don’t exist where thought leadership comes from.)
Look, I understand the impulse to talk about big numbers. Big numbers are exciting. Car guys fall into this trap all the time when we’re talking about dyno numbers. However, what I disagree with is how people have decided that the correct way to evaluate businesses compute is to ask, “What is the largest model I can cram into this machine?” This is not a business metric. This is how my toddlers try and pour Cheerios into a bowl. Also, I want to say this is not uniquely tied to compute. It’s a pretty weird and common human instinct.
People purchase trucks and SUVs based upon towing capacity. Do any of you know what 10,000 or 11,000 lbs. of towing capacity actually means? That’s three Miatas and a trailer. Most Ford F-150s can tow two of themselves, plus the trailer that they’d need to be towed on. Suffice to say, people don’t have a good relationship between what they need and the capabilities they’re buying. That’s why here in Northern Virginia, most office parking lots are populated with quarter-ton pickups and not efficient commuter vehicles. They got the truck because sometimes they’ll go to Home Depot and purchase a bag of cement mix and maybe a shovel. But they got the truck because they have the illusion that they might need the capability.
So when it comes to large language models, especially open weight models being served on edge compute, I think we should look at the numbers that actually matter. Additionally, there’s this one thing that I haven’t seen anyone discuss, and I think it’s because it’s super dense and involves actual machine learning expertise, which none of the “AI LinkedIn thought leaders” have.
You’re supposed to train the fucking model.
The reason they are called open weights is because you can change the fucking weights.
If I sound annoyed, it’s because I am. I have gone into meetings, and these meetings have been pre-poisoned by LinkedIn thought leaders who have given a lot of well-intentioned people very bad ideas about what they need.
The reason you choose open weights is because when you download the model, you’re not just supposed to marvel at how large it is. What you’re supposed to do is take a small, efficient base model and then tune it on your own business data, because having a model running on proprietary business data using a combination of PEFT, LoRA, and QLoRA is the actual value proposition. If you do those things, you will get a model that is smaller, faster, cheaper to serve, and demonstrably better at your specific job than any frontier model that has no idea what your company does.
There is real value in an 8-billion-parameter model that knows your vendors, your menu, your food costs, your staff, your schedule, and your local competitors. That 8 billion parameter model that is trained on your specific business data will absolutely annihilate a 400 billion parameter model that knows everything about the entire Library of Congress… and dick about your business. It will shit on that 400 billion parameter model in terms of accuracy, latency, concurrency, and cost, all at once, and it’s not even close.
And that’s the entire problem I see with pretty much all of LinkedIn thought leadership. Y’all motherfuckers are approaching this like cavemen. Download big thing. Put big thing in memory. Ask big thing question. How the fuck is that actually useful? If that is your mental model for how AI works, the only thing you’re going to do is shop for RAM.
Don’t worry, I’m not done. I’m not nearly done.
I’m going to choose my words carefully here because I know that every once in a while other machine learning engineers will read my blog posts, and I don’t want them beefing in the comments.
I know you can technically fine-tune on Apple Silicon. MLX is a thing that exists, and LoRA on a Mac is a thing that people do, and technically it works. That’s not what I’m trying to say. What I’m trying to say here is that the argument is about the training ecosystem, how optimizations happen. Every kernel, every quantization library, basically every serving stack that anybody uses in production was built around CUDA first, and ROCm second. All of the tools we use, like Flash Attention, bitsandbytes, Unsloth, VLLM with paged attention and continuous batching… I could go on and on. What I’m trying to say is that the tooling is not product-neutral. The tooling is very much built around NVIDIA and its ecosystem, and now AMD does have an ecosystem that is useful from a business context. (And notably, a lot of the AMD hardware is less expensive up front. So there is a trade being made for some small annoyances with the lower capital expenditure.)
If you use Apple hardware, you are going to feel a lot of pain when you are building your business systems.
That’s before we even get to the serving side because Mac OS is a desktop operating system. It was not, and will never be, a server operating system. If you’re running containers, you’re running them in a VM on a machine whose entire purpose was created around logged-in users sitting in front of it doing work. Docker on Linux is native. The reality of Edge compute is that headless, always-on, restart after a power interruption infrastructure… has been a problem that Linux knows how to solve… and has solved for decades.
Yes, I have a Linux tattoo. Yes, I know I have some biases, but there’s also a reason that like 99.5% of servers run Linux. I don’t care if it’s REHL or Ubuntu or Amazon Linux. It’s fucking Linux, goddamn it.
Anyway, yes, the new Mac Studio will load absolutely enormous models. We all agree. But being able to load something versus “can it load, serve, and have high concurrency all day unattended in a business and be reliable” are very different things. And thought leaders on LinkedIn are giving many of you crazy ideas about what Mac OS was designed to do when it is not designed to do those things.
So, let’s talk about an actual business.
This morning I got up at 6:30 in the morning on a Sunday morning because I’m building software for a friend of mine. His name is C.J. He owns a barbecue and beer restaurant here in Northern Virginia. It is the type of barbecue and beer restaurant that you’ve probably been to in your hometown as well. It is small, family-owned, and in a strip mall. It’s extremely popular with the locals because people like supporting family-owned businesses.
What he is not is the type of business that people on LinkedIn like to talk about because his business doesn’t do tens of millions of dollars per year. I’m not going to advertise what his business earns per year, but suffice to say it is not the type of business that most people on LinkedIn want to pontificate about. But this is the real kind of business that AI can actually help.
CJ bought a 24 GB Mac mini and tried to do Openclaw things with it and realized very quickly that he had some technical limitations that he could not address. I’m somebody who comes in from time to time to his restaurant, and he knows what kind of work I do, so we had a conversation about how I perhaps could help. So I started with the hardware he bought and went from there.
There is no fucking universe where I can look at CJ and say, “Hey dude, what you need is an $18,000 Mac Studio.” That would be multiple days of revenue for his business — before the food cost, before the labor cost, before rent, and it would be insane for me to recommend that. Also, that super huge model that we’re talking about that could be loaded onto an $18,000 computer… doesn’t do anything for him. I don’t need to write a research paper about brisket. No one is writing a sonnet about the fucking beer menu.
What CJ actually needs is what I’m actually building, which is several small models, for specifics tasks. Then it’s tuned on his specific business data. Crazy logic, I know.
What business data? You know, these really crazy big ideas like…
- his employees and their names
- their schedules and availability
- his menu and his prices
- his real food costs
- his vendors and suppliers, and the actual shape of those relationships
- the competitors within a few miles of his door
That’s it! That’s how a restaurant works! Ta-da! There is no fucking need for a computer that costs as much as a Honda Accord. The data that we are capturing is not a huge, impressive number. It is a small, trained, and accurate number that is more valuable to his P&L than all the frontier models that I can throw at it. Because his data for his business is worth more than any other esoteric data on the fucking planet.
For his application, I’m using IBM’s Granite family, which is Apache 2.0 licensed. This means that I am allowed to tune them on his data, build a product around it, and then, more importantly, resell that product. That’s the real reason I’m doing this. I want to build software that other restaurants can use as well. That’s what I do as a software engineer. I work with a client. We figure out how it’s going to work on their real business use case, and we collaborate because he knows a lot about running restaurants and I don’t. And with IBM’s granite family of models, I’m allowed to resell the product.
We don’t see a lot of discussions about licensing on LinkedIn, but it’s a very real problem when you’re building software.
For the job of running a restaurant, I need 5 to 8 billion parameters, efficient low-rank adaptation, and high throughput. Every beer that gets ordered, every drink, all the to-go orders, all the reconciliation, everything that happens is a new task, and I need efficiency. You don’t need a Porsche 911 GT3 RS to get to Safeway. You need a Honda CR-V. You can technically get to Safeway in a Porsche 911 GT3 RS. You’re just wasting a lot of money in the process.
Really quickly, let’s talk about PDFs and the old adage that everything looks like a nail when you’re holding a hammer.
My buddy CJ gets vendor invoices as PDFs. These are the same vendors, the same templates, every week. Forever and ever, amen. The LinkedIn thought leader to this problem is an agentic pipeline where they throw a lot of agents of the problem and orchestration, and then there will be some visual diagram that has the word autonomous in it.
The engineering answer is that because these documents are identical every single time and the fields are in the same place on the same axis, I can just map the coordinates, pull the text, write it to a database, and then go make myself a fucking sandwich. That’s OCR and NLP tools in a hundred lines of code, and it’s 100% deterministic and won’t hallucinate. This is good because I don’t want a hallucination that misunderstands and tries to order 900 lbs. of brisket when we only need 90.
The thing I just described to you is called “actual engineering.” This is what engineers do when they understand the problem instead of pattern matching it to whatever bullshit they read on LinkedIn. Every dollar and every watt of compute that I don’t spend making a large language model re-derive a solved problem is saved, and that means I can use that compute for other things. In edge compute, this is an actual really big deal.
In case you’re wondering, that’s the real answer for “How do you run all of this on a Mac Mini?” The answer is “it’s just a couple of containers.” Yes, I’m going to run AWS Backup for redundancy because sometimes the power goes out and I’m not a complete lunatic and edge maximalist. But the goal is for the cloud to be a backup, not the main source of business.
By the way, Chick-fil-A figured this out several years ago. It’s not like I’m some sort of restaurant compute genius.
I took a good idea and I copied it. Shout out to Brian Chambers and Alex Crane from Chick-Fil-A. They wrote this entire edge observability piece about how they operate their edge fleet back in 2023. It’s basically an essay about how they run Kubernetes clusters and each of their roughly 2,800 restaurants. This is how they keep business-critical workloads running without depending upon the internet and how hundreds of thousands of IoT devices can send information to each other. It’s actually brilliant, and I respect the hell out of those dudes for figuring it out.
it’s genuinely a great blog post and the parts that I love the most are probably going to be the most boring for people who like to yap on LinkedIn. They talked about how early versions of their services ate so much bandwidth shipping verbose logs that it started interfering with credit card processing. Interfering with credit card processing is what we will call “not good.” After all, credit cards are the money-making part of the engagement. Their solution was to put the point-of-sale and credit card systems on separate VLANs from the edge network so an overly chatty log pipeline can’t overwhelm payments. They also talked about how their default is shipping only errors, and if a team needs debug-level logs from a specific app at a specific store, they flip a switch through an API that automatically reverts after about 30 minutes. (Because they know that somebody will totally forget to turn that shit off.)
The important thing about their blog post is that this is what edge compute looks like when you’re actually engineering it. You have real compute constraints and bandwidth budgets and failure modes, and you have to create automated guardrails against people forgetting to do things. People who are serious about edge compute are not talking about parameter counts. If you’ve actually done this in the real world, you know about saturating uploads in a way that results in failed business processes.
This blog post is a re-write of another blog post from 2019, so I don’t know where Chick-fil-A is now, seven years later. But structurally, we can learn a lot from the substrate that they built seven years ago. It correct then, and it’s still correct now. Chick-fil-a accidentally decided for the future. If they ever decided to put local accelerated compute in every restaurant, they wouldn’t be restarting a journey. If they wanted to add an NVIDIA Spark to every restaurant, they’d just be adding a node type. Yes, I understand the Chick-fil-A restaurants are franchised, and there’s probably some nuances to how this is deployed, but I’m not going to get into that, nor even pretend I know how it works… because it’s Sunday. You can’t even eat there today. I got you all excited about Chick-Fil-A, and now you can’t even go.
Anyway, here’s what I’m trying to say: these are platform questions, but the platform questions don’t matter because the hard questions have already been answered.
Meanwhile, the Thought Leader Industrial Complex of LinkedIn has discovered Edge AI, and now they would like to explain it to us.
Here’s another cool idea. You don’t have to train your models on the things you run it on.
Ready for me to absolutely evaporate their entire discussion? Here comes the fucking laser gun.
I am not going to fine-tune anything on CJ’s Mac Mini. That happens on my NVIDIA Spark, which is sitting in my office because that’s what a Spark is built for. It’s CUDA native. It runs Linux. It has all the NVIDIA native tools, and it was built specifically for this job. Then, when I’m done doing the training, I take the resulting weights and I load them onto the Mac.
That is because these machines are built to do different things. You train models on hardware built for training and then ideally serve them on the hardware sitting at the location. (If I had to do it all over again, I probably would have purchased different hardware for CJ. There’s a likelihood here that I just give him the NVIDIA Orin AGX that I bought used, because it runs Linux and it’s actually better at serving local compute than the Mac mini is. Unfortunately, he bought the Mac mini when he was Googling too much about people building Openclaw workflows, and so he bought a machine that probably is not built for his business.)
What CJ’s business really needs is a machine that can serve his models well and reliably for something in the span of three to five years. It needs to be small, cheap, and boring. Small, cheap, and boring is what you want in edge compute.
There is a difference between a purchase and capital expenditure.
People are waving their hands around and pretending that spending $18,000 on a Mac Studio is just the kind of purchase you might make. At $18,000, that’s not a purchase. That needs to be defined as capital expenditure. If you’re spending that kind of money, you need to be able to show me the revenue it enables or the cost it removes, and also show me the payback period. If you can do all of those things, buy it. Seriously, I’ll help you spec the whole thing out.
But before you do, I want you to think about two things.
Here’s the first thing: Every cost you are comparing it to is highly subsidized right now. Every company offering models to you via subscription or API is doing so at a highly subsidized rate, and this is not a hidden thing.
Here is the second and more important thing: What you are about to purchase is a depreciating asset in the fastest depreciating category in the history of business equipment. RAM and compute are expensive right now, but I understand something about China that you do not. They have been building fabs. They are in the hardware business, and China is a manufacturing economy. The RAM and compute price climb is about to start a precipitous decline.
In two years, it is highly likely that your hardware will be worth one quarter of what it is worth today. That makes even automotive depreciation look like a good investment.
If you are even thinking about buying one of these computers for personal use, you are smoking crack.
While we’re here deflating things, let’s just slap Jensen Huang real quick…
NVIDIA is not worth an insane sum of money because Jensen Huang foresaw the large language model era. Nvidia got to where they are because they spent decades building hardware to do lighting, shading, and polygon math for video games. It turned out, completely accidentally, that those matrix operations are nearly identical to the ones underneath transformers. It is the luckiest architectural coincidence in computing.
I know that they didn’t foresee this because if they had known, they’d have been buying back stock like maniacs. But they weren’t buying back stock.
I know this because I bought Nvidia stock at $7. Did I buy it because I had this bold idea about the up-and-coming AI boom? No, I didn’t. I was a PC gaming nerd who thought the 1000 and 2000 series GPUs were better than the RX series GPUs from AMD. I ran Ubuntu as a daily driver, where NVIDIA’s drivers were much less of a headache. Eventually, I figured Valve would ship hardware that would run Linux and likely support NVIDIA.
I was right because Valve did ship hardware. And indeed, Sony went for AMD. Does any of this matter? Absolutely not. Gaming has almost nothing to do with the price of either NVIDIA or AMD stock today. I bought stock because I built a machine with an RX 580 and then, in the weeks afterwards, spent the entire time wishing I had just bought Team Green. Whatever. Fuck it. I got paid anyway.
The reason I mention this is because I want you to understand how much of this industry was built upon complete accidents. That level of humility is not something you’re going to read in a LinkedIn post that begins with “Most companies aren’t ready for what this means.”
In closing, should you buy a Mac for your business?
Maybe. I don’t know. It kinda depends, which is not an answer that posts well on LinkedIn. But it’s also the real answer.
If your business has mixed workloads and you need a desktop that also has decent inference, a Mac is a completely defensible purchase, and you will enjoy owning your Apple computer — just like I enjoy my Apple computer. If you’re rendering multiple streams of 8K video, a Mac Studio is the only tool for the job, and you should buy one today.
But for Edge Compute, I’m going to default to Linux because the containers are easier, the services are native, and headless is the default state of the way Linux was meant to run. (It turns out 2026 wasn’t exactly the year of the Linux desktop, but we’re getting really close, because the more of these Spark and AMD machines ship running Linux, the more I realize that people are going to learn that Ubuntu is actually kind of nice.)
Lastly, the models don’t need to be hot at the same time. That’s what I call a “phantom requirement.” There are plenty of workloads that can run overnight when there’s no one in the building. If you have a high fidelity model that takes 90 seconds to load, but if it’s 2AM in the morning, no one is inconvenienced. Many businesses have hours upon hours of dead time, and you can use that when no one is around.
Boy, this blog post got pretty long, didn’t it?
Here’s the lesson: size your edge compute to what you actually need. For most small and mid-sized businesses, for every single workload I’ve encountered building software, a 5-8 billion parameter model with proper PEFT or LoRA on your business data is more than enough for most tasks. Train it somewhere appropriate. Maybe even the cloud if that’s the cheap training environment, then serve it somewhere cheap. Use deterministic tooling for problems that can be solved deterministically. Keep your bills small.
And if you’re not tuning anything on your own business data, what the fuck are we even doing here? Are we even having the same conversation? If you’re pricing out hardware for a workload you have not defined, based upon a benchmark you don’t need, recommended by someone who’s never shipped anything on LinkedIn, you’re probably not making good business decisions anyway.
That’s my real frustration with where we are in the “AI boom.” It isn’t that these people are wrong, because everybody’s wrong sometimes. I’ve been married for seven and a half years, which means that I’ve been wrong for approximately seven and a half years. What I take offense with is the confidence these people have, and the inverse… Which is that they’re all wrong. I really wish these people who have absolutely no doubt about their own ideas, did not have the ability to invent credentials for themselves.
the reason I’m frustrated about it is because these shitty ideas don’t stay on LinkedIn. They land in my meetings. A guy with a restaurant buys a Mac Mini when he really needs something smaller and cheaper. Or alternatively, I have somebody with a manufacturing center who has braced for a five-figure capital expenditure. I have to talk them down into a cheaper computer while they think I might be wrong because I’m telling them to buy something cheaper, and they think that can’t possibly be right.
When I tell somebody that they’re gonna be making a purchase that is one-fifth of the price they’re expecting, I look like somebody making a wild claim when I’m not. They’ve been bracing for this huge purchase, and they think that I’m doing something wrong when, in fact, they’re just listening to shitheads on LinkedIn who’ve never built or shipped anything.
Then I have to politely explain to them, “No, man, you’ve just been reading a lot of crazy bullshit. Let’s go build this thing a lot cheaper.”
Did you like what you read here? Did you find something useful in it? Sick. I don’t have sponsors, and I don’t gate any of my posts with an email list. I don’t endlessly barrage your email inbox because I think it’s annoying. If you like my blog, just bookmark it and come back here sometimes and read it.
If you want to support me, you can buy me a coffee here. I don’t like selling other people’s stuff or using sponsors, so instead I use affiliate links for ShopMy, and everything you see in this store is something I have actually purchased with my own money. I make a small commission if you choose to click any of those links and buy something.
Lastly, I’m not as fast at typing as I am at talking. This entire post, and every other blog post you read on my site was written using Wispr Flow. If you click this link, you can download Wispr Flow, and I’ll get paid a small commission when you use it.