Copilot is Meh

Microsoft could do much better

For as long as Microsoft’s Copilot AI has been available, I’ve been less than enthusiastic about it. I thought it was just a personal feeling. But since I decided to research this post I discovered that my “meh” assessment is broadly held. A quick Google search with the topic “people who hate Copilot” yields a litany of discontent. Google’s AI Overview (admittedly not an unbiased source) reports,

People dislike Microsoft Copilot due to forced integration, data privacy concerns, poor quality/inaccurate responses (hallucinations), disruption to workflows, lack of control (can’t easily disable), generic output, security risks for businesses, and frustration with its inability to handle simple tasks or feedback gracefully, with some feeling it hinders learning or adds bloat rather than value.

It goes to cite a broad list of “Common Criticisms and Concerns,” and includes a section on “Who Hates It?”

My Own Private Meh

I have several reasons for my own Copilot antipathy. First of all, it feels like generic food in the grocery store. I thought that one of our largest and most profitable tech companies should be develop its own language model rather than white-labeling ChatGPT. Three years after that technology entered the world, Copilot still relies primarily on GPT-4 and 5.

Then there was the primary purpose of CoPilot, which appears to be to aid people in the use of Microsoft’s Office suite. Is the best that we can do with this breakthrough to create more Word, Excel, and PowerPoint outputs? It makes CoPilot seem like an advanced version of Clippy. It’s better than Clippy, but perhaps not enough to get excited about.

I have used Copilot on occasion, but it often doesn’t seem to work for me. For example, I just tried clicking on Word Copilot (there are many versions), and I got this helpful message: “AADSTS7000024: Inconsistent broker application IDs asserted by incoming credentials.” Maybe that’s code for “I’m not going to help you write an article that attacks my very existence.”

The Value Issue

But my greatest objection to Copilot is that it contributes to the lack of value for corporate customers that may bring down the entire AI-driven economy. The problem is that Copilot’s aids to personal productivity enable a benefit that is virtually impossible to measure. The savings in creating Office documents is typically a few minutes here and there. Even in the unlikely scenario in which those saved minutes are aggregated—one company I studied did say they had saved more than ten million minutes—but what of it? What are individuals and companies doing with those saved minutes? No organization I’ve come across seems to know.

I spoke with a data and AI leader in a large bank today about this. His organization has a well-defined approach to measuring the value of AI use cases. But he said he struggles with measuring the productivity benefits of Copilot usage. “Maybe they save a few minutes in writing an email or creating a spreadsheet,” he said. “But how do I know that they’re not using that time to check their social media feeds?”

One reason why Copilot is common within large organizations is that it often comes along with the Office 365 deals that Microsoft has with some of them. It’s very easy for an IT organization to say, “We’ve got generative AI covered—we have Copilot licenses for everybody.” But that, of course, doesn’t mean that the organization will get economic value from those licenses. And if the organization has to pay a per-user addon for Copilot—often $30 depending on the Office license—it can add up to a substantial expenditure.

There Are Better Alternatives

When I talk to AI-savvy individuals, I often ask them what language models they use. They never say Copilot. If you’re serious about using AI as an individual, I’d go with ChatGPT, Gemini, Claude, etc. (not necessarily in that order). I don’t recommend Copilot unless you are a very dedicated Microsoft customer who spends all of your time in the Office suite.

The same applies for businesses. I am a fan of enterprise-oriented genAI use cases over individually-focused ones, as this article describes. I never hear an AI leader say that they have built their enterprise AI application on Copilot.

Like most of us, I have spent a lot of time using Microsoft products over the years. Even though I sometimes have problems with their products, I think they are a pretty well-managed company overall since Satya Nadella ascended to the CEO job. I especially admire what they have done with the Power Platform to encourage citizen development (or AI-driven vibe coding, although I think that there are better AI alternatives for that too). But Copilot feels like the Microsoft of old—conservative, slow, and second-rate. It’s the Bing of AI.