Cloudflare is already testing chips from 2023 startup Positron, which just landed $51.6 million in new funding. The company claims its kit outperforms Nvidia’s upcoming Vera Rubin hardware by up to six times in energy efficiency and could slice inference costs to a fraction.

Positron’s stripped-back design focuses entirely on AI inference, the part where your chatbot or image generator spits out a result, and skips the general-purpose fluff that bloats other hardware. Cloudflare’s hardware head Andrew Wee says the early numbers are promising enough to begin wider deployment.

Groq, another startup led by former Google chip boss Jonathan Ross, is pushing its own custom silicon. The company builds chips with integrated memory to slash power use and latency. It reckons it can deliver AI results faster than Nvidia while using just one third the energy.

The goal is to break free of the so-called Nvidia tax, where the dominant chipmaker takes home gross margins of around 60 per cent. Big names like Google, Amazon and Microsoft are also throwing billions at their own inference-focused chip designs in a bid to cut costs and reduce dependence.

Despite all this, Nvidia is still in the game. Senior director Dion Harris says the company’s new Blackwell platform is up to 30 times more efficient than its previous generation. He argues that customers prize flexibility in their AI hardware, given the fast pace of model development.

Wee, a veteran of the Fruity Cargo Cult Apple and Meta, says the growth in AI power consumption, projected at 50 per cent annually until 2030, is simply not sustainable. Cloudflare is one of the few firms testing alternative chips at scale and is now weighing full deployment.

But even with all these gains, Google Cloud’s AI and infrastructure boss Mark Lohmeyer warns that demand will likely outstrip any efficiency improvements. The tech giants are already looking at nuclear and fusion power to feed their data centres.

Anthropic notes that unless the energy issue is cracked, the global power grid could become the real bottleneck in the future of AI. Cutting chip energy waste might be a start, but it’s nowhere near the finish line.