GroqCloud

Our verdict on GroqCloud

GroqCloud is the fastest way we have tried to run open-source models without setting up our own servers. Jordan uses it "sometimes for the open source chat GPT models," and in a live demo it produced ten emails in about five seconds.

That is the extent of our evidence: one demo and some news commentary. We have not built a product on it, and we did not test pricing, limits or other models.

One naming note: Groq with a q is this company. Grok with a k is xAI's chatbot. Our transcripts mix them up too.

What we did with GroqCloud

In Episode 7 (August 2025), the day after OpenAI released its open-weight models, Jordan opened GroqCloud to show what gpt-oss-120b could do:

• He set a system prompt, "you're a marketing expert," and asked for ten emails announcing advertising on Built This Week.
• All ten came back in 5.3 seconds.
• He added "no dashes" and got a rewrite in 4.2 seconds.

His takeaway was not about the emails, which he admitted were not perfect. It was that when a capable model runs this fast and this cheaply, you can afford to run it constantly.

He also flagged a limit up front: the podcast was new, so the model probably had no idea what Built This Week was. Everything it wrote came from the prompt. Speed does not fix missing context, so give a hosted open model the facts it needs.

And if the cost goes to zero or, you know, close to zero, it means that you can do a lot more things.

— Jordan Metzner, Episode 7

It seems to be one of the market leaders in hosting open source models. It's incredibly fast.

— Jordan Metzner, Episode 13

How we use GroqCloud

1. Use it for open-source models. GroqCloud hosts open models, including Facebook's and OpenAI's gpt-oss. For the closed frontier models we go to the vendors directly.
2. Pick the model in the console and set a system prompt. That is all Jordan did before running the email test.
3. Lean on the speed to iterate. When a full rerun takes four seconds, you can tweak one instruction at a time and see the result immediately.
4. Save it for jobs that run often. Jordan's idea in Episode 7 was always-on work: an agent that reads your whole code base looking for bugs all night, or one that keeps documentation current, where near-zero cost matters more than peak quality.
5. Keep it for the cases that fit. Sam's summary in Episode 13: we have used it for "some open source use cases," not as our main model provider.

Where GroqCloud falls short

• Thin evidence from us. One demo and occasional use. We cannot speak to reliability at scale, rate limits or cost.
• Open models only. If you want the best model available on a given day, which is what Jordan said we usually reach for (Episode 37), GroqCloud is not where you will find it.
• Speed is a trade-off. Positron's Mitesh Agrawal explained in Episode 37 that higher tokens per second per user takes more chips, so it costs more. That is useful context for any fast inference provider.

The business side

In September 2025, Jordan covered Groq raising $750 million at almost a $7 billion valuation. He said he likes the product, noted it had been growing in the Middle East and had built its own chips, and called it an NVIDIA-style competitor and maybe an OpenAI-style one too. Both hosts said they would have expected a higher valuation.

For more on why inference speed and cost matter, see AI infrastructure.

Episodes featuring GroqCloud

• Episode 7: the live gpt-oss-120b speed demo on GroqCloud.
• Episode 13: Groq's funding round and how we use it.

FAQ

What is GroqCloud?

A cloud provider that hosts open-source models, including OpenAI's gpt-oss and Meta's models. Jordan called it one of the market leaders in hosting open-source models.

How fast is GroqCloud?

In Episode 7, gpt-oss-120b on GroqCloud wrote ten marketing emails in 5.3 seconds, then rewrote them with a small change in 4.2 seconds.

Is GroqCloud the same as Grok?

No. Groq with a q is the hosting company behind GroqCloud. Grok with a k is xAI's chatbot, which we cover separately.

How much does GroqCloud cost?

We did not cover pricing on the show. Jordan's point was that open-source models only cost you the servers they run on, which is why a fast, cheap host matters.