Llama-3 may have just killed proprietary AI models · Kadoa

Meta released Llama-3 only three days ago, and it already feels like the inflection point when open source models finally closed the gap with proprietary models. The initial benchmarks show that Llama-3 70B comes pretty close to GPT-4 in many tasks:

The official Meta page only shows that Llama-3 outperforms Gemini 1.5 and Claude Sonnet.
Artificial Analysis shows that Llama-3 is in-between Gemini-1.5 and Opus/GPT-4 for quality.
On LMSYS Chatbot Arena Leaderboard, Llama-3 is ranked #5 while current GPT-4 models and Claude Opus are still tied at #1.

The even more powerful Llama-3 400B+ model is still in training and is likely to surpass GPT-4 and Opus once released.

Meta vs OpenAI

Some speculate that Meta's goal from the start was to target OpenAI with a "scorched earth" approach by releasing powerful open models to disrupt the competitive landscape and avoid being left behind in the AI race.

Meta can likely outspend OpenAI on compute and talent:

OpenAI makes an estimated revenue of $2B and is likely unprofitable. Meta generated a revenue of $134B and profits of $39B in 2023.
Meta's compute resources likely outrank OpenAI by now.
Open source likely attracts better talent and researchers.

One possible outcome could be the acquisition of OpenAI by Microsoft to catch up with Meta. Google is also making moves into the open model space and has similar capabilities to Meta. It will be interesting to see where they fit in.

The Winners: Developers and AI Product Startups

I recently wrote about the excitement of building an AI startup right now, as your product automatically improves with each major model advancement. With the release of Llama-3, the opportunities for developers are even greater:

No more vendor lock-in.
Instead of just wrapping proprietary API endpoints, developers can now integrate AI deeply into their products in a very cost-effective and performant way. There are already over 800 llama-3 models variations on Hugging Face, and it looks like everyone will be able to fine-tune for their us-cases, languages, or industry.
Faster, cheaper hardware: Groq can now generate 800 llama-3 tokens per second at a small fraction of the GPT costs. Near-instant LLM responses at low prices are on the horizon.

Open source multimodal models for vision and video still have to catch up, but I expect this to happen very soon.

The release of Llama-3 marks a significant milestone in the democratization of AI, but it's probably too early to declare the death of proprietary models. Who knows, maybe GPT-5 will surprise us all and surpass our imaginations of what transformer models can do.

These are definitely super exciting times to build in the AI space, and we're currently adding llama-3 to Kadoa. Stay tuned!