Six Big AI Projects in the Race Beyond 2 Trillion Parameters
There are six major AI projects in the race beyond 2 trillion parameter models. There is OpenAI, Anthropic, Google/Deepmind, Meta, a UK government project and a stealth project.
It takes $1-2 billion per year of resources to be in the game. There is $1 billion needed for hardware and that hardware will need to be updated every 2-3 years. There is a need for hundreds of AI specialists and staff and some of the lead people will need $1-2 million and stock options.
There are over a hundred other projects and some of those could step up and impact the race.
GPT-5 should be finished by the of 2023 and released early in 2024. It should have 2-5 trillion parameters.
Anthropic plans to build a model called Claude-Next which should be 10X more capable than today’s most powerful AI. Anthropic has already raised $1 billion and will raise another $5 billion and will spend $1 Billion over the next 18 months. Anthropic estimates its frontier model will require on the order of 10^25 FLOPs, or floating point operations. This is several orders of magnitude larger than even the biggest models today. Anthropic relies on clusters with “tens of thousands of GPUs.” Google is one of the Anthropic funders.
Google and Deepmind are working together to develop a GPT-4 competitor called Gemini. The Gemini project is said to have begun in recent weeks after Google’s Bard failed to keep up with ChatGPT. Gemini will be a large language model that will have a trillion(s) parameters like GPT-4 or GPT-5. They will be using tens of thousands of Google’s TPU AI chips for training. It could take months to complete. Whether Gemini will be multimodal is unknown.
Deepmind has also developed the web-enabled chatbot Sparrow, which is optimized for security similar to ChatGPT. The DeepMind researchers find that Sparrow’s citations are helpful and accurate 78% of the time. DeepMind’s Chinchilla is one of today’s leading LLMs and it was trained on 1.4 trillion tokens.
The world’s total usable text data is between 4.6 trillion and 17.2 trillion tokens. This includes all the world’s books, all scientific papers, all news articles, all of Wikipedia, all publicly available code, and much of the rest of the internet, filtered for quality (e.g., webpages, blogs, social media).
There have been emergent capabilities as the larger models have been developed.
There will be the compute power, algorithm improvement to scale models by a thousand to a million times over the next 6 years. Nvidia’s CEO predicted AI models one million times more powerful than ChatGPT within 10 years.
Go paid at the $5 a month level, and we will send you both the PDF and e-Pub versions of “Government” - The Biggest Scam in History… Exposed! and a coupon code for 10% off anything in the Government-Scam.com/Store.
Go paid at the $50 a year level, and we will send you a free paperback edition of Etienne’s book “Government” - The Biggest Scam in History… Exposed! AND a 64GB Liberator flash drive if you live in the US. If you are international, we will give you a $10 credit towards shipping if you agree to pay the remainder.
Support us at the $250 Founding Member Level and get an Everything Bundle – The Sampler of Liberty! - Get it free by going paid as a Founding Member!
Give me Liberty... and give me more! The Everything Bundle includes the latest version of our flagship book on government, along with a collection of potentially life-altering introductions to voluntaryism, agorism and peaceful anarchy.
“Government” – The Biggest Scam in History… Exposed! by Etienne de la Boetie2
Anarchy Exposed! - A former police officer reports on his investigative journeyby Shepard the Voluntaryist and Larken Rose
The Most Dangerous Superstition by Larken Rose
Sedition, Subversion and Sabotage – Field Manual #1 by Ben Stone, The Bad Quaker, and Ken Yamarashi
What Anarchy Isn’t– A short pamphlet by Larken Rose… The perfect introduction to peaceful anarchy
Three Friends Free – A Children’s Story of Voluntaryism
The Liberator is a 64GB wafer flash drive filled with books, documentaries, podcasts, MP3s, short videos, and music from the truth movement’s leading artists. The credit card-sized format makes it convenient to keep in your wallet to share and copy easily.
