First Mover Advantage
In my last post I wrote it does not matter who will develop AGI first. This is incorrect. OpenAI is on the verge of developing a model (GPT-5) smart enough to accelerate the development of its successor. At that moment, they - OpenAI - will rapidly start accumulating an unlimited power. I wonder if some external forces (mainly US government) will ( Read more... )
Comments 5
> OpenAI is on the verge of developing a model (GPT-5) smart enough to accelerate the development of its successor.
"Accelerate" by how much?
By 1%?
GPT helps to improve low-level customer support.
But AGI development is still far away.
Reply
It's hard to estimate the acceleration. Developing a new model involves several stages:
0. Preparing a better dataset
1. Gathering and brainstorming new ideas since the release of the previous model
2. Trying out some of these ideas (multiple experiments on different scales)
3. Launching the main large scale training run, (potentially training multiple different models if using some sort of MoE architecture)
4. Debugging, correction, optimizing this training run
5. Finetuning the converged model
6. RLHF or other alignment methods
GPT-5 will be able to help with 0, 1, 4, and 6. But it's hard to estimate how long each of these stages takes. If, for example, we need a breakthrough and spend all our time brainstorming, the acceleration might be dramatic. But if we already know what to do but the training progresses slowly because there are not enough GPUs, there's little we can do.
All signs indicate we will have GPT-5 by the end of the year. That will mark the beginning of superintelligence era. Not right away of course, but within 1-2 ( ... )
Reply
> GPT-5 will be able to help with 0, 1, 4, and 6
Help by how much?
Outsourcers in Philippines will be able to help somewhat too. So what?
> All signs indicate we will have GPT-5 by the end of the year
Yes, so what?
> That will mark the beginning of superintelligence era.
Nope. GPT-5 will be only an incremental improvement on GPT-4.
Which can read and write, but cannot think.
Reply
I'm curious - how much have you used GPT-4? I only started to understand its limitations after a few hundred hours (I use it 2-3 hours daily).
Reply
Leave a comment