In recent months, there has been a growing debate about an alleged stagnation in generative artificial intelligence. However, this apparent pause could just be the prelude to a new phase of innovation. The key to understanding this situation lies in the sigmoid function, a concept that has transformed the way we interpret technological advances.
Is AI really stopping?
The evolution of language models such as OpenAI’s GPT-2, GPT-3 and GPT-4 has followed a clear pattern: trainings on more powerful data centers with increasing amounts of data. However, this approach seems to have reached a limit. Experts such as OpenAI’s Ilya Sutskerver suggest that simply adding more GPUs and data is no longer enough to achieve significant breakthroughs.
This situation has even led to delays in the launches of models such as OpenAI’s “Orion”. The same is true for other giants such as Google and Anthropic, which face difficulties in delivering noticeable qualitative leaps in their developments.
A pause for rethinking
Although this news might seem alarming, the supposed stagnation is actually part of a natural process in technology. This phenomenon can be explained by the sigmoid function. In its S-shaped curve, initial progress is slow, followed by accelerated growth and, finally, a further slowdown. This cycle does not indicate the end of innovation, but a pause before the next revolution.
Similar examples abound in other sectors. Moore’s law, for example, has faced moments of apparent exhaustion, but advances such as photolithography have renewed its relevance. The transition from propeller planes to jets is also a clear case of how “stagnation” can precede great innovations. Another example is the development of batteries for electronic devices, where new chemistries such as solid-state batteries promise to push current limits.
New paths for AI
If the model of using more GPUs and data has reached its limit, companies are exploring alternatives. One prominent example is OpenAI’s o1 model, designed to improve reasoning capabilities and reduce errors in responses. This approach focuses on the inference phase, optimizing models after they have been trained.
This pause also provides an opportunity to address other essential aspects. Safety and regulation are areas that have lagged amid the rapid development of AI. Spending time establishing regulatory frameworks and improving the safety of models is crucial to ensure responsible and safe use of this technology.
At the same time, this pause invites reflection on the sustainability of AI in energy and environmental terms. Reducing the carbon footprint of data centers is a challenge that could also benefit other technology sectors. These efforts open new doors to develop more efficient and responsible solutions.
Continue your professional education
Understanding concepts such as the sigmoid function and its impact on technological evolution is key for those looking to lead in the field of artificial intelligence. Our Master in Strategic Management with a specialty in Information Technology provides you with the necessary tools to analyze, design and manage innovative technological solutions. With a focus on the latest industry trends and challenges, this program prepares you to face and overcome the current limits of technology, positioning you as a professional capable of contributing significantly to the advancement of AI and other related fields. Don’t miss the opportunity to be part of this technological revolution and propel your career to the next level.
Source:
Against those who think AI is stagnating, one concept gives new hope: the sigmoid function.
Logistic regression: How to calculate a probability with the sigmoid function
