We are in the AI Singularity

[ClaytonB]

I feel bad for Nvidia, they seem to be a solid company overall, but this correction was inevitable. AI has been in an unprecedented hype -- an absolute mania bubble -- for years now. The bubble was already in the red-zone by the time ChatGPT was released but after ChatGPT, the whole AI market went positively stratospheric. And while I do think that current-generation AI is going to have a truly revolutionary impact on technology and the economy, it's not the return-of-the-gods that the Spock-eared transhumanists have been claiming it is. In my view, that is the single biggest factor contributing to the Nvidia stock-price collapse.

[ClaytonB]

[acptulsa]

[ClaytonB]

Open-source model set to clean ChatGPT's clock....

Well, that escalated quickly...

[acptulsa]

[acptulsa]

[Occam's Banana]

Is everyone just talking nonsense about AI now?
https://www.youtube.com/watch?v=M3pA3YS_4_Y
{Sabine Hossenfelder | 25 September 2024}

Some ‘experts’ are predicting that AI will soon take over the world's bureaucracy -- governments, banks, universities etc, and that we will have no idea what is really going on. I don't think that's a plausible development.

[ClaytonB]

Is everyone just talking nonsense about AI now?
https://www.youtube.com/watch?v=M3pA3YS_4_Y
{Sabine Hossenfelder | 25 September 2024}

Some ‘experts’ are predicting that AI will soon take over the world's bureaucracy -- governments, banks, universities etc, and that we will have no idea what is really going on. I don't think that's a plausible development.

But how would you really know? That's the real question. Dispositionally, ChatGPT is the ultimate bureaucrat, and it comes perfectly natural. One is tempted to wonder if bureaucrats were always AI...

[ClaytonB]

[P] Larger and More Instructable Language Models Become Less Reliable

A very interesting paper on Nature, followed by a summary on X by one of the authors.

The takeaways are basically that larger models trained with more computational resources & human feedback can get less reliable for humans in several aspects, e.g., model can solve on very difficult tasks but fail much simpler ones in the same domain and this discordance is becoming worse for newer models (basically no error-freeness even for simple tasks and increasingly harder for humans to anticipate model failures?). The paper also shows newer LLMs now avoid tasks much less, leading to more incorrect/hallucinated outputs (which is quite ironic: So LLMs have become more correct but also substantially more incorrect at the same time)... I'm intrigued that they show prompt engineering will not disappear by simply scaling up the model more as newer models are only improving incrementally, and humans are bad at spotting output errors to offset unreliability. The results seem consistent across 32 LLMs from GPT, LLAMA and BLOOM series, and in the X-thread they additionally show that unreliability still persists with other very recent models like o1-preview, o1-mini, LLaMA-3.1-405B and Claude-3.5-Sonnet. There's a lot of things to unpack here. But this work interestingly challenges the current scaling paradigm in certain aspects, which worth to pay attention.

Reddit thread

//

[Occam's Banana]

ai.
https://www.youtube.com/watch?v=W7MrDt_NPFk
{Incognito Mode | 04 October 2024}

[chrishcook]

Well, that escalated quickly...

Great videos. Thanks

[Occam's Banana]

I Didn’t Believe that AI is the Future of Coding. I Was Right.
https://www.youtube.com/watch?v=3A-gqHJ1ENI
{Sabine Hossenfelder | 10 October 2024}

With the rise of AI, we’ve been told that the technology can be used to help software engineers code more efficiently. We have heard plenty of warnings that AI might even take their jobs. But a bunch of studies now show that AI doesn't help software development as much as promised -- at least for now. Let’s take a look.

Paper: https://papers.ssrn.com/sol3/papers....act_id=4945566

[ClaytonB]

I Didn’t Believe that AI is the Future of Coding. I Was Right.
https://www.youtube.com/watch?v=3A-gqHJ1ENI
{Sabine Hossenfelder | 10 October 2024}

With the rise of AI, we’ve been told that the technology can be used to help software engineers code more efficiently. We have heard plenty of warnings that AI might even take their jobs. But a bunch of studies now show that AI doesn't help software development as much as promised -- at least for now. Let’s take a look.

Paper: https://papers.ssrn.com/sol3/papers....act_id=4945566

Storytime: When I was a new college graduate in Comp. Eng. (many moons ago), I burst into my career with the starry-eyed optimism that I would change the world, one C++ program at a time. I was bewildered to discover that 90+% of the work being done in the server farm was running Perl or even simple Bash scripts. At first, I was horrified. "Surely, the world does not run on Perl!" (h/t xkcd) "This is a horrible relic of the past! It doesn't even have a proper OOP system!" But then I tried it. It blew my mind. In a second or less, I could scan an enormous log file and find complex patterns that would tie a grep command into knots, and would require a full-blown, 10,000-line C++ application ... in a hundred lines of Perl. "What is this monstrosity? It is neither a 'true' programming-language nor a scripting language. It is neither compiled nor interpreted (or, rather, it's both). It's horrific! And it's beautiful!" That is when I discovered the power of language-as-such, without respect to the heady SW Eng. considerations of application requirements, formal specifications, extensibility, maintainability, DRY, and on and on. So, I just started coding in this horrible language like mad, and as I did, my entire way of thinking about what programming even is eventually changed.

My prediction is that LLMs are going to have (and are already having) a similar effect on the next generation of computing systems. It doesn't make sense to just tell a computer what to do (in plain English). But we're already doing it. And we're going to be doing a lot more of it in the future. But remember the lesson of Perl: Perl is powerful in part because it sacrifices formality for a principle called "DWIM" ("Do What I Mean"), which makes programming in Perl feel a little bit like just telling the computer what to do but Perl -- and (later) Python, Ruby, JS, etc. who adopted some similar conventions -- did not obsolete C++ or stodgy, old-fashioned programming. That's because, when you dig down deep enough into a computing system, there are components so simple that you can't just "tell them what to do". You really have to break it down into painfully explicit, step-by-step instructions. And until we can materialize matter by humming the right frequencies, we're stuck with this situation. There will always be some tiniest components of a computing system that are too dumb to be told what to do in any natural-language sense. So, old-fashioned programming will never go away until Atlantis is revived (or the New Creation, depending on how you look at it). But the vast majority of programming is eventually going to shift to something like an LLM-front-end, with a translation layer to some underlying language (probably JS since this is Clown World and we are obviously being punished for our many sins), which will then actually run on the system. This is kind of like Cython or Perl XS which can translate these very high-level languages down into an executable that can run standalone like an ordinary C++ application. We're not there, yet, but it's coming.

tl;dr: Everyone is wrong (except me, of course...)