Is Deepseek a Black Swan event?

ph1l · Feb 2, 2025

Has DeepSeek been DeepSeeked, or is it another case of copy and paste?
https://futurism.com/researchers-deepseek-even-cheaper

Team Says They've Recreated DeepSeek's OpenAI Killer for Literally $30
"The results: it just works!"
/ Artificial Intelligence / Artificial Intelligence / Deepseek / Large Language Models

Image by Getty / Futurism

You might've heard of the hardware guru who crammed the videogame Doom into a pregnancy test. Well, the AI-geek equivalent just figured out how to reproduce DeepSeek's buzzy tech for the cost of a few dozen eggs.

Jiayi Pan, a PhD candidate at the University of California, Berkeley, claims that he and his AI research team have recreated core functions of DeepSeek's R1-Zero for just $30 — a comically more limited budget than DeepSeek, which rattled the tech industry this week with its extremely thrifty model that it says cost just a few million to train.

Take it with a grain of salt until other experts weigh in and test it for themselves. But the assertion — and particularly its bargain basement price tag — is yet another illustration that the discourse in AI research is rapidly shifting from a paradigm of ultra-intensive computation powered by huge datacenters, to efficient solutions that call the financial model of major players like OpenAI into question.

In an post announcing the team's findings on X-formerly-Twitter, Pan said that the researchers trained their model around the countdown game, a number operations exercise in which players create equations from a set of numbers to reach a predetermined answer.

The small language model starts with "dummy outputs," Pan said, but "gradually develops tactics such as revision and search" to find the solution through the team's reinforcement training.

"The results: it just works!" Pan said.

Pan's crew is currently working to produce a paper, but their model, preciously dubbed "TinyZero," is available on GitHub to tinker around with.

AD

"We hope this project helps to demystify the emerging RL scaling research and make it more accessible," wrote Pan.

Though R1-Zero is a small language model at 3 billion parameters — compare that to its heavyweight brother R1's 671 billion — the team's accomplishment could be a bellwether for open-source developers working on stripped down approaches to AI development.

The release of DeepSeek's R1 model has tightened the screws on pie-in-the-sky artificial general intelligence ventures led by the likes of Meta, Google, OpenAI, and Microsoft, sending stocks associated with American AI into a trillion-dollar tumble. The Hangzhou-based open-source AI startup contends that its tech can do exactly what those corporate ventures have burned through billions of dollars doing for a fraction of the cost.

That's spawned a bevy of questions from investors, leading with why the seven wealthiest tech corporations on earth all walk lockstep in glacier-melting efforts when a more elegant solution may have been there all along?

AD

And if reproducing a model like TinyZero can be done with less than $30 and only a few days of work, then what do big tech conglomerates need $500 billion in AI infrastructure for?
More...

https://github.com/Jiayi-Pan/TinyZero

TinyZero is a reproduction of DeepSeek R1 Zero in countdown and multiplication tasks. We built upon veRL.

Through RL, the 3B base LM develops self-verification and search abilities all on its own

You can experience the Ahah moment yourself for < $30

Twitter thread:

Full experiment log: https://wandb.ai/jiayipan/TinyZero

Paper's on it's way!
More...

itg · Feb 2, 2025

Black Swan is AI sector would be China dumping AGI in open source format which outperforms all US models by an order of magnitude. It's coming... the swan has a hardon on the US AI - will start the correction, when ? the future, the future...

metfa · Feb 10, 2025

Pekelo said:
Or help me out, that 500 Billion Stargate project is how many times more than 600 million?
More...

I think Stargate is a different beast. More like some kind of God-computer system. Scary actually.

Log in or Sign up

Is Deepseek a Black Swan event?

Is Deepseek a Black Swan event?

Yes

No

Maybe

ph1l

itg

metfa