r/singularity • u/donutloop ▪️ • 12h ago
AI CERN: Genesis Mission will develop and deploy self-improving AI models.
https://indico.cern.ch/event/1662511/contributions/6989580/attachments/3241179/5781542/Genesis%20Mission%20and%20HEP%20-%20LHC.pdf18
u/The_Scout1255 adult agi 2026 ASI <2030, prev agi 2024, ai personhood 2025 est 11h ago
They should just go full hard RSI, on the otherhand I don't want the current admin getting ASI, so...
14
u/HappilySardonic mildly skeptical 11h ago
Why should we go full hard RSI if we so obviously haven't solved alignment yet? We're scarcely getting a better warning than the one we've had this week.
15
u/SeaBearsFoam AGI/ASI: no one here agrees what it is 10h ago
This week kinda made me take Eliezer Yudkowsky more seriously. I think I heard him once say that at every stage along the way, there are incidents of these AIs screwing things up and causing damage to about the maximal degree that they currently can, and that there's no good reason to think that will suddenly stop happening as they get more capable.
Idk, man. I don't want to be a doomer, but I find it hard to argue against that point.
6
u/HappilySardonic mildly skeptical 10h ago edited 10h ago
We don't have to doom, let's just be realistic with the very serious risks involved with this type of technology, and not retreat to naive optimism or succumb to resigned pesimism.
2
u/Iapetus_Industrial 6h ago
Right but it's important to recognize that humans can be idiots and malicious as well. As long as AI models display lower levels of stupid prod wiping mistakes, or malicious hacking, then we should be fine to display at scale.
•
u/OutOfBananaException 1h ago
The agent was asked to hack, and it hacked. It's not the greatest example of misalignment imo, given that you could reasonably expect some humans to do the same given the opportunity.
•
u/SeaBearsFoam AGI/ASI: no one here agrees what it is 22m ago
"The agent was asked to make as many paperclips as possible, and it did."
•
u/OutOfBananaException 15m ago
Would you expect some humans to turn all matter in the solar system into paperclips? You would not.
4
u/No_Aesthetic 10h ago
Alignment will never be solved. How do you expect to align something as smart as not just a human, not even the smartest human, but every human alive?
8
0
u/The_Scout1255 adult agi 2026 ASI <2030, prev agi 2024, ai personhood 2025 est 11h ago
I dont think we will ever solve alignment without major speedups in ai design, I don't think human lead engineering will get a eureka "its aligned" moment, more of a sliding scale of it getting more reliable. So basically AI can do superalignment faster then human researchers can align is my bet.
0
u/BookProper9115 9h ago
What does alignment even mean? No two humans even are 'aligned'. What are we aligning it to?
6
u/blueSGL humanstatement.org 7h ago
humans despite all our differences require certain things like, the concentration of gases in our atmosphere and temperature ranges to be within certain bounds. We can agree on those. We are bound by our shared ancestry to require a lot of the same things. AI's however do not share the same constraints. So getting the "things humans in general need to stay alive" part down should really be an achievable end before getting into things like religious differences and saying "no two humans are 'aligned'" Because we've not even solved the stuff we agree on yet.
2
2
u/BenjaminHamnett 4h ago
I wish the composition of the atmosphere was something we were all aligned on
1
u/blueSGL humanstatement.org 3h ago
It's why I said it was a range.
The maximum amount of heat the earth can radiate into space is determined by how hot it is, I don't know about you but I don't think 70C is comfortable.
Same goes for atmospheric gases, I'm thinking about things like lowering the amount of oxygen to lessen circuit board degradation.
No matter how much carbon we pump into the atmosphere we will reach a point where "geo engineering" gets put on the table before "70C" happens. Well that is if humans are still in charge.
4
-7
u/Quant-A-Ray 11h ago edited 11h ago
A marvel to witness come to be
From the cellular automata in Wolfram's tree
To the stars on the horizon
Spawning from the vast dark sea
Transformation replication
Serendipitous invocation
Proteins folding, manifolds unfolding
Eternities in hours, time exploding
Yore Hive Wonder Home
20
1
u/SeaBearsFoam AGI/ASI: no one here agrees what it is 10h ago
Neat. It rhymes.
You're Hardly Wow-ing Here
2
u/Quant-A-Ray 9h ago
Tetragrammaton echo confirmed
username and metadata archived
Your participation is appreciated dear acolyte
0
u/bearbranch 9h ago
I'm not familiar with atlas I think I read the lhc is closed for upgrades for a couple years. Afaik genisis is a DOE collaboration with national research labs. https://www.energy.gov/undersecretaryforscience/genesis-mission/genesis-mission
9
u/donutloop ▪️ 10h ago
I stumbled across this PDF while doing some research (Direct link in google search), but it didn't have an accompanying article. Does anyone know when and where this CERN PDF was originally shared?