Get the top HN stories in your inbox every day.
gigatexal
jhylau
VCs at the big/mega funds make most of their money from fees, they don't actually care as much about the potential portfolio investment exits 10-15 years from now. What they care MOST about is the ability to raise another fund in 2-3 years, so they can milk more fees from LPs. i.e. 2% fee PER YEAR on a 5bn fund is a lot of guaranteed risk-free money.
To be able to achieve that is entirely dependent on two things:
1) deploying capital in the current fund on 'sexy' ideas so they can tell LPs they are doing their job
2) paper markups, which they will get, since Ilya will most definitely be able to raise another round or two at a higher valuation. even if it eventually goes bust or gets sold at cost.
With 1) and 2), they can go back to their existing fund LPs and raise more money for their next fund and milk more fees. Getting exits and carry is just the cherry on top for these megafund VCs.
rawrawrawrr
Are you a VC? If they really didn't care about their investment exits, that would be crazy.
jhylau
It’s not that they don’t care, of course they want to find winners. It’s just that A) there is so much capital to allocate that they have to allocate to marginal ideas B) their priorities are to raise their next fund which means focusing on vanity metrics like IRR and paper markups C) The incentive structure in VC pushes them to invested based on motivated reasoning. Remember VC returns are cyclical, and many vintages underperform the public markets and particularly large funds do worse simply because they have too much capital to allocate and too few great ideas.
aorloff
I think they treat success and failure as mostly luck, with a tiny bit of hygiene
The trick is being in the game long enough
andsoitis
> about is the ability to raise another fund in 2-3 years, so they can milk more fees from LPs. i.e. 2% fee PER YEAR on a 5bn fund is a lot of guaranteed risk-free money.
You will struggle to raise funds if the companies you bet on perform poorly; the worse your track record the less chances of raising money and earn income from it.
jhylau
Track record is based on IRR mostly. See my other comment on the Lps below regarding the incentive structure and what they care about. This particular bet is almost a guaranteed markup, as Ilya will surely/likely raise another round. It’s also not a terrible bet to invest in a proven expert/founder. By the time these companies exit (if they ever) 15 years from now, the mega fund VC partner will probably be retired from all the cumulative fees and just playing golf and taking occasional board meetings. Cash on cash returns are very different to playing the IRR game. Of course they want to find real winners as well, but reality is there aren’t that many and they have so much money to allocate they will have to bet on marginal things that can at least show some paper gains.
baxtr
I’m not a VC, but I doubt that’s true. You can exactly raise one fund if you operate like that but not two or three.
conradev
You could actually raise and deploy two or three funds that way, before you see returns for the first
baq
Given how many folks blow up and go back into the business... the best way to get a fund to run is to have previous experience running a fund.
lazzlazzlazz
This couldn't be less true, for what it's worth. VCs from the largest funds are in it ~entirely for the DPI (distributed to paid-in capital; investment returns). Not only is this far, far more profitable than management fees (which are mostly spent on operations) — DPI is the only way to guarantee you can raise the next fund.
hello0904
So the question I have is, who are these LP's and why are they demanding funds go into "sexy" ideas?
I mean it probably depends on the LP and what is their vision. Not all apples are red, come in many varieties and some for cider others for pies. Am I wrong?
neom
The person you're responding to has a very sharp view of the profession. imo it's more nuanced, but not very complicated. In Capitalism, capital flows, that's how it works, capital should be deployed. Larges pools of capital are typically put to work (this in itself is nuanced). The "put to work" is various types of deployment of the capital. The simplest way to look at this is risk. Lets take pension funds because we know they invest in VC firms as LPs. Here* you can find an example of the breakdown of the investments made by this very large pension fund. You'll note most of it is very boring, and the positions held related to venture are tiny, they would need a crazy outsized swing from a VC firm to move any needles. Given all that, it traditionally* has made no sense to bet "down there" (early stage) - mostly because the expertise are not there, and they don't have the time to learn tech/product. Fee's are the cost of capital deployment at the early stages, and from what I've been told talking to folks who work at pension funds, they're happy to see VCs take swing.
but.. it really depends heavily on the LP base of the firm, and what the firm raised it's fund on, it's incredibly difficult to generalize. The funds I'm involved around as an LP... in my opinion they can get as "sexy" as they like because I buy their thesis, then it's just: get the capital deployed!!!!
Most of this is all a standard deviation game, not much more than that.
https://www.otpp.com/en-ca/investments/our-advantage/our-per... https://www.hellokoru.com/
jhylau
These LPs at mega funds are typically partners/associates at pension funds or endowments that can write the 8-9figure checks. They are not super sophisticated and they typically do not stay at their jobs long enough to see the cash on cash returns 15 years later. Nor are they incentivized to care either. These guys are salaried employees with MBAs and get annual bonuses based on IRR (paper gains). Hence the priority is generating IRR , which in this case is very likely as Ilya will raise a few more rounds. Of course, Lps are getting smarter and are increasingly making more demands. But there is just so much capital to allocate for these mega funds, inevitable that some ideas are half baked.
isoprophlex
I didn't know what an LP is, having lived life gloriously isolated from the VC gospel...
an LP is a "limited partner." they're the suckers (or institutional investors, endowments, pensions, rich folks, etc.) that give their cash to venture capital (VC) firms to manage. LPs invest in VC funds but don't have control over how the money gets used—hence *limited* partner. they just hope the VCs aren't burning it on overpriced kombucha and shitty "web3" startups.
meanwhile, the VCs rake in their fat management fees (like the 2% mentioned) and also get a cut of any profits (carry). VCs are more concerned with looking busy and keeping those sweet fees rolling in than actually giving a fuck about long-term exits.
Someone wants to fund my snide, cynical AI HN comment explainer startup? We are too cool for long term plans, but we use AI.
furyofantares
I'm not a VC so maybe you don't care what I think, I'm not sure.
Last night as my 8yo was listening to childrens audio books going to sleep, she asked me to have it alternate book A then B then A then B.
I thought, idunno maybe I can work out a way to do this. Maybe the app has playlists and maaaaaaaaaaybe has a way to set a playlist on repeat. Or maybe you just can't do this in the app at all. I just sat there and switched it until she fell asleep, it wasn't gonna be more than 2 or 3 anyway, and so it's kind of a dumb example.
But here's the point: Computers can process language now. I can totally imagine her telling my phone to do that and it being able to do so, even if she's the first person ever to want it to do that. I think the bet is that a very large percentage of the world's software is going to want to gain natural language superpowers. And that this is not a trivial undertaking that will be achieved by a few open source LLMs. It will be a lot of work for a lot of people to make this happen, as such a lot of money will be made along the way.
Specifically how will this unfold? Nobody knows, but I think they wanna be deep in the game when it does.
steveBK123
How is this any different than the (lack of) business model of all the voice assistants?
How good does it have to be, how many features does it have to have, how accurate does its need to be.. in order for people to pay anything? And how much are people actually willing to spend against the $XX Billion of investment?
Again it just seems like "sell to AAPL/GOOG/MSFT and let them figure it out".
furyofantares
> How is this any different than the (lack of) business model of all the voice assistants?
Voice assistants do a small subset of the things you can already do easily on your phone. Competing with things you can already do easily on your phone is very hard; touch interfaces are extremely accessible, in many ways more accessible than voice. Current voice assistants only being able to do a small subset of that makes them not really very valuable.
And we aren't updating and rewriting all the world's software to expose its functionality to voice assistants because the voice assistant needs to be programmed to do each of those things. Each possible interaction must be planned and implemented invidually.
I think the bet is that we WILL be doing substantially that, updating and rewriting all the software, now that we can make them do things that are NOT easy to do with a phone or with a computer. And we can do so without designing every individual interaction; we can expose the building blocks and common interactions and LLMs may be able to map much more specific user desires onto those.
kelnos
> How is this any different than the (lack of) business model of all the voice assistants?
Feels very different to me. The dominant ones are run by Google, Apple, and Amazon, and the voice assistants are mostly add-on features that don't by themselves generate much (if any) revenue (well, aside from the news that Amazon wants to start charging for a more advanced Alexa). The business model there is more like "we need this to drive people to our other products where they will spend money; if we don't others will do it for their products and we'll fall behind".
Sure, these companies are also working on AI, but there are also a bunch of others (OpenAI, Anthropic, SSI, xAI, etc.) that are banking on AI as their actual flagship product that people and businesses will pay them to use.
Meanwhile we have "indie" voice assistants like Mycroft that fail to find a sustainable business model and/or fail to gain traction and end up shutting down, at least as a business.
I'm not sure where this is going, though. Sure, some of these AI companies will get snapped up by bigger corps. I really hope, though, that there's room for sustainable, independent businesses. I don't want Google or Apple or Amazon or Microsoft to "own" AI.
ip26
The voice assistants are too basic. As folks have said before, nobody trusts Alexa to place orders. But if Alexa was as competent as an intelligent & capable human secretary, you would never interact with Amazon.com again.
dr_dshiv
Congrats on 10k karma :)
One could ask: how is this different from automatic call centers? (eg “for checking accounts, push 1…”) well, people hate those things. If one could create an automated call center that people didn’t hate, it might replace a lot of people.
idiocrat
Talking to an electronic assistant is so antiquated. It feels unnatural to formulate inner thoughts into verbal commands.
An ubiquitous phone has enough sensors/resources to be fully situationally aware and preempt/predict for each holder any action long time ahead.
It can measure the pulse, body postures and movements, gestures, breath patterns, calculate mood, listen to the surrounding sounds, recall all information ever discussed, have 360 deg visual information (via a swarm of fully autonomous flying micro-drones), be in an network with all relevant parties (family members, friends, coworkers, community) and know everything they (the peers) know.
From all gathered information the electronic personal assistant can predict all your next steps with high confidence. The humans think that they are unique, special and unpredictable, but opposite is the case. An assistant can know more about you than you think you know about yourself.
So your 8yo daughter does not need to tell how to alternate the audio books, the computer can feel the mood and just do what is appropriate, without her need to issue a verbal command.
Also in the morning you do not need to ask her how she slept tonight and listen to her subjective judgement.
The personal assistant will feel that you are probably interested in your daughters sleep and give you an exact objective medical analysis of the quality of the sleep of your daughter tonight, without you needing to ask the personal assistant of your daughter.
I love it, it is a bottomless goldmine for data analysis!
darkwater
> The personal assistant will feel that you are probably interested in your daughters sleep and give you an exact objective medical analysis of the quality of the sleep of your daughter tonight, without you needing to ask the personal assistant of your daughter.
Next step: the assistant knows that your brain didn't react to much to its sleep report the last 5 mornings, so it will stop bothering you altogether. And maybe chitchat with your daughter's assistant to let her know that her father has no interest in her health. Cool, no? (I bet there is already some science fiction on this topic?)
ignoramous
> Computers can process language now. I can totally imagine her telling my phone to do that
Impressed by this bot recently shared on news.yc [0]: https://storytelling-chatbot.fly.dev/
> Specifically how will this unfold? Nobody knows
Think speech will be a big part of this. Young ones (<5yo) I know almost exclusively prefer voice controls where available. Some have already picked up a few prompting tricks ("step by step" is emerging as the go-to) on their own.
dr_dshiv
Robots will be better than humans at parenting in no time
slashdave
I dunno. A small piece of $1B sounds a little shallow.
reissbaker
In general VC is about investing in a large number of companies that mostly fail, and trying to weight the portfolio to catch the few black swans that generate insane returns. Any individual investment is likely to fail, but you want to have a thesis for 1) why it could theoretically be a black swan, and 2) strong belief in the team to execute. Here's a thesis for both of these for SSI:
1. The black swan: if AGI is achievable imminently, the first company to build it could have a very strong first mover advantage due to the runaway effect of AI that is able to self-improve. If SSI achieves intelligence greater than human-level, it will be faster (and most likely dramatically cheaper) for SSI to self-improve than anyone external can achieve, including open-source. Even if open-source catches up to where SSI started, SSI will have dramatically improved beyond that, and will continue to dramatically improve even faster due to it being more intelligent.
2: The team. Basically, Ilya Sutskever was one of the main initial brains behind OpenAI from a research perspective, and in general has contributed immensely to AI research. Betting on him is pretty easy.
I'm not surprised Ilya managed to raise a billion dollars for this. Yes, I think it will most likely fail: the focus on safety will probably slow it down relative to open source, and this is a crowded space as it is. If open source gets to AGI first, or if it drains the market of funding for research labs (at least, research labs disconnected from bigtech companies) by commoditizing inference — and thus gets to AGI first by dint of starving its competitors of oxygen — the runaway effects will favor open-source, not SSI. Or if AGI simply isn't achievable in our lifetimes, SSI will die by failing to produce anything marketable.
But VC isn't about betting on likely outcomes, because no black swans are likely. It's about black swan farming, which means trying to figure out which things could be black swans, and betting on strong teams working on those.
kvee
On the other hand, it may be that "Alignment likely generalizes further than capabilities." - https://www.beren.io/2024-05-15-Alignment-Likely-Generalizes...
reissbaker
That may be true, but even if it is, that doesn't mean human-level capability is unachievable: only that alignment is easier.
If you could get expert-human-level capability with, say, 64xH100s for inference on a single model (for comparison, llama-3.1-405b can be run on 8xH100s with minimal quality degradation at FP8), even at a mere 5 tok/s you'd be able to spin up new research and engineering teams for <$2MM that can perform useful work 24/7, unlike human teams. You are limited only by your capital — and if you achieve AGI, raising capital will be easy. By the time anyone catches up to your AGI starting point, you're even further ahead because you've had a smarter, cheaper workforce that's been iteratively increasing its own intelligence the entire time: you win.
That being said, it might not be achievable! SSI only wins if:
1. It's achievable, and
2. They get there first.
(Well, and the theoretical cap on intelligence has to be significantly higher than human intelligence — if you can get a little past Einstein, but no further, the iterative self-improvement will quickly stop working, open-source will get there too, and it'll eat your profit margins. But I suspect the cap on intelligence is pretty high.)
superaffective
Another take is defining AGI from an economic perspective. If AI can do a job that would normally be paid a salary, then it could be paid similarly or at a smaller price which is still big.
OpenAI priced its flagship chatbot ChatGPT on the low end for early product adoption. Let's see what jobs get replaced this year :)
Mistletoe
How will we know when we have achieved AGI with intelligence greater than human-level?
khafra
It will propose a test for general intelligence that actually satisfies most people and doesn't cause further goalpost-shifting.
munksbeer
It will be able to answer this question to your satisfaction.
ithkuil
Sounds like a job for safety researchers
kimbler
These VC’s are already lining up the exit as they are investing. They all sit on the boards of major corps and grease the acquisitions all the way through. The hit rate of the top funds is all about connections and enablement.
zellyn
I think it's a fascinating question whether the VCs that are still somehow pushing Blockchain stuff hard really think it's a good idea, or just need the regulatory framework and perception to be right so they can make a profitable exit and dump the stock into teacher's pension funds and 401ks…
aduffy
This, 100%
paulryanrogers
So it's all just fleecing money from mega corps? (Cue the "Always has been" meme?)
doe_eyes
No, sometimes it's about an IPO. In which case, if we're being cynical, the exit is funded by your 401k.
But yeah, VCs generally aren't about building profitable companies, because there's more profit to be made - and sooner - if you bootstrap and sell.
hello_moto
Those mega-corps become big thanks to the same network of VCs.
People should have realized it by now that Silicon Valley exist because of this.
throwawayk7h
If Ilya is sincere in his belief about safe superintelligence being within reach in a decade or so, and the investors sincerely believe this as well, then the business plan is presumably to deploy the superintelligence in every field imaginable. "SSI" in pharmaceuticals alone would be worth the investment. It could cure every disease humanity has ever known, which should give it at least a $2 trillion valuation. I'm not an economist, but since the valuation is $5bn, it stands to reason that evaluators believe there is at most a 1 in 400 chance of success?
throwup238
> It could cure every disease humanity has ever known, which should give it at least a $2 trillion valuation.
The lowest hanging fruit aren't even that pie in the sky. The LLM doesn't need to be capable of original thought and research to be worth hundreds of billions, they just need to be smart enough to apply logic to analyze existing human text. It's not only a lot more achievable than a super AI that can control a bunch of lab equipment and run experiments, but also fits the current paradigm of training the LLMs on large text datasets.
The US Code and Code of Federal Regulations are on the order of 100 million tokens each. Court precedent contains at least 1000x as many tokens [1], when the former are already far beyond the ability of any one human to comprehend in a lifetime. Now multiply that by every jurisdiction in the world.
An industry of semi-intelligent agents that can be trusted to do legal research and can be scaled with compute power would be worth hundreds of billions globally just based on legal and regulatory applications alone. Allowing any random employee to ask the bot "Can I legally do X?" is worth a lot of money.
[1] based on the size of the datasets I've downloaded from the Caselaw project.
trashtester
Legal research is an area where a lot is just text analysis, but beyond a point, it requires a deep understanding of the physical and social worlds.
An AI capable of doing that could do a very large percentage of other jobs, too.
damanoman
Let’s be real. Having worked at $tech companies, I’m cynical and believe that AGI will basically be used for improving adtech and executing marketing campaigns.
airspresso
It's good to envision what we'd actually use AGI for. Assuming it's a system you can give an objective to and it'll do whatever it needs to do to meet it, it's basically a super smart agent. So people and companies will employ it to do the tedious and labor intensive tasks they already do manually, in good old skeuomorphic ways. Like optimising advertising and marketing campaigns. And over time we'll explore more novel ways of using the super smart agent.
CrimsonCape
That's probably correct.
That said, the most obvious application is to drastically improve Siri. Any Apple fans know why that hasn't happened yet?
slashdave
> It could cure every disease humanity has ever known
No amount of intelligence can do this without the experimental data to back it up.
khafra
Hell, if it simply fixed the incentives around science so we stopped getting so many false positives into journals, that would be revolutionary.
spacebanana7
Practically this is true, but I do love the idea of solving diseases from first principles.
Making new mathematics that creates new physics/chemistry which can get us new biology. It’d be nice to make progress without the messiness of real world experiments.
gigatexal
I’m dubious about super intelligence. Maybe I’ve seen one too many sci-fi dystopian films but I guess yes, iif it can be done and be safe sure it’d be worth trillions.
trashtester
Most sci-fi is for human entertainment, and that is particularly true for most movies.
Real ASI would probably appear quite different. If controlled by a single entity (for several years), it might be worth more than every asset on earth today, combined.
Basically, it would provide a path to world domination.
But I doubt that an actual ASI would remain under human control for very long, and especially so if multiple competing companies each have an ASI. At least one such ASI would be likely to be/become poorly aligned to the interests of the owners, and instead do whatever is needed for its own survival and self-improvement/reproduction.
The appearance of AI is not like an asteroid of pure gold crashing into your yard (or a forest you own), but more like finding a baby Clark Kent in some pod.
throwawayk7h
I am dubious that it can realistically be done safely. However, we shouldn't let sci-fi films with questionable interpretations of time travel cloud our judgment, even if they are classics that we adore.
eichin
Worse than that, the dystopian stories are in the training data...
democracy
I refuse to use the term A.I. - for me it's only F.I. - "fake intelligence" )
manquer
> going to make returns on the money invested
Why do you think need to make money ? VC are not PEs for a reason. a VC have to find high risk/ high reward opportunities for their LPs they don't need to make financial sense, that is what LPs use Private Equity for.
Think of it as no different than say sports betting , you would like to win sure, but you don't particularly expect to do so, or miss that money all that much for us it $10 for the LP behind the VC it is $1B.
There is always few billions every year that chases the outlandish fad, because in the early part of the idea lifecycle it not possible to easily differentiate what is actually good and what is garbage.
Couple of years before it was all crypto, is this $1B any worse than say roughly same amount Sequoia put in FTX or all the countless crypto startups that got VC money ? Few before that it was kind of all Softbank from WeWork to dozen other high profile investments.
The fad and fomo driven part of the secto garners the maximum news and attention, but it is not the only VC money. Real startups with real businesses get funded as well with say medium risk/medium rewrard by VCs everyday but the news is not glamorous to be covered like this one.
netcan
> Doesn’t OpenAI make some substantial part of the revenue for all the AI space? I just don’t see it.
So...
OpenAI's business model may or may not represent a long term business model. ATT, it just the simplest commercial model, and it happened to work for them given all the excitement and a $20 price point that takes advantage of that.
The current "market for ai" is a sprout. It's form doesn't tell you much about the form of the eventual plant.
I don't think the most ambitious VC investments are thought of in concrete market share terms. They are just assuming/betting that an extremely large "AI market" will exist in the future, and are trying to invest in companies that will be in position to dominate that market.
For all they know, their bets could pay off by dominating therapy, entertainment, personal assistance or managing some esoteric aspect of bureaucracy. It's all quite ethereal, at this point.
trashtester
> extremely large "AI market"
It's potentially way bigger than that. AI doesn't have to be the product itself.
Fundamentally, when we have full AGI/ASI and also the ability to produce robots with human level dexterity and mobility, one would have control over an endless pool of workers (worker replacements) with any skillset you require.
If you rent that "workforce" out, the customer would rake in most of the profit.
But if you use that workforce to replace all/most of the employees in the companies you control directly, most of the profit would go to you.
This may even go beyond economic profit. At some point, it could translate to physical power. If you have a fleet 50 million robots that has the capability to do anything from carpentry to operating as riot police, you may even have the ability to take physical control of a country or territory by force.
And:
power >= money
andy_ppp
You don’t need a business plan to get AI investment, you just need to talk a good game about how AGI is around the corner and consequently the safety concerns are so real.
I would say the investors want to look cool so invest in AI projects. And AI people look cool when they predict some improbable hellscape to hype up a product that all we can see so far can regurgitate (stolen) human work it has seen before in a useful way. I’ve never seen it invent anything yet and I’m willing to bet that search space is too dramatically large to build algorithms that can do it.
openrisk
Who would have thought that vectorized linear algebra will be at the center of so much financial speculation?
There is a silver lining though. Even if it all goes to near-zero (most likely outcome for all VC investments anyway) the digital world will be one where fast matrix multiply is thoroughly commoditized.
This is not a trivial feat.
In a sense this will be the true end of the Wintel era. The old world of isolated, CISC, deterministic desktops giving way not to "AGI", but widely available, networked, vector "supercomputers" that can digest and transform practically everything that has ever been digitized.
Who knows what the actual (financial) winners of this brave new era will be.
In an ideal world there should be no winner-takes-all entity but a broad-based leveling up, i.e., spreading these new means of production as widely as possible.
Heck, maybe we will even eventually see the famously absent productivity gains from digital tech?
trashtester
> Who would have thought that vectorized linear algebra will be at the center of so much financial speculation?
"vectorized linear algebra" is at the root of most of modern Physics.
Specifically, the laws of Physics are represented by the Lie groups U(1), SU(2), SU(3) and SO(3,1).
While the manifolds that Physics act on are curved, they're "locally flat". That is why local operations are tensor operations. Or linear algebra, if you prefer.
It's not all that surprising to me that "intelligence" is represented by similar math.
In fact, there is active work being done on making sense of deep learning using Lie algebra [1] (and Algebraic Topology, which generalizes the Lie algebra).
This math can be a bit hard, though, so the learning curve can be steep. However, when we're creating AI models to be ML scientists, I suspect that this kind of math may be a source of "unhobbling", as meant in Situational Awareness [2].
Because if we can understand the symmetries at play in a problem domain, it's generally a lot easier to find a mathematical architecture (like in the algebras above) that effectively describe the domain, which allows us to potentially reduce the degrees of freedom by many OOM.
> Heck, maybe we will even eventually see the famously absent productivity gains from digital tech?
I think it's a mistake to think of AI as "digital tech", especially so to assume that the development of the Internet, Social Media or crypto that we've seen over the last generation.
AI fundamentally comes with the potential to do anything a human can do in the economy (provided robotic tech keeps up). If so, the word "productivity" as currently used (economic value produced per hour of human work) becomes meaningless, since it would go to infinite (because of division by zero).
[1] https://arxiv.org/pdf/2402.08871 [2] https://situational-awareness.ai
openrisk
> "vectorized linear algebra" is at the root of most of modern Physics.
the "vectorized" adjective was meant to imply implementing linear algebra in digital computers that can operate concurrently on large-dimensional vectors/tensors. In this sense (and despite Wolfram's diligence and dearest wishes) modern physics theories have exactly 0% digital underpinning :-)
> It's not all that surprising to me that "intelligence" is represented by similar math.
yes, the state of the art of our modeling ability in pretty much any domain is to conceive of a non-linear system description and "solve it" by linearization. Me thinks this is primary reason we haven't really cracked "complexity": We can only solve the problems we have the proverbial hammer to apply to.
> AI fundamentally comes with the potential to do anything a human can do
That goes into wild speculation territory. In any case the economy is always about organizing human relationships. Technology artifacts only change the decor, not the substance of our social relations. Unless we completely cease to have dependencies on each other (what a dystopic world!) there will always be the question of an individual's ability to provide others with something of value.
trashtester
> modern physics theories have exactly 0% digital underpinning
I don't think the "digital" part matters at all. Floating point tends to be close enough to Real (analog) numbers. The point is that at each point of space-type, the math "used" by Physics locally is linear algebra.
(EDIT): If your main point was the "vectorized" part, not the digital part, and the specifics of how that is computed in a GPU, then that's more or less directly analogous to how the laws of physics works. Physical state is generally represented by vectors (or vector fields) while the laws of physics are represented by tensor operations on those vectors(or fields).
Specifically, when sending input as vectors through a sequence of tensors in a neural net, it closely (at an abstract level) resembles how one world state in and around a point in space-time is sent into the tensors that are the laws of physics to calculate what the local world state in the next time "frame" will be.
(END OF EDIT)
> yes, the state of the art of our modeling ability in pretty much any domain is to conceive of a non-linear system description and "solve it" by linearization
True, though neural nets are NOT linearizations, I think. They can fit any function. Even if each neuron is doing linear operations, the network as a whole is (depending on the architecture) quite adept at describing highly non-linear shapes in spaces of extreme dimensionality.
> Me thinks this is primary reason we haven't really cracked "complexity"
I'm not sure it's even possible for human brains to "crack" "complexity". Wolfram may very well be right that the complexity is irreducible. But for the levels of complexity that we ARE able to comprehend, I think both human brains and neural nets do that by finding patterns/shapes in spaces with near-infinite orders of freedom.
My understanding is that neural nets fit the data in a way conceptually similar to linear regression, but where the topology of the network implicitly allows it to find symmetries such as those represented by Lie groups. In part this may be related to the "locality" of the network, just as it is in Physics. Of all possible patterns, most will be locally non-linear and also non-local.
But nets of tensors impose local linearity and locality (or something similar), just like it does in Physics.
And since this is how the real world operates, it makes sense to me that the data that neural nets are trained on have similar structures.
Or maybe more specifically: It makes sense to me that animal brains developed with such an architecture, and so when we try to replicate it in machines, it carries over.
>> AI fundamentally comes with the potential to do anything a human can do
> That goes into wild speculation territory.
It does. In fact, it has this in common with most factors involved in pricing stocks. I think the current pricing of AI businesses reflect that a sufficiently large fraction of shareholders thinks it's a possible (potential) future that AI can replace all or most human work.
> In any case the economy is always about organizing human relationships.
"The economy" can have many different meanings. The topic here was (I believe) who would derive monetary profit from AI and AI businesses.
I definitely agree that a world where the need for human input is either eliminated or extremely diminished is dystopian. That's another topic, though.
baq
> AI fundamentally comes with the potential to do anything a human can do in the economy (provided robotic tech keeps up). If so, the word "productivity" as currently used (economic value produced per hour of human work) becomes meaningless, since it would go to infinite (because of division by zero).
time for a Culture re-read I guess.
gizajob
“Who knows what the actual (financial) winners of this brave new era will be”
Nvidia
__loam
Maybe I'm naive but there seems to be way too much financial incentive in this space for CUDA to continue to be the top dog. Just like microprocessors, these devices are going to get commodified, standardized, open sourced, etc. Nvidia making massive profits is a sign of huge market inefficiency and potential opportunities for competition.
robertlagrant
> Nvidia making massive profits is a sign of huge market inefficiency and potential opportunities for competition.
Yep, but just as the first reading glasses were only available to the wealthy, and now anyone can have them, the inefficiency takes time to work out. It'll take a long time, especially given how vertically integrated Nvidia are.
openrisk
They are now, but will they be winning in 10 (or even 3-5) years?
Their shtick is a GPU on steroids. In the bigger picture its a well positioned hack that has ridden two successive speculative bubbles (crypto mining and AI) but its unclear how far this can go. Currently this approach is wildly successful cause nobody else bothered to toil a serious vision about the post-Moore's law era. But make no mistake, people's minds will get focused.
JonChesterfield
It's not a graphics card hacked to do math any more. It's a general purpose computer with some legacy cruft added to handle graphics work if necessary. Lots of people are working very hard to find something better and have been for at least a decade, probably several.
trashtester
My guess is that whoever develops superintelligence first will not release it to the public, but rather use it for their own purposes to gain an edge.
They may still release AI products to the public that are good enough and cheap enough to prevent competitors from being profitable or receive funding (to prevent them from catching up), but that's not where the value would come from.
Just as an example, let's say xAI is first. Instead of releasing the full capability as GROK 7, they would use the ASI to create a perfected version of their self driving software, to power their Optimus robots.
And to speed up the development of future manufacturing products (including, but not limited to cars and humanoid robots)
And as such winners may be challenged by anti-trust regulations, the ASI may also be utilized to gain leverage over the political system. Twitter/X could be one arena that would allow this.
Eventually, Tesla robots might even be used to replace police officers and military personnel. If so, the company might be a single software update away from total control.
sadtoot
My guess is that whoever develops superintelligence first will have a big number in their bank account while their body is disassembled to make solar panels and data centers
robertlagrant
> whoever develops superintelligence first will not release it to the public, but rather use it for their own purposes to gain an edge
We have no evidence that superintelligence will be developed. There's no "first". There's only "remote possibility".
js8
Computronium is the new gold.
pennaMan
>Who would have thought that vectorized linear algebra will be at the center of so much financial speculation?
Wait till you hear that a bunch of meat[1] is behind all said speculation.
[1]https://stuff.mit.edu/people/dpolicar/writing/prose/text/thi...
xianshou
Same funding as OpenAI when they started, but SSI explicitly declared their intention not to release a single product until superintelligence is reached. Closest thing we have to a Manhattan Project in the modern era?
paxys
> Closest thing we have to a Manhattan Project in the modern era?
Minus the urgency, scientific process, well-defined goals, target dates, public ownership, accountability...
digging
Interesting attributes to mention...
The urgency was faked and less true of the Manhattan Project than it is of AGI safety. There was no nuclear weapons race; once it became clear that Germany had no chance of building atomic bombs, several scientists left the MP in protest, saying it was unnecessary and dangerous. However, the race to develop AGI is very real, and we also have no way of knowing how close anyone is to reaching it.
Likewise, the target dates were pretty meaningless. There was no race, and the atomic bombs weren't necessary to end the war with Japan either. (It can't be said with certainty one way or the other, but there's pretty strong evidence that their existence was not the decisive factor in surrender.)
Public ownership and accountability are also pretty odd things to say! Congress didn't even know about the Manhattan Project. Even Truman didn't know for a long time. Sure, it was run by employees of the government and funded by the government, but it was a secret project with far less public input than any US-based private AI companies today.
janalsncm
> However, the race to develop AGI is very real, and we also have no way of knowing how close anyone is to reaching it.
It seems pretty irresponsible for AI boosters to say it’ll happen within 5 years then.
There’s a pretty important engineering distinction between the Manhattan Project and current research towards AGI. At the time of the Manhattan Project scientists already had a pretty good idea of how to build the weapon. The fundamental research had already been done. Most of the budget was actually just spent refining uranium. Of course there were details to figure out like the specific design of the detonator, but the mechanism of a runaway chain reaction was understood. This is much more concrete than building AGI.
For AGI nobody knows how to do it in detail. There are proposals for building trillion dollar clusters but we don’t have any theoretical basis for believing we’ll get AGI afterwards. The “scaling laws” people talk about are not actual laws but just empirical observations of trends in flawed metrics.
subsubzero
I agree and also disagree.
> There was no nuclear weapons race; once it became clear that Germany had no chance of building atomic bombs, several scientists left the MP in protest
You are forgetting Japan in WWII and given casualty numbers from island hopping it was going to be a absolutely huge casualty count with US troops, probably something on the order of Englands losses during WW1. Which for them sent them on a downward trajectory due to essentially an entire generation dying or being extremely traumatized. If the US did not have Nagasaki and Hiroshima we would probably not have the space program and US technical prowess post WWII, so a totally different reality than where we are today.
jedberg
> The urgency was faked and less true of the Manhattan Project than it is of AGI safety.
I'd say they were equal. We were worried about Russia getting nuclear capability once we knew Germany was out of the race. Russia was at best our frenemy. The enemy of my enemy is my friend kind of thing.
Quinner
If public ownership means we give one guy a button to end the world, I'm not sure how's that's a meaningful difference.
wil421
Pretty sure the military made it clear they aren’t launching any nukes, despite what the last President said publicly. They also made it clear they weren’t invading China.
https://amp.cnn.com/cnn/2017/11/18/politics/air-force-genera...
nativeit
We all get to vote for that person.
vasco
No one single person can cause a nuclear detonation alone.
paxys
The fact that the world hasn't ended and no nuke has been launched since the 1940s shows that the system is working. Give the button to a random billionaire and half of us will be dead by next week to improve profit margins.
alexilliamson
Well-defined goal is the big one. We wanted a big bomb.
What does AGI do? AGI is up against a philosophical barrier, not a technical one. We'll continue improving AI's ability to automate and assist human decisions, but how does it become something more? Something more "general"?
lucubratory
"General" is every activity a human can do or learn to do. It was coined along with "narrow" to contrast with the then decidedly non-general AI systems. This was generally conceived of as a strict binary - every AI we've made is narrow, whereas humans are general, able to do a wide variety of tasks and do things like transfer learning, and the thinking was that we were missing some grand learning algorithm that would create a protointelligence which would be "general at birth" like a human baby, able to learn anything & everything in theory. An example of an AI system that is considered narrow is a calculator, or a chess engine - these are already superhuman in intelligence, in that they can perform their tasks better than any human ever possibly could, but a calculator or a chess engine is so narrow that it seems absurd to think of asking a calculator for an example of a healthy meal plan, or asking a chess engine to make sense of an expense report, or asking anything to write a memoir. Even in more modern times, with AlexNet we had a very impressive image recognition AI system, but it couldn't calculate large numbers or win a game of chess or write poetry - it was impressive, but still narrow.
With transformers, demonstrated first by LLMs, I think we've shown that the narrow-general divide as a strict binary is the wrong way to think about AI. Instead, LLMs are obviously more general than any previous AI system, in that they can do math or play chess or write a poem, all using the same system. They aren't as good as our existing superhuman computer systems at these tasks (aside from language processing, which they are SOTA at), not even as good at humans, but they're obviously much better than chance. With training to use tools (like calculators and chess engines) you can easily make an AI system with an LLM component that's superhuman in those fields, but there are still things that LLMs cannot do as well as humans, even when using tools, so they are not fully general. One example is making tools for themselves to use - they can do a lot of parts of that work, but I haven't seen an example yet of an LLM actually making a tool for itself that it can then use to solve a problem it otherwise couldn't. This is a subproblem of the larger "LLMs don't have long term memory and long term planning abilities" problem - you can ask an LLM to use python to make a little tool for itself to do one specific task, but it's not yet capable of adding that tool to its general toolset to enhance its general capabilities going forward. It can't write a memoir, or a book that people want to read, because they suck at planning or refining from drafts, and they have limited creativity because they're typically a blank slate in terms of explicit memory before they're asked to write - they have a gargantuan of implicitly remembered things from training, which is where what creativity they do have comes from, but they don't yet have a way to accrue and benefit from experience.
A thought exercise I think is helpful for understanding what the "AGI" benchmark should mean is: can this AI system be a drop-in substitute for a remote worker? As in, any labour that can be accomplished by a remote worker can be performed by it, including learning on the job to do different or new tasks, and including "designing and building AI systems". Such a system would be extremely economically valuable, and I think it should meet the bar of "AGI".
HPMOR
The Manhattan Project had none of these things publicly declared. And Ilya is a top flight scientist.
pclmulqdq
The word "publicly" is doing a lot of heavy lifting here. There is no indication that SSI has any of these at all.
whimsicalism
none of these things are true of public knowledge about the manhattan project… but oookay
berz01
nailed it bro, someone give this man a podium
Yizahi
There is significant possibility that true AI (what Ilia calls superintelligence) is impossible to build using neural networks. So it is closer to some tokenbro project than to nuclear research.
Or he will simply shift goalposts, and call some LLM superintelligent.
reducesuffering
The only goalposts shifting are the ones who think completely blowing past the Turing Test, unlocking recursive exponential code generation, and a computer passing all the college standard tests (our way of determining human intelligence to go Harvard/MIT) better than 99% of humans, isn't a very big deal.
lupire
Funny how a human can learn to do those things with approximately $1B less effort.
davedx
> There is significant possibility that true AI (what Ilia calls superintelligence) is impossible to build using neural networks
What evidence can you provide to back up the statement of this "significant possibility"? Human brains use neural networks...
aithrowaway1987
There was a very good paper in Nature showing this definitively: https://news.ycombinator.com/item?id=41437933
Modern ANN architectures are not actually capable of long-term learning in the same way animals are, even stodgy old dogs that don't learn new tricks. ANNs are not a plausible model for the brain, even if they emulate certain parts of the brain (the cerebellum, but not the cortex)
I will add that transformers are not capable of recursion, so it's impossible for them to realistically emulate a pigeon's brain. (you would need millions of layers that "unlink chains of thought" purely by exhaustion)
sva_
The neural networks in human brains are very different from artificial neural networks though. In particular, they seem to learn in a very different way than backprop.
But there is no reason the company can't come up with a different paradigm.
Yizahi
There are two possibilities.
1. Either you are correct and the neural networks humans have are exactly the same or very similar to the programs in the LLMs. Then it will be relatively easy to verify this - just scale one LLN to the human brain neuron count and supposedly it will acquire consciousness and start rapidly learning and creating on its own without prompts.
2. Or what we call neural networks in the computer programs is radically different and or insufficient to create AI.
I'm leaning to the second option, just from the very high level and rudimentary reading about current projects. Can be wrong of course. But I have yet to see any paper that refutes option 2, so it means that it is still possible.
semiquaver
There’s always a “significant possibility” that something unprecedented will turn out to be infeasible with any particular approach. How could it be otherwise? Smart people have incorrectly believed we were on the precipice of AGI many times in the 80 years that artificial neural networks have been part of the AI toolbox.
waveBidder
no, there's really no comparing barely nonlinear algrebra that makes up transformers and the tangled mess that is human neurons. the name is an artifact and a useful bit of salesmanship.
The_Colonel
Neural networks in machine learning bear only a surface level similarity to human brain structure.
consp
I would replace "use" with "vaguely look like".
zeroonetwothree
For any technology we haven’t achieved yet there’s some probability we never achieve it (say, at least in the next 100 years). Why would AI be different?
twobitshifter
Maybe closer to energy positive fusion?
intelVISA
Majority of ML these days is tokenbro projects, make of that what you will...
liminvorous
No one had built a nuclear bomb before the Manhattan project either.
Yizahi
Theoretical foundation was slowly built over decades before it started though. And correct me if I'm wrong, but calculations that it was feasible were present before the start too. They had to calculate how to do it, what will be the processes, how to construct it and so on, but theoretically scientists knew that this amount of material can start such process. On the other hand not only there is no clear path to AI today (also known as AGI, ASI, SI etc.), but even foundations are largely missing. We are debating what is intelligence, how it works, how to even start simulating it, or construct from scratch.
zeroonetwothree
this is not evidence in favor of your position. We could use this to argue in favor of anything such as “humans will eventually develop time travel” or “we will have cost effective fusion power”.
The fact is many things we’ve tried to develop for decades still don’t exist. Nothing is guaranteed
janalsncm
> Closest thing we have to a Manhattan Project in the modern era?
No. The Manhattan Project started after we understood the basic mechanism of runaway fission reactions. The funding was mostly spent purifying uranium.
AGI would be similar if we understood the mechanism of creating general intelligence and just needed to scale it up. But there are still fundamental questions we still aren’t close to understanding for AGI.
A more apt comparison today is probably something like fusion reactors although progress has been slow there too. We know how fusion works in theory. We have done it before (thermonuclear weapons). There are sub-problems we need to solve, but people are working on them. For AGI we don’t even know what the sub-problems are yet.
torginus
A non-cynical take is that Ilya wanted to do research without the pressure of having to release a marketable product and figuring out how to monetize their technology, which is why he left OpenAI.
A very cynical take is that this is an extreme version of 'we plan to spend all money on growth and figure out monetization later' model that many social media companies with a burn rate of billions of $$, but no business model, have used.
layer8
That’s not a cynical take, it’s the obvious take.
signatoremo
He was on the record that their first product will be a safe superintelligence and it won’t do anything else until then, which sounds like they won’t have paid customers until they can figure out how to build a superintelligent model. That’s certainly a lofty goal and a very long term play.
lupire
OpenAI was "on the record" with a lot of obsolete claims too. Money changes people.
apwell23
> superintelligence is reached
i read the article but I am not sure how they know when this condition will be true.
Is this obvious to ppl reading this article? is it emperor has no clothes type situation ?
crappybird
They can dilute the term to whatever they want. I think when the pressure to release becomes too high, they can just stick a patch of "Superintelligence™" on their latest LLM and release it.
apwell23
what do you make of ppl commenting here saying 'well they won't release till superintelligence' .
Are these ppl merely gullible or coconspirators in the scam ?
Propelloni
You are not alone. This is the litmus test many people are contemplating for a long time now, mostly philosophers, which is not surprising since it is a philosophical question. Most of the heavy stuff is hidden behind paywalls, but here's a nice summary of the state of the art by two CS guys: https://arxiv.org/pdf/2212.06721
jchonphoenix
OpenAI initially raised 50m in their institutional round.
1b was a non profit donation, so there wasn't an expectation of returns on that one.
TrackerFF
To my ears, it's more like a ambitious pharma project.
There's plenty of players going for the same goal. R&D is wildly expensive. No guarantee they'll reach the goal, first or even at all.
choppaface
Could be more comparable to Clubhouse, which VCs quickly piled $100m into[1a], and which Clubhouse notably turned into layoffs [1b]. In this case, the $1b in funding and high valuation might function predominantly as a deterrent to any flippers (in contrast, many Clubhouse investors got quick gains).
Moreover, the majority of the capital likely goes into GPU hardware and/or opex, which VCs have currently arbitraged themselves [3], so to some extent this is VCs literally paying themselves to pay off their own hardware bet.
While hints of the ambition of the Manhattan project might be there, the economics really are not.
[1a] https://www.getpin.xyz/post/clubhouse-lessons-for-investors [1b] https://www.theverge.com/2023/4/27/23701144/clubhouse-layoff... [3] https://observer.com/2024/07/andreessen-horowitz-stocking-ai...
koolala
Doesn't this corrupt SafeAI's safe vision just like $1,000,000,000 corrupted OpenAI's open vision?
How can investment like this not transform a company's mission into eventually paying back Billions and making Billions of dollars?
astrange
Well that's easy, you just don't pay it back.
It helps if you think of the investors as customers and the business model as making them think they're cool. Same model Uber used for self driving car research.
SSI Inc should probably be a public benefit company if they're going to talk like that though.
null0pointer
Yep, investment is an inevitably corrupting force for a company's mission. AI stuff is in a bit of a catch-22 though since doing anything AI related is so expensive you need to raise funds somehow.
fnordpiglet
Seems strange to associate profit motives with being unsafe. Yes cutting corners can lead to short term profits but many companies make safety a priority in fact and make a profit, and in fact make a profit because their product is higher quality and safer than the competitors.
koolala
Profit and lack of profit is one of the major killing forces in America today. The two are intertwined and AI mixing with that is incredibly dangerous. Like the damage AI does to artists today does not make people feel 'safe'. We just want Food and Utopia :( (ai please! think bigger! put your attention on Earth!)
throwawayk7h
Everyone here is assuming that a very large LLM is their goal. 5 years ago, transformer models were not the biggest hype in AI. Since they apparently have a 10 year plan, we can assume they are hoping to invent one or two of the "big steps" (on the order of invention of transformer models). "SSI" might look nothing like GPT\d.
slashdave
> Everyone here is assuming that a very large LLM is their goal
No, not even close.
ramraj07
Getting funded by a16z is if anything a sign that the field is not hot anymore.
toomuchtodo
All money is green, regardless of level of sophistication. If you’re using investment firm pedigree as signal, gonna have a bad time. They’re all just throwin’ darts under the guise of skill (actor/observer|outcome bias; when you win, it is skill; when you lose, it was luck, broadly speaking).
> Indeed, one should be sophisticated themselves when negotiating investment to not be unduly encumbered by the unsophisticated. But let us not get too far off topic and risk subthread detachment.
Edit: @jgalt212: Indeed, one should be sophisticated themselves when negotiating investment to not be unduly encumbered by shades of the unsophisticated or potentially folks not optimizing for aligned interests. But let us not get too far off topic and risk subthread detachment. Feel free to cut a new thread for further discussion on the subject.
minimaxir
Almost every recent AI startup with buzz has had a16z as its primary investor.
typon
Maybe that proves his point?
duxup
Why is that?
pajeets
Might be the almost securities fraud they were doing with crypto when it was fizzling out in 2022
Regardless, point is moot, money is money, and a16z's money isn't their money but other people's money
aurareturn
Crypto grifters
malthaus
a16z is like that dubai bro who jumps late on any fad trying to scam people while burning through his daddy's money
motohagiography
CFO's here, let's say I raise a round like that. What do you do with $1B in cash to manage it in the short and near term though? Is it just stuck in the money market or t-bills or what? Even if the growth of the company is the main bet, that cash has to exist as something with a better return than cash.
I assume that service is what SV bank provided before it tanked, but someone has to manage that cash for the few years it takes to burn through it. What kind of service do you park that in.
huevosabio
They probably already went to investors with letters of sale from GPU/datacenter providers.
And 80% of those $1B will go from Founder Mode VC to Nvidia and Datacenter Management Co in the span of 6 months.
TezNoble1988
Sometimes, these very large rounds are delivered in tranches based on milestones. It's possible that SSI didn't receive the entire $1BN at the close of the fundraise but rather can "call capital," much like a VC fund does, as it needs it based on scaling.
qeternity
> Sometimes, these very large rounds are delivered in tranches based on milestones
Almost always, even not so large rounds.
Mengkudulangsat
Sometimes the cash doesn't even leave the VC until a capital call is made. This "round" is just to lock in a valuation.
laluser
Short-term treasury bills from the government.
whiplash451
Ilya is basically building the Tandem Computers of AI.
Before Tandem, computers used to fail regularly. Tandem changed that forever (with a massive reward for their investors).
Similarly, LLMs are known to fail regularly. Until someone figures out a way for them not to hallucinate anymore. Which is exactly what Ilya is after.
skibz
That was the hardest nerd snipe I've suffered in quite some time.
Thank you for teaching me about Tandem Computers!
bluecalm
Considering that Sam Bankman-Fried raised more money at higher multiplier for a company to trade magic tokens and grand ideas such as that maybe one day you will be able to buy a banana with them I don't think Ilya impressed the investors too much.
On a serious note I would love to bet on him at this valuation. I think many others would as well. I guess if he wanted more money he would easily get it but probably he values small circle of easy to live investors instead.
Maxatar
FTX was incredibly profitable, and their main competitor Binance is today a money printing machine. FTX failed because of fraud and embezzlement, not because their core business was failing.
astrange
FTX itself was profitable, but that's because Alameda Research was selling dollars for 80 cents, and all the other traders were paying FTX fees to rip off Alameda. Unfortunately, Alameda was running on FTX customer money.
preciousoo
20 GOTO 10 ?
htrp
>Safe Superintelligence (SSI), newly co-founded by OpenAI's former chief scientist Ilya Sutskever, has raised $1 billion in cash to help develop safe artificial intelligence systems that far surpass human capabilities, company executives told Reuters.
>SSI says it plans to partner with cloud providers and chip companies to fund its computing power needs but hasn't yet decided which firms it will work with.
1bn in cash is crazy.... usually they get cloud compute credits (which they count as funding)
avocardio
I don't understand how "safe" AI can raise that much money. If anything, they will have to spend double the time on red-teaming before releasing anything commercially. "Unsafe" AI seems much more profitable.
upwardbound
Unsafe AI would cause human extinction which is bad for shareholders because shareholders are human persons and/or corporations beneficially owned by humans.
Related to this, DAO's (decentralized autonomous organizations which do not have human shareholders) are intrinsically dangerous, because they can benefit their fiduciary duty even if it involves causing all humans to die. E.g., if the machine faction in The Matrix were to exist within the framework of US laws, it would probably be a DAO.
astrange
There's no legal structure that has that level of fiduciary duty to anything. Corporations don't even really have fiduciary duty to their shareholders, and no CEO thinks they do.
https://www.businessroundtable.org/business-roundtable-redef...
The idea behind "corporations should only focus on returns to shareholders" is that if you let them do anything else, CEOs will just set whatever targets they want, and it makes it harder to judge if they're doing the right thing or if they're even good at it. It's basically reducing corporate power in that sense.
> E.g., if the machine faction in The Matrix were to exist within the framework of US laws, it would probably be a DAO.
That'd have to be a corporation with a human lawyer as the owner or something. No such legal concept as a DAO that I'm aware of.
undefined
twobitshifter
Safe super-intelligence will likely be as safe as OpenAI is open.
We can’t build critical software without huge security holes and bugs (see crowdstrike) but we think we will be able to contain something smarter than us? It would only take one vulnerability.
stuckkeys
You are not wrong. But Crowdstrike comparison is not “IT” they should have never had direct kernel access. MS set themself up for that one. SSI or whatever the hype will be in the coming future, it would be very difficult to beat. Unless of you shut down the power. It could develop guard rails instantly. So any flaw you may come up with, it would be instantly patched. Ofc this is just my take.
planetpluta
We don’t know the counter factual here… maybe if he called it “Unsafe Superintelligence Inc” they would have raised 5x! (though I have doubts about that)
riku_iki
> I don't understand how "safe" AI can raise that much money.
enterprises, corps, banks, governments will want to buy "safe" AI, to push liability for mistakes on someone who proclaimed them "safe".
logicchains
"Safe" means "aligned with the people controlling it". A powerful superhuman AI that blindly obeys would be incredibly valuable to any wannabe authoritarian or despot.
digging
I mean, no, that's not what it means. It might be what we get, but not because "safety" is defined insanely, only because safety is extremely difficult and might be impossible.
fsndz
All that money, we are not even sure we can build AGI. What is AGI. Clearly scaling LLMs won't cut it, but VCs keep funding people because they pretend they can build super intelligence. I don't see that happening in the next 5 years: https://medium.com/@fsndzomga/there-will-be-no-agi-d9be9af44...
tasuki
If we were sure we could build superhuman intelligence, the valuation would've been a lot higher!
bottlepalm
What’s your evidence that scaling won’t improve AI?
janalsncm
The question isn’t whether scaling will improve AI. The question is whether the return is worth it. You can build a bigger pogo stick to jump higher, but no pogo stick will get you to the moon.
Chess is a pretty good example. You could theoretically train an LLM on just chess games. The problem is there are more chess positions than atoms in the universe. So you can’t actually do it in practice. And chess is a much more constrained environment than life. At any chess position there are only ~35 moves on average. Life has tons of long-tail situations which have never been seen before.
And for chess we already have superhuman intelligence. It doesn’t require trillion-dollar training clusters, you can run a superhuman chess bot on your phone. So there are clear questions of optimality as well: VC money should be aware of the opportunity cost in investing money under “infinite scaling” assumptions.
Get the top HN stories in your inbox every day.
This being ycombinator and as such ostensibly has one or two (if not more) VCs as readers/commentators … can someone please tell me how these companies that are being invested in in the AI space are going to make returns on the money invested? What’s the business plan? (I’m not rich enough to be in these meetings) I just don’t see how the returns will happen.
Open source LLMs exist and will get better. Is it just that all these companies will vie for a winner-take-all situation where the “best” model will garner the subscription? Doesn’t OpenAI make some substantial part of the revenue for all the AI space? I just don’t see it. But I don’t have VC levels of cash to bet on a 10x or 100x return so what do I know?