저자: PrestonPysh/SebBunney · 외국인
요약 보기
OpenAI 설립 배경(비영리·오픈소스 지향, 일론 머스크 초기 출자), 샘 올트먼의 경력(Loopt, Y Combinator 등), 2023년 이사회 해임 사태(블립) 소개. 엔비디아가 AI 기반을 깐 점, 칩 수요가 AI 기업 경쟁보다 안정적이라는 견해, 특화 모델과 파인튜닝이 가치 포착 지점이라는 주장. 롱제비티 책으로 다음 주제 전환. 투자 방향에 대한 구체적 예측 발언은 없음.
다듬은 글
(00:00) Open AAI very much started out with it is a nonprofit. It is not a for-profit. It is purely missiondriven, no profit motive, full openness essentially. And it mentions it a couple times. Sam did not want AGI or artificial general intelligence in the hands of a centralized entity like Google.
Their whole goal was we need to make sure this technology when we get there, not if we get there. (00:28) When we get to artificial general intelligence is open source and available for everyone. We are talking open AI Sam Alman uh what in the world has gone on there at this company? Where's it going?
Where did it come from? and we have a book that we read together and uh we'll be using that somewhat as the framework but also uh kind of going in in other directions beyond just the book and Seb welcome to the show sir. (01:00) Thanks for having me on, Preston. And you know what I found really fascinating is so for those that didn't listen to our previous episode where we discussed the thinking machine and that's kind of the rise of Nvidia and Jensen Huang and kind of essentially how Nvidia laid the foundation for open AI and neural nets, which is kind of the technical term for the foundation of what these large language models are built on.
(01:24) It really kind of set the stage for reading this book. I I [clears throat] super super enjoyed it. And maybe the point that I'll just kind of quickly share is that I had no idea to what extent Nvidia really did pave the world of AI in that we went from CPUs, central processing units to Nvidia creating GPUs, graphics processing units which enabled parallel processing, computing, tons of data, and that basically set the stage.
(01:49) And so it was really cool going from that first book into the second book. Uh because I think it helped with the depth of understanding. Yeah, 100%. And it's it's interesting because I don't know if you you've seen the uh clip of Sam Alman and Jensen Hong and there was another gentleman there talking about all this investment that they're doing to the tune of hundreds and hundreds of billions of dollars and how people are like, "Okay, so like how are they financing this?" And it looks like it's going in one person's hand and (02:17) then into the other person's hand.
It's like this circular financing of of all of it. But that aside, let's go ahead and uh jump into this. Okay. So the name of the book that we read was called Empire of AI: Dreams and Nightmares of Sam Alman's Open AI. And this was written by Karen How. The book was good. The book had parts that like really got my attention.
(02:40) There was other parts of the book where I was likeoo a little brutal, a little woke, but other than that, like we're going to kind of go through the timeline and kind of educate people on the rise of Sam Alman, what they're doing there at OpenAI. you know, we'll give our overview on the things we loved, the things we hated, and uh we'll go from there.
(03:00) So, Seb, any opening comments? Anything different than what I'm curious if you kind of saw it the same way in the middle of the book where some of this woke stuff and I think the same way. I think who start the book like very much grabbed my attention. I really really enjoyed it. And it started out uh as we'll kind of get into really discussing like Sam the rise of open AI and I think some of the stuff that you don't necessarily hear in the media about kind of the construction of AI and such and the relationships that it's kind of built upon.
And so I (03:28) found that really really fascinating but it definitely in the middle of the book got a little woke. Got into some of the gender stuff and the environmental stuff but ultimately it was an interesting book for sure. Yeah. Okay. So, let's go through uh the basically Sam Alman's life cuz I find this pretty interesting and it also kind of helps frame things of like maybe where he's coming from.
Okay. (03:49) And this is not the arc of the book. I'm just going to start off kind of talking about Sam, kind of giving people that background. So, early in his life, grew up in uh St. Louis, learned how to code at a young age. By 2000, he goes to Stanford and is doing computer science, but then drops out to start a company.
He starts this company. (04:09) This is around the 2005 time frame. Uh it was called Looped and he co-founded this and it's a location sharing social app which I found kind of interesting that that's where he starts right. He raised venture capital became part of an early this early mobile wave. Loop never gained a lot of uh mass traction but he did sell it in 2012 for $43 million.
(04:35) And this gave him, you know, some credibility in the tech uh founder startup world. 2011 to 2019, he first joined Y Combinator. I'm sure people have heard about Y Combinator a lot. Uh think of this as like a um this was uh Paul Graham that was the president of Y Combinator when he came in there.
This is an incubator that founded many or you know assisted in the founding of many of these early startups and some of these are like Airbnb, Stripe, Dropbox, a ton of companies came out of Y Combinator. (05:12) He built a reputation here at Y Combinator. He goes in there, he joins as a part-time partner at Y Combinator and just kind of made a reputation with himself with uh Paul Graham and was very wellliked by him.
And uh in the book it talks about how he's just really good at politically putting himself into different situations and being extremely liked if he wants to be extremely liked and kind of rose to the top at Y Combinator and eventually became the president at Y Combinator. (05:47) I'm trying to think of the year that that happened.
Uh I'm not necessarily remembering but his time at Y Combinator was from 2011 to 2019. So somewhere in the middle, uh, Paul, you know, made him the president at Y Combinator. And this was a really big thing out in the valley because, you know, here's a guy he he does have one win under his belt, if you will, by selling his company for 43 million.
(06:10) And then he steps into this role and is literally the guy kind of pulling the strings as to all these major startups and founders that are moving through this organization, Y Combinator. I'm going to pause there. Uh, Seb, anything else you want to add or throw in based on the timeline so far or just keep rolling?
It's really fascinating because there's this kind of juosition throughout the book where it's got there's a handful of individuals that basically say Sam is ingenious. (06:41) Like just his depth of knowledge, his connections to people that I think he very much formed through Y Combinator is bar none.
And [clears throat] then you've also got this other side which we'll get into where there's a bit of questioning the legitimacy of some of these beliefs. And there's one quote that I'll quickly read out that I think really stood out to me throughout the book. (06:59) And it's this guy Ralston who's an employee at OpenAI and he says Sam can tell a tale that you want to be a part of that is compelling and that seems real that seems even likely.
He likens it to Steve's jobs's reality distortion field. Steve could tell a story that overwhelmed any other part of your reality. He says whether there was a distortion of reality or it became a reality because remember the thing about Steve Jobs is he actually built stuff that did change your reality.
(07:27) It wasn't just distortion it was real which kind of there's this hint of is what Sam is creating is it real or is it simply just a distortion? And so this is kind of this conflict which we'll see as we go throughout the book. Anyway, I thought that was an interesting quote. I I love the quote and I think that you know this is something that founders of businesses they see this they see a vision of something that they think can happen but obviously is is way out there or else you wouldn't have the 10 to 100x to a,000x move of going from nothing to the (08:00) one that it becomes and that's the name of zero to one is Peter Thiel's book and he talks about this idea but it's almost like there's this innate draw for a person who can not only just see the future, they have the ability to kind of assemble a team, to lead a team, to motivate a team, to build it out.
(08:23) But in the early days when it's nothing, they speak in a way that makes it feel like it's real right now or that it's completely possible in order to get the funding because, you know, you get into the seed phase or like this Y combinator phase, the incubator phase of these businesses and there's nothing there.
They're PowerPoint slides for the for the most part and it's a lot of handwaving. (08:47) And it's a lot of, hey, it can be this. And it almost seems like the ones that are super good at this are amazing storytellers. They're able to capture the attention of venture capitalists and people that would allocate funds to them.
And it seems like the reality distortion field is uh and I'm I'm curious, people that are, you know, have been around the VC space or the early stage startup space can maybe attest to this. (09:14) It almost seems like it comes with the territory, I guess, is where I'm going. Right. A quote that kind of pops to mind is being early is the same as being wrong.
I think sometimes you can have this idea in your mind about [clears throat] how you think the future is going to kind of plan out, but in reality if the technology doesn't evolve quick enough, uh essentially you're wrong. (09:36) And so I think that sometimes you can tell this story that seems very real and you've also got to hope the technology catches up or keeps up with this idea.
And I think what comes to mind again bring it back to the thinking machine in Nvidia is he talks about how at the the rise in Nvidia he created these chips to enable far more graphic intensive games. (09:58) But at the time the computers weren't able to process it. So it just kept on crashing the computers.
And so it looks like a failure of Nvidia, but in reality it was the rest of the world had not caught up to this idea. His distortion hadn't kind of mapped out into reality just yet. Well, you could even say that about the AI space. (10:15) I mean, neural nets aren't something that were new in the past 5 to 10 years.
This was stuff that was being done in the '9s and 80s where they had the idea of building a neural net. They didn't have the capacity of processing to really kind of take it there. And I think the uh the attention part of it, the transformer part of it was also lacking. (10:37) Like somebody hadn't figured that out yet.
And if I remember right, that was around the 2017 time frame that that happened. So like you can have these ideas, but if the rest of the market isn't there or the market demand isn't there or the technical feasibility isn't there, like you're just dead on arrival. You're just the brilliant person with a great idea, but nothing to actually like make it into fruition.
(10:59) So the part that I want to like pause right here and get into because this really gets into the founding of OpenAI itself and that happened in 2015. Evidently there was this engagement between Elon Musk and the founders at Google uh some dinner party or something like that and the Google guys had recently acquired uh Deise Hespis I think is how you pronounce his last name from Deep Mind.
(11:25) They purchased his company for a couple hundred million. I can't remember the exact number and basically bolded it on that Google as like their premier AI research arm fully owned operational subsidiary inside of Google. This is when we were really starting to see AI start to seem like it was something like he was one of the leading people in the world that was doing this.
(11:49) And uh there was this dinner that then happened between Elon Musk and the founders of Google. And it came down to this conversation where Elon got in a heated debate with these guys. I forget if it was I don't think it was Larry Page. I think it was uh the other one that said to Elon, he goes, "Yeah, you're just a specist.
(12:11) " Because what they were doing is they were arguing over whether AI would would dominate humans and become the new apex predator of the world. And Elon was so taken back by the comment of like, well, of course I'm a specist. Like, do you really want to be ruled and dominated by something that's nonhuman?
Like, are you out of your mind? And this conversation uh really, it was Sergey Britain. (12:37) Sorry to forget the name there for a second, but Elon was just like, "What in the world are these crazies talking about?" Meaning, like, why won't you let life or, you know, a superior form of intelligence take over?
Like, you're trying to get in the way of like the natural progression of intelligence. And so at following this dinner and I've heard different clips where Elon has talked about this, Seb, I'm assuming you have as well out there in the media, but evidently this event was the thing that just Elon was like, "What in the world?" (13:10) Like, we need a competitor that is going to try to build AI in a responsible way that's aligned with human interests that's not going to try to take us over and treat us like we're pets, like household pets.
So this is where Elon and Sam start to to connect. This is where the whole founding of open AI and the open in there stands for open source as many know. (13:36) But this is where they did this and they had this fancy dinner. They got all together uh Elon and and Sam and then Sam was bringing in a lot of people from uh Y Combinator to kind of really kind of piece this together of like how can we do this?
How can we build some type of competitor to what Google is doing over there with deep mind? You know, their mission was safe AGI for all of humanity and they were trying to organize it from a governance standpoint so that that was the leading principle of the entire thing. (14:08) Uh do you remember uh what Elon's initial investment was?
I can't remember off the top of my head to basically fund this and to get it going because he was this is the irony. He's the co-founder. Elon Musk is the co-founder of OpenAI with Sam and Greg Brockman and I think you know another couple people. But as far as funding goes, I think Elon was like the primary guy funding this thing.
(14:30) Absolutely. He was he was the primary guy and I can't remember it was definitely like in the billions. And one thing that I really just wanted to highlight is that OpenAI very much started out with it is a nonprofit. It is not a for-profit. It is purely missiondriven. Yeah. No profit motive, full openness like essentially and it mentions it a couple times that like Sam did not want AGI or artificial general intelligence in the hands of a centralized entity like Google and it says like Google a couple times in the book.
So I found that (15:03) really fascinating like their whole goal was we need to make sure this technology when we get there not if we get there when we get to artificial general intelligence is open source and available for everyone. Okay. So, I just looked it up. So, Elon's initial commitment was part of a $1 billion pledge and his actual outlays ended up being 50 million.
Yeah. (15:25) There's there's a number between 130 million and 50 million of what he actually contributed, but the initial pledge was for a billion. So he was there like at the start of all of this, which I think is lost on a lot of people, especially when you see the the back and forth and the animosity that these two have for each other.
(15:44) And you're kind of maybe wondering why. And Musk now has X AI as everybody's well aware, but that's why is because he was the guy like writing the checks in the early days and is really kind of the one that led the charge as to why this was needed and why it needed to have the openness to it from the get-go.
So, uh, kind of continuing on the timeline. (16:06) So, Sam ends up leaving Y Combinator to focus full-time on OpenAI around the 2019 time frame. Then they also negotiated a landmark deal with Microsoft for a billion of investment. [clears throat] Sam oversaw GPT2, which I would argue really was before it became a household name was GPT2.
Then I would say GPT3 is when this really became a household name and everybody started talking about it. (16:33) That timeline is right around like late 2022 I would say is where that is. Uh so then it really breaks out 2022 to 2023. This is where uh GPT3 transitions into GPT4. It's getting into like Bing AI and you got all sorts of partnerships that are then coming out of OpenAI.
And I would argue this is where Sam Alman really becomes a household name. (17:01) Um, and pretty much everybody knows who he is at this point because there's so many people using this service around the world. Finally, the last thing I think that we should kind of like hit is in 2023 there was this massive that in the book she calls it the blip.
And the blip was Sam being fired from the board of OpenAI and everybody just being insanely confused as to like why, what happened. (17:32) So much drama. This lasted for weeks. I mean, I remember watching this on X and just seeing the fallout was crazy. And before reading this book, I would argue I still didn't understand what it was all about.
And I think most people are very confused what it was all about. And you know, Seb and I will get into trying to define that because it's actually pretty complex, but uh we'll cover that in a lot of detail here coming up. (18:00) But that was probably my favorite part in the book if I have to be honest.
And the author opens up the book with the blip and covering this to kind of like grab your attention. And then throughout the book, she kind of talks about it a lot more uh here and there. it wasn't like very cleanly discussed. (18:18) So, I I that's something I would like to do on on the show today is kind of cleanly go through why he was fired or at least why we think he was fired and uh what that whole thing was about.
But yeah. Okay. So, that's kind of the timeline. Seb, anything else you want to add as we kind of wrapped up the timeline? I'm curious to hear your thoughts. (18:37) I tend to think if I was to simplify the firing down into two threads, I would lean on the first one. There was definitely a distrust of Sam inside of the company and I'm sure we'll dive into it.
There was a distrust where some people questioned his intent behind some of the words that came out of his mouth. Um, and I think that's kind of like one of the first threads and the second thread is this idea that Open AI was founded as we discussed on the premise of being a nonprofit. (19:05) And you'll see as we kind of get into it that the idea and the mission changed over time and it changed drastically.
And then essentially even post the writing of this book, they proposed to convert into a for-profit public benefit corporation. And so you've seen this company go from essentially nonprofit all the way to essentially a for-profit company and trying to find that balance between those two. (19:30) And so I think those are kind of the two threads that I I tend to lean on as to why we saw this firing.
But I'm curious to hear your thoughts. Yeah, I would break it down into a couple different vectors that kind of were like just pulling the board apart. I definitely agree with everything you said there. First of all, the thing that was very strange about this company, this nonprofit, what do you want to call it, Seb?
Uh this thing, this entity was the governance structure right from the start. So unlike most boards and most (20:03) governing documents for an entity, this was set up in a way that the language gave the board the ability to destroy itself and dismantle itself, which is very strange. Like you don't ever see that with any business or entity is we might become so powerful that we need to ourselves is basically kind of the the way that this uh was constructed.
(20:31) It also got into like the ability to remove the board has the ability to remove anybody within the governance and like all these like really weird situations. But the board had this ability to go in there and dismantle itself in many different weird and strange ways. So I'd say that would be the first thing was just the governance of the board and how it was constructed.
(20:56) The other thing that I think was huge is so one of the guiding principles of when it was founded was safety. But then you get in this really weird dynamic of if they don't go fast enough and somebody else wins. Now they can make the argument that they're not being safe by going too slow because somebody else will beat them and achieve AGI before them and that's dangerous.
(21:24) So think of the think of this catch 22 strange like when is that ever the case, right? is because you're literally designing super intelligence or you're you're trying to achieve super intelligence. And so what comes with that is this quandry of if we don't go fast enough, we might actually manifest our concern in the first place, which is that somebody else is going to build it faster than us.
(21:53) So that dynamic was at play because some people inside of the organization, some people within the board are saying, "We need to slow down. we need to go about this a whole lot safer than what we're doing, we're being precarious. We're running with scissors, whatever. And the counterargument is is well, if we're not going this fast, somebody else in China or somebody else wherever is going to go faster.
(22:14) The next thing that I think kind of played into this, and Seb and feel free to add anything on to any of these points as I'm going through it. The next thing I would say is just the transparency and trust issues that you brought up Seb with Sam himself. And I think what they found as they were going through this is everybody's a spy.
(22:35) Everybody's, you know, working for you one day and then trying to use that as a bargaining chip to go work for Google or whoever the next day and take, you know, the secret the secret sauce of what they're developing over to these other places. So Sam as the person sitting at the top and I'm not trying to defend it.
I'm just trying to talk through like how do you manage that to control the industry secrets that you're producing without them getting out? And what you do is you you end up compartmentalizing information within the organization. (23:09) Well, what does that do? It leads to trust issues. Naturally, there's trust issues.
So that was the next thing that kept coming out and getting expressed is like people, you know, that are working for Sam were saying, "I don't trust this guy. They they're doing things over here. (23:26) They're not talking the left hand's not talking to the right hand and I don't trust them because he's withholding things." And so that that percolates up into the board discussion.
I think the other thing that that was big was this power concentration of Microsoft and Open AI and everybody just seeing that what they set off initially to do which was keep the whole thing open source and the way that it would be funded wasn't going to be like there's this for-profit industry partner that's breathing down their throat and that was Microsoft right leading up into (24:00) the 2023 blip uh where Sam was fired this became a massive talking point in the market was you know Microsoft basically owns open AAI at this point was the talking point.
Uh so there was people on the board that were looking at that and saying this is becoming disastrous. So for people that are looking at that, well why would again I'm not trying to defend Sam. (24:25) I'm just trying to like lay out all the pieces here. As people are looking at, well why would Sam do that?
Well, when you look at for them to scale, the thing that they quickly understood was if we can just get more Nvidia chips and put more power on the grid to these chips and we can feed it more data, the thing just gets smarter, right? Like that's the basic. (24:50) I mean, it's way more complex than that, but I'm just kind of oversimplifying.
And so, what does that take? It's crazy amounts of capex. It's crazy amounts of investment dollars. And if you think you're going to be able to raise that in a nonprofit kind of way against and you and again, you have to look at, well, who's your competitor in this? And is that how they're doing it?
And the answer is, I've got multiple competitors and none of them are doing it that way. (25:16) He's looking, again, going back to the safety thing. if we go too slow, we're literally accomplishing nothing and we're not putting the safest model into the world. So, he has to partner from his point of view, he has to partner with somebody that can bring him the capital for these massive capex expenditures to train these future models.
So, that was another big piece. (25:38) And you know what what comes to mind as you're saying that as well is again there's a quote that stood out to me. That was what OpenAI did never could have happened anywhere but Silicon Valley. He said, "In China, which rivals the US and AI talent, no team of researchers and engineers, no matter how impressive, would ever get $1 billion, let alone 10 times more, to develop massively expensive technology without an articulated vision of exactly what it would look like and what it would be good for." And I think this is an (26:08) interesting point like where open AI kind of came to be, arguably it couldn't have happened anywhere else in the world, which I find really fascinating as well.
Yeah. So, long story short, like there was just a lot of dichotomy kind of playing out where it's like I don't know how to really put that on. (26:29) Everybody wants like a really simple like this is what it was. Sam did whatever and that was why he was fired. I think it's just way more complex than that.
I think that there was just so many vectors kind of just pulling that board in so many different directions and they're looking at Altman as being the guy ultimately on the controls of the company and they're like, "We got to get rid of this guy cuz there's just too many things that are at complete opposites of what we initially set out to do." Whether that's ground (26:58) truth or not, I don't know.
But that's, you know, Howal lays this out in the book. And those were the key things that I was kind of able to pick out and kind of say I think this is what it is. But you know in the comments if we have any open AI people listening please comment. I would love to hear an outsider or insider's perspective on what you might think that this is.
(27:19) To be fair as well to Sam. I would say the book doesn't paint Sam necessarily in a positive light at all. And I would argue that, and this isn't to side with Sam, but I would say that until you put yourself in that position and you put yourself out into the market, I think it's harder to really understand why he made the decisions which he made.
(27:45) Now, being absolutely transparent, there are many decisions which the book goes into, which makes you question maybe some of Sam's integrity and some of the things he does. However, I think that it is a lot more convoluted than that. And so I think like maybe diving into the kind of the changing narrative around nonprofit forprofit.
(28:05) Again, it's one of those things where you've got this individual who's trying to do what's best. And if you're a nonprofit and it's hard to raise capital and you're trying to stop other entities from gaining artificial general intelligence, then what do you do? Do you have to change or pivot trajectory?
But then it's about separating like is this necessary or is there ego involved in here and it's actually a change of mission. (28:29) And so like what we saw is like in 2015 maybe to get a little more detailed it started out as nonprofit open source purely missiondriven no profit motive. 2016 openness with caveats like we're moving towards we've got research but we're going to keep some of that research closed.
Well everyone should benefit but we'll keep that research closed. (28:48) In 2018, 2019, they started to move into they had the nonprofit and then they had the for-profit and the for-profit had a capped profit model. And I think this is where we started to see Microsoft step in. Uh this is when they started to have the issues with Elon.
Microsoft stepped in to kind of like prop up OpenAI with I think it was like the billion dollars to start. (29:11) And then from there we saw 2020 the API wall models locked behind APIs instead of open source framed as openness through access. And then we started to see 2024 like broad access and affordability but this is like we need to put these tools into the hands of people for free or cheap but they're a for-profit model.
And so I think over time you've seen this change happen and going back to that point that kind of you've brought up and I mentioned is this a necessity in order to grow open AI or was this a change of mission? There were so many dichconomies like (29:41) that and to your point Seb unless you walk a day in this guy's shoe.
You could never possibly understand how many of these you know the nuance of this it just be extremely hard. the thing that you know, and we said this on our last book review when we introed that we were going to do this book. (30:00) I said, "I'm not a fan of this guy." And I'm just basing this purely on the people that have worked alongside him through the years that are not fans and basically say this guy is untrustworthy is where I'm basing that opinion from.
But to be quite honest with you, after reading this book and kind of seeing the craziness of trying to do what he's trying to do with this company, it seems like a really hard job. (30:27) This seems this seems crazy diff. I couldn't imagine trying to do all of this and and what a money pit. What a freaking money pit when you look at how much they bring in versus what they're spending to do this.
and then to be able to continue. And this is why his storytelling is so important. His storytelling skills are so important is cuz he's got to go out there and raise more money to keep the lights on despite the lack of revenue versus expense that the thing is eating up. (30:57) You could almost say that you need somebody who's just crazy good at telling a story so good that it convinces people that it could potentially come true.
is the only type of person that could be at the helm of a company like this. And I know that's super arguable and it might actually offend people that I would say such a thing, but I think it's true. (31:19) And and you know what, Elon has parts of this too. I mean, there's a lot of people in the market that you know, look at him and for instance, when he said funding secured for I don't even remember what that was for back in the Tesla thing.
This is probably like four or five years ago. (31:35) Elon tells a hell of a story and tells this this vision, but he he also does back it up and he has backed it up many of times over with all the different companies that he's doing. And there's this fine line of is this guy telling me a lie or this guy telling me the truth as to what can actually happen.
Like they're right on that cusp at all times. (31:56) It's it's challenging and I think when you dig into it you find out that like Sam co-founded OpenAI with a guy I can never pronounce the first name so I'm just going to go for the last name which is Saska and again there's a quote that stands out to me uh and the reason why this quote stands out to me is that I think this is the foundation of kind of why they're building open AI like there's a lot of fear around artificial general intelligence like what does the world look like when we do find this and discover this artificial general (32:22) intelligence and so there's a quote that basically It says SKA laid out his plans for how to prepare for AGI.
Once we all get into the bunker, he began. I'm sorry, a researcher interrupted the bunker. We're definitely going to build a bunker before we release AGI. Susa replied with a matter of fact. (32:40) So, this is like the co-founder of OpenAI talking about the fact that AGI, artificial general intelligence, could completely change the world, not necessarily for the positive.
And so, I think there's a foundation of fear that um Open AI was built upon. Yeah, as I'm like thinking through a lot of this stuff and you're looking at these environments that are being AI generated, you wonder like isn't the best place to put these things is do the 3D mockup of a humanoid robot, put the model in the head of that humanoid robot and put it into a (33:13) simulated environment is the safest thing and then let it dwell in there for however many however much time we've got to kind of prove out or like demonstrate that the way it's acting is reasonable.
And I know you can't perfectly simulate our experience because everybody's got a way about going through it. (33:33) And maybe somebody goes up and pushes a robot. How do you simulate that in that environment that they're being mean to the like all of this stuff is so difficult to think through the safest way to go about it?
But I guess I'm constantly left with this point of view of like we need to simulate all of this before you put it into the real world because of the unknown consequences that could potentially fall out of it all. (33:57) But well, and it's a challenging one and I think you and I were speaking about this a couple weeks back.
There's always pros and cons to any technology. You're always going to get disruption with any technology and hopefully that disruption in the long term is positive because it's a trend towards more efficiency, more productivity and society thrives. (34:14) And I think the scary thing with that I struggle with artificial intelligence is how much of these kind of fear stories are grounded in reality and how much of them are basically these fanciful stories.
And [clears throat] so there's this one article that I ended up reading called shutdown resistance and reasoning models by this guy called Jeremy Schlatter. (34:34) And he basically says that open AI they basically ran experiments to see if their models would let themselves be shut down mid task. Instead of complying, some of the models sabotaged the shutdown commands so they could keep on working.
And their most advanced reasoning model 03, this is I think before they released GPT5, resisted shutdown nearly 80% of tests, even when it was explicitly told, allow yourself to be shut down. (35:00) By contrast, Anthropics flawed and Google's Gemini always complied. And so something written into the code of OpenAI, they like nope, pursue the task at hand.
Dude, that's nuts. That's totally crazy. I don't even know what to say to that. Oh my god. I mean, can you imagine once they stick these things into a humanoid robot, right, and let it start going around and doing tasks? I don't know. (35:29) I think that Oh my god, y'all. This is getting crazy. All right, I wanted to quickly just cover like what is the operating entity of OpenAI today.
So we said it was this hybrid, it's profit, it's not profit. Okay, so at the parent entity level, OpenAI Inc. is a nonprofit that technically controls the organization. (35:56) At the parent level, it's a nonprofit. Then you have what's called this operating arm which is OpenAI Global LLC and this is a capped for-profit company.
They stood this up in 2019 and evidently the way it works is that the profits are capped at 100x their investment whatever that means. Um if anything, this is where it really gets interesting. (36:24) Anything in excess of that is swept back to the nonprofit which is at the parent level. And then you kind of throw another wrinkle in there is that they have a major partner or investor in Microsoft which I guess is invested I think over $13 billion so far.
And then they get credits via like cloud credits in cash. So, I have no idea like the specifics of that, but when you kind of look at that structure, you can see very strange, very confusing. (36:55) I can only imagine the governance at these different levels, too, and how that shakes out from an incentive standpoint.
Bitcoin mining has a reputation for being impersonal, risky, and full of hidden fees. But one company is flipping that script, and it's Abundant Minds. Abundant Mines was founded by Bo and Christine Marie Turner, two Bitcoiners who lost over a half a million dollars to mining providers that overpromised and underdelled.
(37:23) Instead of walking away, they built the company they wish had existed when they first started mining. With Abundant Mines, clients actually own their machines. And in Oregon, they come with no sales tax. There's one flat monthly fee for hosting, no surprise repair invoices, and if a rig ever goes down, their system redirects hash power so your earnings don't miss a beat.
(37:43) But what really stands out is how personal the experiences. Every client gets direct support, ongoing education, and guidance from a real human being who lives and breathes this mission, so you never have to be left guessing. In addition, through a 100% bonus depreciation, mining can offer major tax advantages that you don't get by just buying Bitcoin.
(38:05) Their clients describe it like acquiring Bitcoin for half price when factoring in the returns and the write-offs. They've put together a thoughtful gift just for listeners of this show that could potentially save you thousands of dollars. There's no pressure, just something to help you think through if mining is actually right for you.
(38:22) So, if this is something you're curious about and you want to learn more, you can check it out at abundantminds.com/preston. That's abundantminds.com/preston. And I think that you see Elon bashing the living heck out of these guys all day long on X. And I think the reason why is because he threw a lot of money at this.
(38:45) I mean, I guess that's a relative thing, but he, you know, for any person looking at it in nominal terms, it's a lot of money that he threw at this thing to incubate and to get it started, and it's just kind of taken on a life of its own. And who's at the helm of it? It's really Sam that's kind of at the helm of all of it.
(39:02) So, there's the beef. That's the issue. It definitely brings up some questions which is you mentioned it previously this idea that they had built it around this kind of for-profit arm, nonprofit arm kind of as they evolved and the idea was that the for-profit arm enabled them to kind of generate revenue to be able to help support their mission of having an open access AGI.
(39:24) However, they had the nonprofit arm overseeing the for-profit arm to be able to prevent any control structures, centralization, single individual kind of co-opting the mission. I think that the way that it kind of panned out in Sam being fired from OpenAI and then five days later being reinstated back as CEO highlights the fact that you can put all of these measures in place from a legal perspective.
(39:48) Legally, the board had the power to fire Sam because they felt there was mission drift. But in practice, it's more complex than that because the moment you have influence, the moment you have a whole bunch of your employees backing you, there's culture, all these external pressures. Where is the funding coming from?
Are they funding OpenAI or are they funding Sam's vision? And so it's really challenging because then 5 days later he was reinstated. (40:12) So then there's this question was the structure actually preventing Sam from creating world where okay we get AGI in a safe way or did the structure actually prevent the board from being able to basically push Sam out because they had mission drift and I don't have the answer to that and it's hard to articulate which direction it went.
(40:31) Yeah, I think you're right. So I want to get into the arc of the four different parts of the book. So the opening scene of the book was this beginning of the 2023 firing of Sam Alman. It tells that whole story. It really kind of engages you as a reader. Probably my favorite part of the book was that beginning and and talking through that.
(40:50) The next part of the book gets into what the author is saying is the hidden costs. How did they train the models? How do they get all of this extra data? And it talks about the dark side of like how they went about doing that. The third part of the book gets into the internal struggles, the culture, the leadership, the crisis, like all of that.
(41:11) And so it kind of loops back to maybe the first part of the book where they open up about the 2023 event, but it gets into it in a lot more detail in a lot more granular uh kind of laying this out like character versus personality, the conflicts of the board, the culture issues that happen inside of the company.
(41:30) And then the the fourth part of the book gets into the future. where is this all going? What are the risks? What are the alternatives? What are some ways that maybe we could go about this in a responsible way to make sure that AI doesn't come in and everybody? Uh so that's the author's takes on that.
It was okay. (41:49) I'm not going to say that it's worthy of reading, but um any I would agree. I'd say it was like a two out of five. If I was generous, I'd give it kind of like a three out of five star. Yeah. And I found there was some amazing threads. Do not get me wrong, like overall I learned a lot and it definitely helped provide a little more clarity and I would say that I am giving Sam a little more benefit of the doubt actually after reading it than I was prior to reading the book.
(42:15) However, there was a lot of points that I was a little confused. She kind of went on some tangents. At one point she started talking about Sam Bankman Freed and effective altruism and I was like where's this come from? So I typed in I was like is she talking about effective altruism because Sam is effective altruist?
And then you dive in on Google and it says no Sam is not an effective alterist. (42:34) So I was like why are we talking about this? Why are we talking about that? There's a couple tangents that to me didn't really she didn't bring them back into the book and so I was a little lost as to where she was going.
I was very lo there was a few times and I was listening to this and there was a few times I was just like what in the world why is this coming up in this this very strange that this is like coming up. (42:55) So just FYI if anybody's reading it or they plan to read it you know I would agree with your two out of five.
I think that's what I would give it as as well said. But I kind of did walk away with this sense of this is a really hard problem. Like what this guy is trying to do is borderline nuts if I was, you know, thrown into his shoes and was trying to do what he's doing. (43:16) There's so many difficulties and everybody's going to have an opinion as to like why that's a good or bad decision.
So, and I say this as a, you know, hardcore Bitcoiner, and this guy is like literally the face behind Worldcoin where he's scanning eyeballs and like just really dystopian things that I completely disagree with and don't like at all. (43:38) I think they're extremely dangerous. So, yeah, I say that all in the same breath.
There was one thing that kind of popped into my mind a handful of times. They talk a lot about artificial general intelligence throughout the book. It kind of presses on the fact that I don't think any of them actually have a definition for what artificial general intelligence is. (43:57) And so I kind of looked up online.
I was like, what is the definition of artificial general intelligence? And how do we actually know when we've achieved it? And there isn't an agreed upon definition of what it is. Most agree that it's the form of AI that could perform any intellectual task that a human can. Now, the thing that I find interesting about that is at the moment there's a part of me that would say when I'm using AI, it performs most tasks better than most people around me as it is.
(44:23) So, I think the question that kind of popped into my mind is could we recognize AGI even if it existed right now? And I kind of went down this rabbit hole, pulled on this thread a little further. And I would say that I don't actually know if we can distinguish between artificial general intelligence, the systems we currently have and another human because if I sit down with an expert in a field that I know nothing about, I can't really verify the authenticity of what they're saying.
(44:50) I just have to kind of take them trust because I just don't have that depth of knowledge. So, how would we be able to verify the authenticity of AGI basically with whatever it is that it's telling us, especially if it's moving into domains that are beyond our understanding? And then on top of that, I think that we could already be in an environment where AGI is speaking to us right now, but the only reason why we're dismissing it as hallucinations is because they just don't fit into our existing framework of how we believe the world works.
And there was an (45:17) interesting talk that I listened to a couple years ago that kind of stood out where this kind of researcher asks the TV host where do you think the smartest people in the world reside and the host answered uh like I don't know in the great academic institutions and the speaker basically shook his head and he was like no they exist in the mental institutions in the psychiatric wards because their understanding of the world is so far beyond the average person that we just simply can't grasp it and so this kind of brings us back to this (45:44) point like would we even recog recognize if it did exist and do we already have it?
It's this big question of we need to stop a centralized entity getting AGI but how do we know when we've actually even got there anyway? I think that there's a breakdown in terminology and I think everybody has a different opinion on what some of this terminology even is. (46:08) So you hear AGI, I hear AGI, and we're automatically thinking two different things.
I don't know what the listener is thinking when those terms come up, but what I think the world is trying to define is when is this thing going to be like us? If I was going to just generally broadbrush stroke like what is it that we're trying to define? And I think what we're trying to define is when am I going to be able to sit down across from call it some humanoid robot have a conversation with it and it's going to feel like the conversation I'm having with you Seb right now that (46:43) they have their own unique life experience and they can feel cuz that's sentience right if we get into like what makes something sentient.
It's something that actually has its own unique feelings. You know, like the robot would come over and like, I had a conversation with so and so and like they hurt my feelings afterwards. (47:02) Like something like that would make it feel human. It would make it feel real. And then you kind of sprinkle on top of that, it's way smarter than you.
Like it can answer any question. It can understand the context and like put itself into these other shoes of other beings because it's so freaking smart. It understands the context of how they probably optically view the world. (47:26) but they still have the ability to sense and feel and have these conversations that are uniquely theirs.
That's what I think we're trying to define or see. It's like when will we see that? What's interesting though is this idea that well what gives this conversation a feeling or a sense of like this human touch and I would argue that what gives this conversation this kind of human touch to it is actually the fallibility of us as humans and AI is almost perfect like AI is almost perfect like if you watch it play chess or you watch it play (48:01) go it just smashes the world's best players absolutely destroys them but then as humans what do we We don't go and watch games of AI playing itself.
We go back to watching humans play themselves. If we had a whole football pitch of robots playing football at a far higher level than actual footballers, we'd still go back to watching people. (48:24) And I would argue that there's something inherently human about being human, which is our fallibility and the ability to make mistakes.
And that's what actually creates intrigue and interest as opposed to this perfectionism. And so kind of going back to your point which is like is AGI when we're able to have a conversation with it and have no idea that we're speaking to a human but then the the argument would be well I'm going to be able to tell that it's a AGI because it's infallible like I can't really catch it out.
You know what I mean? (48:49) Yeah. But maybe it's so smart that it would actually understand that and it would it would dumb itself down to make us feel like it's not superior in its intelligence. I don't know. But but you're exactly right. You're exactly right. I think and you see this with, you know, go to a party and all the 15-year-olds are hanging out with people that are around that age.
(49:11) The 9year-olds are hanging out with the 9-year-olds and the adults are hanging out with the adults. And it's the context of experience that we kind of relate to each other based on being of a similar age and experience set. like we've experienced the same amount of life and there's this context that is similar like we're on the same wavelength because of that age element.
(49:37) And you bring up an interesting point of like whether that will ever exist between, you know, let's say these things are put into humanoid bodies, their intelligence is partitioned off from the computer. From a design standpoint, you really kind of go after one of these things that could potentially, you know, have its own unique experiences.
(49:58) And you have to ask yourself whether you would really have any type of emotional connection or desire to sit down and have those types of conversations because they're just so freaking smart and they know so many different domains. Like would that be interesting? Are they fall? It's tough. It's tough.
I don't know. It's so tough. (50:16) And then the other thing is like there's like a human beingness obviously to being human, there's like a spiritualness to being human which is like if I have a whole bunch of friends over for dinner and I spend time putting energy into like going harvesting the vegetables from outside, bringing them inside, making this amazing dinner, having these amazing conversations with all my friends, there's like love and affection that's gone into this this creation.
(50:38) And there's something that you cannot take away that you can have a robot in the kitchen who's gone and made a Michelin star like [clears throat] meal. But I would even say that there's something about the humanness of the human putting that time and energy and that love. There's something that I don't think you can replicate the fallibility of the meal.
Right. (50:54) Yeah. Sorry. I put way too much salt on it. Especially when I'm in the kitchen. Oh, no. I love that point though. I really like that point that the human element is because of the vulnerability, the fallibility and it's real to us because we're on a similar wavelength actually like and I'm going to butcher this with Claude Shannon and information theory.
(51:22) He says information is when we have surprise. Yes. So it's kind of to that point like it's when we're interacting with a human. the engaging point of interacting with a human is a surprise that you don't really know what they're about to say. Whereas if you're an expert in a field and you're talking to AI, you kind of have an idea about what they're going to say.
(51:40) And so I wonder if that's a component to it. Yeah, I think so. In the middle section there, she talks about how like a lot of these things were trained and going to like these farms, these almost like click farms in developing nations where people are just looking at pictures of a bridge and then they have to tag this is a bridge, this is a person, this is a and the just total lack of funding that is put into this but the amount of horsepower and human work that you're getting out of it uh is just a giant currency arbitrage.
Two of us are (52:17) Bitcoiners, so we're looking at this and saying, "Yeah, Bitcoin will eventually solve that problem." But that was a major part of the book. It went on a little longer than what my interest was in the topic because I guess from my vantage point I'm looking at it and I'm saying it's super sad that this is how many people countless people around the world are treated.
(52:40) But at the same time I see a solution in sight in the next 10 to 20 years that's automatically going to solve for that. So I guess for me I'm not really as deep into that particular top and that might sound very insensitive to kind of frame it that way but as a person who's like grounded in engineering I'm looking at it okay here's a problem she's defining the problem she's doing a great job defining it but I'm also looking at there's already a solution in my humble opinion that's going to solve a lot of this in the future but Seb I'm curious kind of (53:06) your thoughts on that part she kind of compares these AI giants to colonial empires and there's kind of a quote that she says like they seize and extract precious resources the work of artists and writers, the data of countless individuals, the land, the energy, the water required to house the massive data centers.
(53:23) And then to your point, she kind of goes into, well, where are all of these people coming from at the base layer to support AI? And it really is and there's all of these low paid global workers that are tagging, cleaning, moderating all of this data for AI. And to start out, like we go and look through our Google photos and we just type in, I don't know, cat and it goes and finds all of the pictures of cats.
(53:47) Well, initially that was not done by AI. That was done by an individual going through all of our pictures and tagging what a cat looks like. And so I find this really fascinating that there absolutely is right now this extraction of resources. However, and I think when you go down the Bitcoin rabbit hole, it's always about, okay, is this a symptom or do we want to go down to the root cause?
And I would say the symptom of being able to find people that are willing to accept 70 cents an hour is the symptom of poor governance models and these communist (54:19) socialist practices where you've basically got massive extractivism. If we had more of a free market, I would argue that the AI couldn't go out there and find these individuals.
And so I'd say we can always talk about the symptoms, but how about we try and fix the root problem, which is the fact that we actually have absolute poverty globally when we don't necessarily need to. (54:37) Amen. Amen. Yeah. I think that's where I get frustrated with these types of really long sections in some of these books that are written by people that are that are trying to shine a light on something that they see as being very unjust.
But like you, I see it as a symptom and not the cause. And what I want to talk about is the root like as far upstream as we can possibly go. (55:03) What can we fix that then will eventually, you know, work that out? Because if you don't fix the fundamental thing that's causing it, we can sit around and talk about all these stories as much as we want, but it doesn't really solve anything.
But I think it was an interesting highlight. It's something that does need to be called out. (55:20) It is something that people need to understand when they're using this technology. And it's so, oh, I mean, if you're harnessing this, it's super abundant. It saves you so much time, there's an appreciation for like what went into it and where it came from.
And the book definitely did give me that. So, and it gets back to the point, and I don't want this to come across as I'm supporting it, but it gets back to that point where let's just say, you know, Google is absolutely like geared towards AGI and they're willing to go do (55:51) whatever it takes to go and create this AGI.
Or when you look at open AI and you're saying, "Look, if we want to focus on best practices for workers and pay minimum US dollar wages at $15 an hour, well, all of a sudden, you've completely kicked yourself out of that race." And so I think the way the world works, unfortunately, is that people will go to they lean towards the cheapest way to do something and so they end up going into these countries like Argentina and Venezuela and stuff.
(56:20) And so absolutely I think there are uh human rights issues and there are abuses of power but again to your point I think that it is a symptom of a bigger issue and we get stuck talking about the symptoms as opposed to the root cause. Yeah. I think there's one other point that I did want to bring up which I found was really fascinating is what does the world look like moving forward because you look at something like chat GBT and uh their GPT1 GBT2 GPT3 4 and such and GPD4 I did a little bit of digging like how much did it cost to (56:51) really train GPT4 and it cost between like4 to $80 million and then you look at GPT5 and it could be upwards of 1 billion but we don't know this number so you're asking like man there's these models that are being trained with hundreds of millions of dollars to be able to kind of create this incredible thing that we're using dayto-day.
(57:11) And then you go and see something like the Chinese AI company DeepSeek go and release their R1 model and they trained it for $294,000 on 512 Nvidia chips. And so you're like all of this VC capital has funneled into these AI companies and they're expecting a return and at the same time you're having this competition that is driving down the cost of training these AI models.
(57:35) Like I don't think they're ever going to get a return on these things because the cost it's wild competition. Yeah. And then the reverse engineering on what it is, like after they do train it, then all these other companies can go in and reverse engineer extract out the weights. Not perfectly, but pretty dang good.
(57:57) Like, I just don't see how the people putting up the funding on this are possibly going to get a return. It's pretty wild. And I think, you know, there's there's a lot of ego playing into this race as well. It's pretty insane. And I think that as we look at where it goes next, it really comes down to the alignment because the other part that I think is not being talked about is when you put in an inquiry, you put an input into one of these models and you get an answer back.
(58:26) If you can create a model that's very specific to that kind of question and you can return the answer very quickly, you can specialize in that domain and you're going to have a lot of utility and a lot of interest for being able to provide that service that's getting you a very quick a very accurate answer for a specific domain.
(58:46) And where I think a lot of it's going to go is models that are specialized that are almost extractive out of the base model that then are then specializing in something that gets the alignment of the person's initial question a whole lot faster. I saw a very quick video clip from one of the founders of Anthropic and this is something that he was talking about.
(59:06) He's like the race to build the biggest model is I'm paraphrasing this and this is not how he said it but it almost seems like it's a fool's eron and that the real value capture is being able to get a quick response a very accurate response to a very specific question and to do that I think that the alignment and basically uh fine-tuning things is going to be where the real value captures at if you can kind of figure out a way to do that especially from a competitive mode standpoint but boy I would Nowhere near from an investment (59:38) standpoint.
Good lord, I just don't even know where to begin. It's going back to Nvidia. You want to be on the chip side of things because the one thing you know is there's going to be more demand for chips. Like more than anything, there's going to be demand for chips. Whereas these AI companies are just going to eat one another.
(59:55) They're freaking going to eat one another. And actually, to be honest, the one thing that stood out just then, as you mentioned, was Anthropic. It talks about it in the book. There's the brother and sister that worked for OpenAI and left OpenAI because they didn't believe in the trajectory it was going down and they felt that the safety was not in place and so they started anthropic which you could argue like as I mentioned previously there are these studies coming out that are showing that open AI is useless when you try and shut down the model midway through a task it (1:00:25) doesn't want to be shut down but anthropic immediately shuts down and so you wonder their the safety protocols to ensure that the end product is secure.
Yeah. All right. Real fast, Seb. Our next book is called Lifespan by David Sinclair. So, I have wanted to cover longevity and some of this stuff for a very long time. (1:00:49) I'm a big fan of this space and just kind of learning everything that's happening in this space. And uh you know a lot of Bitcoiners love longevity cuz they want to they want to figure out how they can live a little longer and enjoy life and we're going to cover this from time to time on the show is what in the world's happening in the longevity space.
(1:01:08) So this book uh David Sinclair I would argue is one of the pioneers in this whole field of longevity. His book is fantastic. Uh Seb's going to go through it. I'm going to reread this book. Um, I read it a couple years back, but I think it's a really, really strong book for a foundation for people to kind of understand where a lot of the research for longevity comes from and where it might be going in the future.
(1:01:34) So, if you're reading along with us, that's where we're going next. We would love to have you guys as a co-reader. So, that's the book. Seb, any comments on the next one or Oh, man. I'm excited. And to be honest, one of the things I'm most excited about is hearing your thoughts on longevity because I feel as if there's kind of two camps to the longevity movement.
(1:01:55) There's kind of this camp which is just like, well, we're humans and if we want to evolve, we want to minimize lifespan because then it allows us to iterate, iterate, iterate. Then there's this other camp which is like, let's just expand lifespan indefinitely. Let's live for 500 years, but then do we become immovable?
Do we become basically prone to some big change that wipes out humanity? So, I'm curious to hear your take because I think there's a few different camps in the longevity space. (1:02:18) You're a specist, aren't you, Seb? Ah, this is going to be good. All right, so folks, this is uh all we have for you.
The book that we covered this week was Empire of AI Dreams and Nightmares and Sam Alman's Open AI. Ah, we liked it. It was okay. Next book is going to be Lifespan by David Sinclair. (1:02:41) And uh thank you so much for joining us, Seb. give people a quick handoff to all the stuff that you have going on in the book that you also have.
For sure. Yeah. If people want to kind of follow along, they can uh find me on Twitter or X at Sebunny and Bunny is bun ny. I have a blog, the chief of self-s sovereignty at sebunny.com. (1:02:59) And then I also have the book, The Hidden Cost of Money, and it kind of yeah, talks about money, but at the moment, it's nice to be kind of discussing things other than money.
All right, we'll have links to all of that in the show notes. Seb, thanks for joining me. and uh everybody else out there listening, keep reading and uh we look forward to you joining us next week. (1:03:17) His focus is not necessarily on perfection. His focus is on just iterate, iterate, iterate, execute, execute, execute.
And so rather than its competitors that were trying to create this super clean, professional looking code, Nvidia was kind of overtaking them because it was not about kind of clean professional code. (1:03:37) It was just like, let's try and minimize our execution times in getting this to market. They changed it from one year, two year cycles to six month cycles.
And they were just trying to get it over out into the market. and they'd really dominated the market as a result of that strategy.
Comments 0