Chet Kapoor: GenAI, My Agent, Human-like, Reinvention | Work 20XX Ep32

English Transcript
Chet Kapoor: GenAI, My Agent, Human-like, Reinvention | Work 20XX podcast with Jeff Frick Ep32

--

Cold Open
All right. So if you're ready to go, Chet.
Absolutely am.
All right, so I will, I'll count us down, and we will go in three, two, one.

Jeff Frick:
Hey, welcome back everybody. Jeff Frick here. Coming to you for another episode of ‘Work 20XX.’ And I'm excited to have this next guest. He's an industry veteran. I think he's been around this business and this valley as long as I have. And, we're kind of a rare breed to find these days, so I'm excited to welcome in. He's Chet Kapoor, the Chairman and CEO of DataStax. Chet, great to see you today.

Chet Kapoor:
Thank you Jeff, it's great to be here.

Jeff Frick:
Absolutely. So before we jump in, Chairman and CEO of DataStax, give everyone kind of the 101 on what DataStax is all about.

Chet Kapoor:
DataStax started about 14 years ago with data at scale. Right, if you think about all the different apps you use—Spotify, Apple, Netflix, ordering Starbucks, ordering a phone from Verizon, getting a FedEx package—they all use technology that DataStax helped create. So anything that was data at scale, that's what they did. Now think about two years ago, in comes this thing called ChatGPT. And guess what happens? There is no AI without data. There's no AI without data at scale. We do data at scale. So DataStax is the company to do GenAI apps because we can. We make it easier for people to develop GenAI apps. We make them more relevant, and we obviously scale them like crazy. So that's what DataStax does.

Jeff Frick:
All right. Great. And we're going to get deep into the whole AI conversation. But before we go there, I want to get a little bit technical. Tell me a little bit about what is a vector database. How is that different? You know, DataStax was famous for Cassandra back in the day. You know, Oracle was the first one with relational databases back in the day. What is different about a vector database, and how is that really an enabling technology that we couldn't do before?

Chet Kapoor:
So the thing that GenAI needs. So if you think about LLMs, right, large language models [LLMs], they need to know the proximity of information. So Jeff's been in the Bay area for 30 years. Chet's been in the Bay area for 30 years. They need the proximity of both of us being in the Bay area, not just Jeff's record and Chet's record. And it may not be zip codes, but so what you do is you encode my information into an x, y axis, and we take your information into a number, x, y axis, and actually put them together. So when we search for the area, we get proximity of Jeff and Chet being in the Bay area together. So what vectors are is a great way for things to relate to each other. There are numerical representations of what is available to large language models so that they can actually go and use that information to make it happen. So language models actually need vectors to go off and make it happen. They use it themselves. But that's also how you interact with them to create more relevant applications. So if you think about all of the data you have, let's just take FedEx. All my packages being tracked forever. Now it is there. But I need to ‘vectorize’ them so that I can actually work with the LLMs to create more relevant applications going forward.

Jeff Frick:
And is it just a faster way to draw the relationships or a faster way to connect the relationship? Is that the secret?

Chet Kapoor:
No, it’s a different way. So it is a different way that makes it faster for large language models to use the information.

Jeff Frick:
Okay.

Chet Kapoor:
So then let's talk about your pivot to GenAI and supporting GenAI. What did you see specifically, and what was really involved in this pivot and kind of reprioritizing around this hot new thing that hit the scene a couple Novembers ago?

Chet Kapoor:
So, we started tracking ChatGPT very closely and, you know, a bunch of us from Google. So we knew about foundational models happening. So we've been tracking it for a while, and then we really saw ChatGPT come in in November of ‘22, and we started talking about it at the all hands. We started a bunch of us started talking about it at the dinner table. It became a part of how it was going to change the world. And so we're geeks, we are technologists. So we wanted to start talking about it, and it was very clear by about January or February [2023] that we had to do something about taking all the data that we store. Right, all these terabytes, petabytes of data that we have, how do we vectorize that so that you can make more relevant GenAI apps happen? The first thing we did was we said, we went and created some open source technology called ‘JVector,’ which basically takes information as it is coming in, vectorizes it, indexes it, and stores it so that you can use it right away. So that's the first thing that we did. We basically created the first what people would call a hybrid vector database. That means it actually serves as a regular database, but it's also a vector database all in one. And it is the most scalable one in the industry. And we did that in, I would say, June of ‘23. Now the moment we did that, we realized that that was not going to be good enough. If you think about every stack, right. This is my fifth wave: client server, web, mobile, cloud, and now GenAI. Every time there's a new wave, there's a new stack that developers use to build applications. Databases are part of them. So we started with a vector database and we delivered that to the market. But it was very clear that it was the wild, wild West. And people had not figured out how to build apps to go and make this happen. And so we said, we're going to go up the stack and not just have a greatly optimized vector database, but actually go up the stack and give developers an opinionated view from open source technologies on how they should build apps. And we've been on that journey. We started about in August of ‘23, and now one year you fast forward. It is amazing. We are delivering an AI PaaS, which is a platform as a service. We have a visual editor. We have a bunch of ways to ingest data. We're using a bunch of technologies from Nvidia for embedding services. So we're making it like at least 10x faster and more agile for developers to deliver relevant applications at scale.

Jeff Frick:
So you've been on a little bit of a roadshow on your RAG Roadshow.

Chet Kapoor:
Yes.

Jeff Frick:
Which I was happy to attend, the one up in San Francisco. I think you were most recently in New York. So, you know, when ChatGPT launched and then I think the first OpenAI developer conference, they talked about, you know, have your own ChatGPT, you know, put in your own data, customize it your own way. Well, you know, that's not really training it. RAG [Retrieval-Augmented Generation] is a real way to train it. And, you know, kind of put in your own data, make it relevant. So tell us a little bit about the RAG adventure and it’s still, it’s one step closer to this vision of citizen developers. You know, it's not quite there yet. I watched, I'm like, I'm not, don't think I'm quite ready to write that demo, Netflix app that you had, but it's getting closer every day.

Chet Kapoor:
This is, it's a great question. So the first thing is, LLMs do really well, and they'll always have a great spot for us. And you can talk about how OpenAI was going after the search market and going after advertisers. All that stuff is for the consumer market. Now, if you think about you and I, and we think about this, right. [Chet holds up his mobile phone] You use an AI agent today. What's the AI agent called? It's called Google Maps, right? It's not like we’re new to this. We've been using AI agents for quite some time. How does an AI agent perform? How do you agent-ify something? You take all this information you have about maps. I'm going to give you an example. All of this information about maps and things like that. And then you need to take all the information about me, which is what is Chet like? What kind of food does he want to do when he travels? Does he actually have a, in the Bay Area, does he have a sticker? Can he use the carpool lane? Things like that. Right. All that is information that's stored with me. You are not going to give that to the LLM because that's personal information—it’s PII information. [Personally Identifiable Information]. So now when I’m going to create an app like Google Maps, I’m going to take all of this information about maps and freeways and roads and all that stuff. And then I'm going to take all this information I have available for Chet. And I need to combine these two to make it relevant so I can serve a relevant experience to Chet. Right, and the mechanism to do that is ‘retrieval augmented generation’ [RAG]. That's what RAG is. RAG is taking everything that the LLM gives you, everything that Chet has about him, combining it so that we can deliver the most relevant experience for Chet on a personal basis. To make that happen. Whether you're doing it through a chatbot, through personalization, or whatever else it might be. RAG is the best technique so far. There'll be other techniques that come up as well, but that's the best technique. And we've jumped on it, and so has the industry.

Jeff Frick:
Right.

Chet Kapoor:
And the open source ecosystem is phenomenal. It's growing really, really well, which is great.

Jeff Frick:
Right.

Chet Kapoor:
We have a saying at DataStax, if it's not being done in open source, it's probably not worth doing.

Jeff Frick:
Right. right.

Chet Kapoor:

That's how fast things are going. So we have this thing about RAG, and it is going exceedingly well now. And taking off. And so we had this conference that you were in San Francisco.

Jeff Frick:
That was great.

Chet Kapoor:
You should have been in New York. That was just two weeks ago. We had standing room only, over 500 people there, and it was really interesting. Generally, a company like us, you know, goes and says you want a bunch of folks to come over. Ninety percent of the people in the room were not our customers. We had Nvidia on the stage, it was just a great time.

Jeff Frick:
It was standing room only in San Francisco. I was in the back helping the catering people because they were tripping over me every time they came in and out from the back. So yeah, you had a full house. No doubt about it.

Chet Kapoor:
And we’re doing one in London next week.

Jeff Frick:
Yeah. So, you know, you talk a lot about reminding everybody that we're still in the early days. And I think one of your fun examples, you talk about early mobile. Everyone wanted to program Angry Birds. I think I pulled up an original or early, early Yahoo web page the other day to highlight to people. We're just getting barely started in this thing. And if you look at where web apps are now, you know, compared to Angry Birds and everything is on our phone, this thing is hardly getting started. So let's talk about adoption and people being afraid of it and how people should learn to use it, especially in the context of an assistant. Because that's the one we talk about every day.

Chet Kapoor:
So we think that 2024 is the year of production AI. A lot of people found when we said this in February, people were like, ‘No!’ It was very controversial. Now you show up in the middle of September and you realize that every enterprise that we know of, right, 800 plus customers doing lots of different things, everyone will put one app in production this year. It may be a small one, it may be just internal, but they'll put something. So I think 2024 is going to be the year of production. 2025 is when it gets interesting. That's when we have what we call ‘transformative’ AI use cases. Where people start saying, Ah, I think I really like this travel agent or the assistant I have, and a bot and things like that. But now I'm starting to change my business model and try to experiment with different things. So I believe 2024 is the year where people put static websites on mobile apps, right? They basically had stuff already on the web and all they did was make it available in mobile. And that's where I call the ‘Angry Bird’ stage, right? You're just doing some things that are simple, but they're unique because the form factor changes and things like that. But next year and the year after, it gets really, really, really awesomely interesting because we have not seen the Amazons of GenAI yet. We've not seen the Netflixes of GenAI yet. They’re still yet to come. We've not seen companies like Walmart who have reinvented themselves around GenAI yet. I think all of that happens in '25, '26, '27. But the one thing that's different this time around, it'll go faster. It'll go much faster. And the reason is because this technology is more human-like than anything that we've ever seen in the last 500 years. Or forever in the history of humankind. We have never seen technology that is more. I mean, you think about the steam engine, right? It's not human-like, but it changed our lives forever. But this thing is, like, more human-like than anything else.

Jeff Frick:
I wish we had six hours, but we don't. But we'll keep going, so on the app side, what percentage of the apps do you think are going to be standalone apps that somebody engages with versus AI influences within other apps? Because it seems like that's really where the giant impact is, because, as you said, it's going to be a part of the way everything works in software.

Chet Kapoor:
I think yes and yes. I think the answer, so let's take the first thing. Every app will have GenAI features, no doubt about it, but I think people will rewrite those apps to be Generative AI apps. And by the way, this is how technology works, right? If you think about cloud, let's take a different one than mobile. What did customers do in the beginning of cloud? Lift and shift. It's the same app. Shut down my data center, let me go and just run it on AWS. That's what the use case was. And then they realized, Ahhhh, we got to, like, rebuild the application to take advantage of everything that the cloud does. And so that'll be the natural progression. People will just say, yeah, I have a website. Priceline is a good example. I have a website. I'm going to go and put Penny [Penny is Priceline’s AI chatbot]. Instead of just having a regular chatbot, I'll have a Generative AI chatbot, so Chet can get relevant information about his travel plans. Right. That's all good. But how do we change the experience of how Chet does—he’s still coming to a webpage. And so people like Priceline are going to start figuring out what is the next—what does GenAI give me where I can change the experience for Chet? Right, and go proactive in making it happen.

Jeff Frick:
Right. On the human-centric side, which is such a good point. And I actually think the most underreported aspect of this whole thing is the fact that, as you said, I now have access to a supercomputer, first of all, and I can talk to it. And the fact that it's trained on text and literature and all these people generated content databases actually makes it a really good conversational assistant and really understands the way people talk and the way people can communicate. And I think that the fact that you've got that at your fingertips all the time, you can ask it any question, and it comes right back and helps you sort things out. I think that is way underappreciated because we finally have a conversational interface with a computer.

Chet Kapoor:
I would agree 100%. I've been wanting. So if you really think about it, UI [User Interface] has not changed since what PARC did [Xerox PARC] and what Steve Jobs took from PARC and said, we're going to go and do it on the Mac, right? The GUI interface. [Graphical User Interface] And now instead of a mouse, you use your fingers on a phone and things like that, which is great. Lots of good progress. You know, you can see everything going in the right direction. But it's not significantly changed. It's like, you know, saying the steering wheel has been on the right or the left and generally in the same place since cars came to be, since cars were invented, there’s no significant change there. I think conversational interfaces will change the game significantly in a massive way. I think that is awesome. LLMs can do that. You can do that Chat. By the way, ChatGPT wouldn't have gotten to a billion people without a conversational piece to it.

Jeff Frick:
Right

Chet Kapoor:
It went faster than any other technology in our past. What I'm really looking forward to, though, is I want the technology to think for me. Right? I want it not just to think for everybody. I don't want it to homogenize it. I want it to actually be specific to me. So I love the interface piece that you mentioned, but I want it to be next level because I process my email differently than you do. And by the way, my leadership team processes emails differently than I do. And I want the agent, the email agent, to be my email agent. I want it to do it the way I think, and I want it to be specific to me. That's what I'm looking forward to. I want to train agents for me.

Jeff Frick:
Right. All right, so let's shift gears a little bit and talk about work because this is Work 20XX. In the context of work, people have a hard time adopting it. It's still surprising how many people have not tried it out. And I try to tell people, you know, think of it as a calculator. It's a tool that you need to use. You need to get comfortable using it. The difficulty is, at this stage in time with hallucinations, I don't double-check the work of the calculator when I do a square root. And what I find really interesting about hallucinations is if it's a topic that you're aware of and you're knowledgeable about, you can pick out the hallucinations. And more importantly, you can identify whether it's important or not. But the problem is, if you do like ten queries in a row about something you know about, and then you do something that you don't know about, you know, it's a lot harder to pick up the hallucinations. How do you think that people should think about incorporating this tool into their life? Because today, what I would love for it to do that it doesn’t do is tell me what I should be working on right now. You know, what's the highest leverage activity that I should be doing today based on my calendar, based on my email, based on all these other things? And it's still not doing that. It’s still coming back from queries, but it's not really taking the lead and helping me be more efficient with my time.

Chet Kapoor:
This is a year and a half old.

Jeff Frick:
Right, right.  Angry Birds

Chet Kapoor:
Give it a little time. And for a lot of people, don't forget that in the early days of Google or Yahoo or everything you had, search queries were not accurate. In fact, they were not very good. You could actually go and say, I did this and you got a different result. This is just technology. It’ll mature, right, in all different ways. It'll mature with relevancy. It'll mature with governance, and it’ll mature with security. It'll happen across the amount of capital being invested is obscene. It is absolutely going to get better and will get better faster than most people think.

Jeff Frick:
Right, right. So you've talked about this thing, the productivity paradox. And as we're recording this, you know, Cisco just announced another round of layoffs. They had a couple this year. You know, Amazon just announced their five days a week back in the office, return to office, which I thought was kind of interesting. Some people think that might be just kind of a soft layoff that nobody's talking about. But there is this idea where, you know, can you substitute the increases in productivity based on this tool with fewer people, or should you really be thinking about it as a way to get more out of the resources and the people that you have? Because this is an increasingly accelerating and competitive world, it seems like it really should be the latter.

Chet Kapoor:
So, great question. Something that we think a lot about. Right. So, we think every enterprise will fall in one of three categories: delegate, accelerate, and reinvent—or invent. The people in the delegate category are looking for efficiency. And what they're going to say is I want to be 30% more efficient. And I'm picking 30%; it could be 20%, could be 40%, but 30% more efficient. And they will do layoffs just the way it goes. Accelerate will be people who say, I want to be 30% more effective. Yes, I'll take the efficiency gains, but I want to be 30% more effective. And those people are going to go and change the game, and they're going to go off and say, I'm going to accelerate. I'm not just going to delegate down and say, give me bottom line results, but I want to actually start thinking about being more effective. And so I can actually increase my top line and go from there. The really fun stuff happens when people say they want to reinvent themselves, and they want to be 300% better, not 30%, 300% better. When they take the model, they don't say, I am better with GenAI. They flip the model and say, I have GenAI, let me see how I would structure a company. And that's what Amazon did, right? Amazon reinvented what Walmart was doing because they said it's the web. Right, so we should think about everything differently because we have no legacy. We have what I call a blank sheet of paper. And you can do whatever you want with it.

Jeff Frick:
Right.

Chet Kapoor:
Right, so those are the three categories I think you will find companies actually go through those phases. There'll be companies that go through delegate and go to accelerate. They're not going to stay stagnant. There'll be some companies that just stay in delegate because they don't have, because they are a regulated industry and they do not have the imagination, or they don't have the income statement to make that happen. They don't have a balance sheet to make that happen. So you’ll find different people. But what's really encouraging is people like JP Morgan, the largest bank in the world, have come back and said, we want to talk about not just efficiency but effectiveness.

Jeff Frick:
Right.

Chet Kapoor:
Right, and that is extremely encouraging. Now, does that mean they won't do layoffs? No. It means that they will probably do layoffs as well. But the good news is they are looking at this as top line growth, not just bottom line. So, this point that a lot of people don't realize is that. You may not—let’s try to simplify this. I have 100 people in the company. I get to be 30% more effective. I have a choice. I can take the 100 people and make it 70. Or I can leave the 100 people and just become 25% more effective on the top line. And that is the decision that CEOs and boards have to make. I think most of them will realize that they're going to come into the accelerate category, then stay in that delegate category.

Jeff Frick:
Right.

Chet Kapoor:
So they will not—I don’t think over time people will realize they're not going to do layoffs. Now the models might change. They might do some. But I think they'll realize that this is a mode for them to become not just efficient, but also become more effective.

Jeff Frick:
Yeah, it's really interesting, right, because that so parallels the cloud story exactly, where people first thought about it as efficiency gained and then the smart people looked at it as an effectiveness game to change the model. I just had that conversation with Julie Whalen from CBRE, even trying to get real estate people to start thinking about from efficiency, how many butts per floor, how many cubes per floor, etc., etc. into effectiveness. You know, how are we helping people to deliver better results for the company? And it's such a different way to really think about the world.

Chet Kapoor:
I’ll make it really simple. We run the company on a weekly basis. We have a 40-page thing that we do that gives us stats and data and knobs or metrics across the company. What we do is we take everything that comes up for a week. We actually go and, you know, send it to an LLM that we’ve trained, and it gives us stats all over the place, and it gives us, of course, trending patterns and something that now, that work was being done by somebody for about 10 or 12 hours a week. Now, we actually just wait for five minutes and get a result back. What do we do with that person's 12 hours? We make them do other things.

Jeff Frick:
Right, right.

Chet Kapoor:
And that's how, if you simplify it to a task, it makes a difference.

Jeff Frick:
Right. All right, well, I'm going to take you back in the hot tub time machine here. We're going to go back to your early days. And you actually read books back when people still read books.

Chet Kapoor:
Still do, still do.

Jeff Frick:
And you’ve talked about the two big books that really had a big influence on you. Mike Moritz’s *The Little Kingdom*, talking about Apple and the excitement around the development of Apple, and the other one that I was amazed you brought up. I actually read it, *The Cathedral and the Bazaar* [by Eric Raymond], which is really the story of open source. And I think the open source, it's kind of like Moore’s Law. There's the actual thing, and then there's the concept and really the spirit behind it that I think is so powerful, like in Moore's Law. Open source, I think, is one of the most amazing contributions to the world in terms of this ethos. And, you know, I had Martina Lauchengco on, she's at Costanoa, and she talked about when she was at Microsoft back in the day working on Word. She goes, you know, you were stuck with the product you shipped in the shrink wrap for three years. So you better be happy with the features because you can't touch it. I mean, open source changed everything so much. You just talked about the cadence at which you run your company, and you also talked about having lots of eyes and using the accelerated development pace that open source provides. I wonder if you can share a little bit more. You've been in the business for a long time. You've seen kind of both sides of the equation. What is really the power of open source that has really transformed things?

Chet Kapoor:
I'll start by saying, I think I said this earlier. In 2024, if it's not being done in open source, it's probably not worth doing. That's how—and as a startup person, that's how I think about it, right. Now, you may choose to have a different monetization model. You may do 100 lines of code, and you may only want to open source 80 lines of code. We are not believers in that. But there are a lot of people who do that, and they have premium, freemium models and things like that. But if it's not being done in open source, it's probably not worth doing. Open source is single-handedly the best way to get innovation in the early days of a project. It's the best way to get diversity. It's the best way to get transparency. There's no hiding behind anything. Everybody sees everything. It's the best way to have meritocracy because it doesn't matter what school you went to or what company you work for. Your code does the talking. Imagine those four things in companies and how our social structures make it so hard to make that happen? We have this beautiful thing called OSS, or open source software, that lets you go off and make it happen. I am a firm believer, right, in that you have to figure out a way to make it happen. Now, the one thing I'll tell you, and the reason I said early days of a project, open source is very different when you are starting out versus when you mature. Linux was very different when we started out, or even Apache Cassandra, the project that we’re involved in, very different in the first seven years with Linux, the first 15 years, and very different now. Because now you make a small change that is wrong and you have a CrowdStrike thing happening because everything runs on it. So you've got to be really careful. In the early days, you're putting features in on a regular basis, so projects change and they morph. The reason I bring this up is because this is where *The Cathedral and the Bazaar* comes in, right, which is the four things I talked about are very hard to implement in the cathedral. And so for the early days, you just go and do it in the bazaar. It's the way to do it because you get diversity, transparency, innovation, just great. And meritocracy. But as time goes on and things get more mature, people are not going to start working on really cool features. The vector, our JVector product, was not something we did in open source. We actually did it in the cathedral. Just for the record, we did it ourselves, but the moment we finished it, we dropped it into the bazaar and said, please, people who want to use it can go off and do it. But people were not getting up in the morning thinking that they wanted ‘vectorization’ of a Cassandra database. Right. But OSS just, it just rocks in so many different ways. Right, and I think anybody that doesn't implement it, they should do it. They should, they just have a blind spot.

Jeff Frick:
Right. So different topic. You might not smile quite so big, but hopefully you'll smile a little bit. And that's governance and law of unintended consequences. And there's a couple of things I want to throw out before you and have you react. One is, Dr. Rumman Chowdhury, used to be the head of ethics at Accenture. Her great line is, ‘Good brakes allow us to go faster.’ Which when I heard that, I was like, wow, that is just genius. And, then you look at what's happening, say, in privacy as a proxy for our ability to regulate. And because of our system with states and the feds, you know, it's really hard to get a consistent rule, even for something like breach notification. And you compare that with, say, GDPR in Europe as a way to pull this about. And then the last thing is, you know, the law of unintended consequences, which we've seen a little bit on the social media side where, you know, unfortunately, to get more clicks, it turns out more extreme content gets more clicks. And you've got kind of these things that weren't necessarily set out that way. But it happens. As we look at the power of this technology and where it's going to be. And, you know, there's always the pushback. We don't want to govern too much. We don't want to kill innovation. But, you know, unfortunately, things left unfettered don't necessarily always turn out the way we wish. What's kind of your take as we're getting enough traction where this is becoming more and more of a topic of conversation. In fact, I think, [California Governor Gavin] Newsom just signed some paper sitting next to [Salesforce CEO Marc] Benioff up at Salesforce the other day.

Chet Kapoor:
Yeah. So I'll start with, first my biases. I'm a geek. I like innovating stuff. I like building great products that change people's lives. That's the essence of what I like doing, right? And obviously building businesses around it. But it starts from building a product that changes someone's lives. And I don't want anybody coming between me and the product and the person who uses it. Nothing. Because that's what, that's how innovation happens. It's really hard, right? It's not meant to be. Right, and changing user behavior is really, really hard. So what I want to do is I want to build a product and I want the product to interact with the user, and that's the only thing I care about. That's my bias. Having said that, for all technology waves, until this one, I would have said, "Ahh, don't worry about governance and regulators and all that." But this one, because of what we talked about earlier, this being more human-like than anything else, I think I'm really glad that we actually are getting people involved. Now, the problem is when you get this, when, you know, minds are mature, I build a product, and markets and governments are not, because they have many minds involved. That's just the nature of how things happen. And so what is going to happen here is we're going to—we're going to do it wrong. There's not a single technology wave, there's not a single technology wave in our entire history that has not had—that has not had, you know, it has had bad consequences. Every technology, unintended consequences happened with every technology. There’s a flip side, it's two sides to a coin. It will happen with GenAI. We're not, you can put all the regulation you want and it'll still happen with GenAI. Right, it doesn't matter. So I think we should—the regulators being involved now is great. We are not going to get it right. But the fact that we're working on it, my just request would be the people who work on this should be people who understand the technology a little bit.

Jeff Frick:
Right

Chet Kapoor:
They should not be. That is my one thing that I want. I do not, I want any policymakers involved in this to take the time to understand the technology and the basics of it. They don't have to become programmers or things like that, but they should actually be well-versed rather than just say it's wrong without understanding it. Because the technology is moving very fast. And please don't forget we're in the Angry Birds part.

Jeff Frick:
Right, right.

Chet Kapoor:
That's right.

Jeff Frick:
Yeah, hopefully it's not the senators that still have their emails printed for them.  And handed to them on a piece of paper.

Chet Kapoor:
That's my point. That is my point.

Jeff Frick:
Yeah. Yeah. I just was fortunate Ray Kurzweil just came out with his new book, *The Singularity Is Nearer,* and he was speaking at one of the local bookstores. Was very cool to see, and there's a lot of really interesting concepts. I also interviewed Jack Nilles, the grandfather of telecommuting, who did the first research in 1973, when the fastest bandwidth you could get was a T1, which I think was 1.4 megs on a good day, according to my friends in the business. So exponential curves, I want to talk a little about exponential curves. Just to take you back. I had to look it up. The Cray supercomputer in 1976 cost equivalent dollars today of $36 million. It had eight megs of RAM, not gigs. It could do 80 million floating point operations compared to a modern smartphone that can do 500 billion. So you can't even try to figure out orders of magnitude. I tried to figure out for the bandwidth. How do you compare 5G to 1.4 megs? Struggled, ChatGPT and it and said, okay, how about compared to a garden hose? If a garden hose is 1.4 megs, how much water do you need to have 5G? And it was like 50 fire hoses, you know, bigger, faster, stronger. So when you think of what's possible in the not-too-distant future, you said this is going to go faster than anything before. You know, where do you see things like small language models? Where do you see things like local language models running, running on your laptop or even more on your phone? As you know, these exponential curves on the technology side continue to ramp. And, you know, how do you run a business in the world of exponential curves as these things get faster and faster and faster?

Chet Kapoor:
It's a great question. Something I think about quite a bit, right? So, I'll give you two perspectives. I think I'll give you a consumer perspective, and I'll give you an enterprise perspective. From a consumer perspective, small language models, the Distil model, it's done, it's going to happen. Right, I want my language model. I want my small language model that is disconnected from anything that's happening with a large language model. This is mine. And by the way, then I would want one for DataStax. And I would want one for DataStax and marketing, and on and on and on. So that part is done. I think it is going to happen because something I had, just to be clear, I don't want, like we talked about, I process email differently than somebody else does in our same company. And I want to make sure that they have their own small language model, because it might start with email, but it could go to calendaring as an example. It could be how do you create content? How do you process content? What do I look for in content, versus somebody else, so that ship has sailed. People will figure out a way to do it. And if I want it, I want it available on this. [Chet pointing to his wristwatch] I want it available on this, by the way, not my phone. That's actually an easy one. This, I want it available everywhere.

Jeff Frick:
Right.

Chet Kapoor:
Right. I want it able in my glasses I wear. Right, so small language models are going to show up everywhere no matter where you go. It'll show up in everything we do. And that is awesome because it is all about me. And I don't mean that in an egocentric way, but it's about how does this change my behavior, then changes a functional behavior, then changes a company behavior, then an industry behavior, then a country's behavior, and then civilization's behavior, right. But I think it needs to zoom in and out through all of that. So we're going to find some crazy awesome stuff happening that along this way you have ‘agentification’ making it happen. So I think that's one part. On the enterprise piece, a little harder. 90% of the IT budget in enterprises is about running systems, 10% is on innovation. This AI stuff ain't cheap. And so it's going to create a crunch on their IT budgets. And you're not going to go from a $5 billion budget to a $10 billion budget. You may increase it to $5.5, $5.7. And that's not going to be good enough for you to reinvent yourself. But if you're in the delegate phase, fine, but you won't be able to reinvent. So it's going to change how enterprises spend time on technology. And the best example is the same thing happened with web, same thing happened with cloud, right? But they were replacements. There's no replacement here. You're actually going to gut out your system and make it happen. And so the enterprise IT function is going to go through a bunch of changes in the next 4 or 5 years because it is going to create this structural problem where they have to continue to run what they already have. But while they're running it, they need to completely reinvent themselves in an app to go off and make it happen. And that is going to be a problem. It's going to create tension on how much money you spend on services, how much money you spend on software, how much you spend on LLMs, on small language models, on databases, all those kinds of things. And we think it's long overdue. That structure has not changed, at least for the 34 years I've been doing this.

Jeff Frick:
Yeah, well, and it's not going to be expensive for long, right? It's not going to be expensive for long. That same example, the T1, I think cost $8,000 a month in today's dollars.

Chet Kapoor:
Correct.

Jeff Frick:
For a T1 compared to what you pay for your phone.

Chet Kapoor:
Very much so.

Jeff Frick:
Kind of piggybacking on that. You know, a couple of years ago I did some stuff with GE and there was a lot of talk about digital twins, you know, digital twins for modeling behavior. So you could do different types of tests And you could do lots of stuff without actually, say, putting a jet engine through a sandstorm to find out how it's going to act. So a lot of talk about digital twins and a lot of good things about it. What's really crazy when you start talking about what you're doing now and RAG and training, you just talked about the way you do email versus the way I do email. When you think about digital twins and digital twins not for jet engines, but digital twin of Chet. And like you said, it might be a couple of digital twins, the one that goes to work and then the one that stays at home and plays with the family.

Chet Kapoor:
And does laundry.

Jeff Frick:
How do you think about that? That's a whole different one. That could be your next billion-dollar idea right there. The digital twin does laundry. But as you think about digital twins and how that's going to change the way, you know, there’s talk of, you know, my twin’s going to go attend meetings for me. You know, there's so many different ways that you can, it’s kind of mind-bending to think of what you would do with a digital twin, scenario planning for health. I mean, there's so many vectors in which this is applicable, pretty interesting possibilities.

Chet Kapoor:
I'll be controversial. I don't think, I think there'll be variations of what people call the digital twin. My perspective on this. And we talk about job loss and things like that. This is not about me versus AI. This is not humans versus AI. This is about humans versus humans with the AI. And what I mean by humans with AI or people with the AI is I'm going to have a bunch of things helping me, but I am still the center of everything. The agents or the digital twins will never be able to do what I do, because judgment will still be mine for quite a long time to come. Quite a long time to come, right? I mean, otherwise, the agent has to learn my childhood issues.

Jeff Frick:
Right.

Chet Kapoor:
And that is going to be hard. So my point is, I behave a certain way because I am thinking about what I want to be, what I used to be, and what I am. And that's kind of hard for an agent to pick up, no matter how much they go and search the web and say, you know, Chet was born and brought up in Calcutta, and he came here because of the Steve Jobs book or whatever else it might be. The nuances are only known to me. And so my take is, I think digital twins, there'll be variations of it, just like every other technology. We reach, we reach, we reach, and we land up—you know, we reach for the moon and we land on the trees. Right, I mean, we've been talking about self-driving cars for a long time. They were going to be ready four, five years ago—they’re now coming into being right.

Jeff Frick:
Right.

Chet Kapoor:
They’re still not ready to roll them out. So there's—I think there's a part of this that we just have to realize. But I am really looking forward to a bunch of different agents that are very specialized for me. Like you said, for laundry, for hanging out with the family, helps me out. For example, I would love to know what my daughter's schedule is today. And instead of going and looking at it, I would say, you know what? Can I spend time with her between 4 and 6? Because I think I'm going to go home early and find out. And so clicking 16 places, looking at her calendar, looking at my calendar. It would be an easy thing for somebody to get done.

Jeff Frick:
Right, right.

Chet Kapoor:
And then say in aggregate, did I spend 14 hours with her or 10 hours with her, or 6 hours with her this week? How do we make sure we spend ten hours with her next week? And I say we, meaning me and my agent, we.

Jeff Frick:
Right, right. Well, and the other thing too, just what is time? And you talked about self-driving cars have been on the horizon for a while. They're just getting here. But was that a long time or a short time? You know, in many ways, it feels very, very short, which is what we're kind of getting to the end of our time. So I want to give you the last word. You've been through a lot, as you said, five different waves. You know, back in the day when people's main function was really being an API between applications, we took data from one piece of paper and keyed it into another piece of paper. What are some of the things you're excited about? I mean, I know you're excited about this next year is going to be the year of production GenAI. What are some of the other things, maybe more specifically, or some examples that you've seen or that you guys are involved with, whether it's in health care or transportation or whatever, that you can say, wow, this is really—this is really amazing. This is some of the reason I came here. This is some of the magic I read in the book about early Apple days. It's now coming to fruition in some of these other fields.

Chet Kapoor:
I love—I am really looking forward to GenAI making individuals more productive, because if you can start there. And by the way, the individuals could be consumers, right? If I was in the retail world or if I wanted to go off and be Verizon or Best Buy or Priceline or whatever else it might be. I'm looking forward to GenAI with LLMs and with SLMs—small language models with RAG—making it so much more productive for individuals to do things right. It could be my email, it could be me going on travel. It could be me buying something on BestBuy. It could be any of those, or me talking about doing a bill problem, right. How frustrating is that? How do I make it easy? I think that part happens in the next 18 months, and I still think that's, by the way, Angry Birds plus plus, right. The really fun part I'm really looking forward to is—I’m not so, another thing in that we said, health care. How do I take ten hours? Take ten hours out of doctors' times. Right, they spend a lot of time reading charts and saying, and it’s like, give me the TL;DR on what Chet’s about, right? I've seen him for 18 years. Tell me what I should go and focus on. But if any anomalies that you see in the last ten years because he's been 18 years, he's coming to me for the same tests, like a physical. So just tell me what it is and let me go and take a look at. Given how old he is, things like that, that's all great. What I'm really looking forward to is how does it fundamentally change my life? There is a—there, how do I—I'm asked, I'm talking about something more effective. Amazon and I go back to Amazon or Uber. Changed my life, like significantly changed my life, like it was a step function change. Now I don't go to a store, you mean, and so how do you make—what is the GenAI version of that? That part I don't know yet. And by the way, nor do all my colleagues who are building companies, they don't know that yet. Right. So we just—but you have to believe in the concept and go off and make it happen.

Jeff Frick:
Right. I love that, we’ll all get past Angry Birds. We can enjoy the Angry Birds while we're here, but we'll all get past Angry Birds. Well, Chet, I really—

Chet Kapoor:
My one thing. The one thing I would tell you though, Jeff, is the one though—you know, it's—the interesting thing about GenAI, you can see how interested I am, how enthusiastic I am. It comes from a place of being a technologist. But guess what? If you're not—my only one request to you is, lean into it. This is not going away. You have to lean into it because it's going to change your life whether you like it or not.

Jeff Frick:
Yeah, well, the last little concept, I just want to close on. We didn't cover it earlier, but I used to ask people all the time, if storage, compute, and networking were free, what would you build? Right. And we're asymptotically approaching that every single day. But what this is enabling is this thing it’s called the Lost Einsteins concept, which is a paper that came out in 2018, which is, if you know some person in Africa, some young girl has the cure to cancer in her head that if she only got access to the education and the resources that she would cure cancer, this now really starts to get to put the power again of that supercomputer for very little cost in the hands of anyone—today with a mobile phone, tomorrow with a watch. I think that is so, so, powerful.

Chet Kapoor:
I would agree. It's a great way to put it. The Lost Einsteins.

Jeff Frick:
All right. Well, Chet, thanks again for sharing your perspective, your enthusiasm. I really appreciate the time.

Chet Kapoor:
Thank you. It was great. I'm very glad you had me.

Jeff Frick:
Absolutely, thank you.

Chet Kapoor:
Thank you.

Jeff Frick:
All right. Well, he's Chet, I'm Jeff. You're watching Work 20XX. Thanks for watching. Thanks for listening on the podcast. We'll see you next time. Take care.

Cold Close:
Great.
Was that good?
Thank you. It was awesome.
Okay, great. That was fun.

------

English Transcript
Chet Kapoor: GenAI, My Agent, Human-like, Reinvention | Work 20XX podcast with Jeff Frick Ep32

Disclaimer and Disclosure

All products, product names, companies, logos, names, brands, service names, trademarks, registered trademarks, and registered trademarks (collectively, *identifiers) are the property of their respective owners. All *identifiers used are for identification purposes only. Use of these *identifiers does not imply endorsement. Other trademarks are trade names that may be used in this document to refer to either the entities claiming the marks and/or names of their products and are the property of their respective owners.

We disclaim proprietary interest in the marks and names of others.

No representation is made or warranty given as to their content.

The user assumes all risks of use.

© Copyright 2024 Menlo Creek Media, LLC, All Rights Reserved

‍