Inside Google's AI Teams: Logan Kilpatrick on Gemini, Builders, and Enterprise Adoption

Adam Davidson: 00:00:00

This is Adam Davidson, host of the FeedForward podcast. I'm really super excited to have on the podcast one of the guests I think our members like the most this year. Logan Kilpatrick from Google, who.

How do we properly describe your relationship to Gemini? What is. I want to say who kind of runs Gemini. Is that a fair description?

Logan Kilpatrick: 00:00:22

Yeah, it's a good question.

My role is I lead a bunch of our developer products, and one of them is the Gemini API, where companies and developers get access to our latest models, but also AI Studio, which is our sort of platform for AI builders and developers as well. So I. I wear many hats, but I don't run. Gemini is very big. Gemini is everywhere. So I don't run all of Gemini, but for people.

Adam Davidson: 00:00:50

You don't run all of. You run all of Google. You run all of.

Logan Kilpatrick: 00:00:54

No, no, I also don't run all of Google. That is. That is the wonderful and illustrious Sundar Pichai running all of Google.

Adam Davidson: 00:01:00

All right, all right. Okay, gotcha. All right. So I do want to, like, not sleep on AI Studio, by the way. Like, it is.

I feel like one of the potential downsides of having so many products is that some maybe get a little lost. You know, like, Gemini 3.0 really captured the world and was much talked about.

Maybe it's just my impression, but AI Studio is like a major suite of tools that. I don't know, I don't hear people talk about it as much. I mean, obviously, I'm sure you know the user numbers and it's.

But maybe just tell us a bit about AI Studio since you brought that up.

Logan Kilpatrick: 00:01:37

Yeah, I think AI Studio has been on this interesting arc. So I'll sort of frame today's context, which is today there's sort of two main experiences that are a part of AI Studio.

On one hand, you have the Vibe coding experience, which we're making this huge investment, and I think there's this massive opportunity. We should talk about Vibe coding. I think it's, like, changing, and I see this now inside of Google, which I think is really interesting.

Like hundreds of people every day sort of realizing that this is possible and changing the way that they work.

And I get pings from, like, every corner of the company, from somebody who's in sales to somebody who's a PM to somebody who's in legal, to, et cetera, about how sort of they're changing the way that they work using Vibe coding, and they're doing it through AI Studio, which is really exciting.

The sort of traditional AI Studio use Case was we have a playground and it sort of lets people who want to experience the fullness of the Gemini models and sort of their most basic form experiment with what can you do what's possible. And in some sense like the vibe coding story is an accelerant of that original user journey.

I think it's just like a more salient version of that and it is like, I think it's like AI Studios. This was according to the Andreessen Horowitz folks, but I think it was like the top 10 most visited Gen AI websites in the world.

So it does actually, I guess I'm wrong.

Adam Davidson: 00:02:57

Yeah, yeah.

Logan Kilpatrick: 00:02:58

Our audience is, is focused more on sort of the builder Persona. So it's not going after right now the sort of long tail of consumers, which I think is why it sort of like comes up in less conversation.

So it's a focused, high intent audience, but there's millions of developers every month and tens of millions of people using AI Studio and many millions vibe coding. So it's, it's a lot of fun.

Adam Davidson: 00:03:20

And it, I mean, I guess the shift in my mind is a few months ago it was kind of intimidating looking like you had to make a lot of choices about which model you want use and you know, and, and you still can make those choices, but it's just, it's just more obviously useful in a way that I think a wider range of people could just jump in and start building whatever they want to build. So I highly recommend people jump into aistudio.google.com like there's, you know, some of the other tools like Claude Code or Codex from GPT.

Like I think the leap to CLI command line interface, you know, I'm seeing that as like a real division between people who, you know, it's actually not that hard, but it looks terrifying the first time you're confronted with that. And AI Studio I feel like has turned that corner. It's now, it's a very serious piece of work. But it, it's friendly now or it's use.

It's, it has an entryway that's accessible 100%.

Logan Kilpatrick: 00:04:20

Yeah. And I think the only other comment is I think if you're looking for a Universal personal assistant, that's where Gemini app comes in.

So I think if you're like looking for this assistant product that's hooked up to your Gmail and your calendar and your Google Data and does all these things and you want to go shopping, like that is very squarely the, the Gemini app use case. If you want to build stuff. And you're like, hey, I have this idea, I have this company, I have this product. I want to put AI into my products.

Even that's sort of where AI Studio comes in, comes into the picture.

Adam Davidson: 00:04:52

Yeah. I will say something I highly recommend to folks who haven't built a tool for themselves is start in the chat interface.

Start in just regular old Gemini and just talk to it. You could do it on your voice memo. You can type. Yeah.

You can give an audio file from your phone and just talk about your job or talk about something you want to do with your family. Just kind of blather about it and then say, you know, analyze this. What are some AI tools I could build that might help me in doing this?

Augment it, make it better. And then when it has an idea that you like, say, how would I start that in AI Studio? And then you dump that response in AI Studio and you can.

I find that workflow is really fun and it's like my mom could do it. It's not that hard.

Logan Kilpatrick: 00:05:44

Yeah, I agree. I think there's this discoverability standpoint which like people don't back to like AI Studio is this high intent builder space.

I think there's a lot of people who aren't high intent but want to build something or they're potentially curious to build something. They just like they're not naturally ending up in AI Studio or they don't actually yet know that they should be building something.

They have this idea, but it sort of hasn't clicked. I want to build yet. And that's where I think people actually have this incredible aha moment.

Or I think the other audience that we see a lot back to some of the internal folks at Google, and I hear the same from other companies outside of Google as well, is people who have like been trying to build this thing but, you know, maybe went to one of these CLI products or something and. Or a more technical developer product and were like, ah, this is very clearly not for me. I'm not at that level yet.

And sort of are lost in the sea of, of other options and then made it into a studio and it just clicked and it worked. And I feel like we were talking off camera before this.

I think the model quality has also been a huge part of the story, which is like the models are now finally getting good enough across all these dimensions where like these use cases, you sort of had to squint your eyes and imagine, maybe this will work in the future and maybe my company should be investing in this and maybe I should be investing in this too now. Actually works like, it just works out of the box. It's a single shot.

You have the thing you want in a hundred seconds and then you're like, holy crap. Actually, I should be. This is like the promise of the technology is finally becoming true.

And I feel like I see that happening in all these new domains these days, which is fascinating to actually see play out.

Adam Davidson: 00:07:23

in the. I would say December: 2024

feel like we started December: 2024

d like, you ended December of: 2024

I mean, this is impressionistic, but I talked to a lot of Fortune 100 leaders of AI deployment.

It's like it's gone from like, I know it's going to be a big deal, but I don't know when and I don't know how to like, okay, we got to do this, we got to do this. I think we can do this. I don't quite know how far it's going to go, but I feel like this is another pivotal moment.

Logan Kilpatrick: 00:08:27

Yeah, I completely agree. And I think like, one of the interesting implications is I think we'll actually start to see agents as an example, really starting to work.

There's been a huge investment on the model side into like making a lot of these things functional. I think you're kind of seeing this in coding, like coding.

Agents, I think, like, sort of predominantly are working now and, and sort of developers are seeing that. But that's sort of always been the conversation around AI.

For what it's worth, if you look at like the most successful deployments of AI so far, it's actually been mostly coding related.

So I think we're actually going to get enough improved improvement in performance with this like, current suite of, of Gemini models with Gemini 3 and sort of the other, the rest of the models in the ecosystem of the same generation. Like, I think those use cases actually really start to work. So I think we'll see like product adoption across, you know, shopping and support.

And I think research, like deep research is really working and continues to work really well.

So I think there'll Be like more of these examples where there's a true consumer demand or a user demand for a bunch of these things, which I think is also part of the gap in a lot of the investment that was happening before is you were sort of preemptively making this investment to lay the infrastructure foundation, but there wasn't a huge amount like consumers didn't actually want that or users didn't actually want that thing yet. And I think it's now it's gotten good enough that they actually do want it, which is really exciting to see, so it makes it easier to build.

Adam Davidson: 00:09:56

One big area for me is AI as storyteller. Like its writing quality and its audio quality.

You know, I worked on the NotebookLM Google Labs audio podcast feature and I work with a bunch of academics kind of on the frontier of AI and storytelling and same deal like last few weeks. I'll be honest, I still think it has a lot of room to grow.

I think base competent writing, it's quite good at but narrative structure and we're not at the point where an AI tool could write a whole movie and have it be innovative and stuff.

Maybe you could jerry rig something that put something together, but just what it feels possible has expanded in that space and that's like an ultimate. You know, there's no obvious evals. It's not obvious. You know, any 10, you know, world class writers would disagree on which sentence is the best.

Or they might disagree. They might all agree on which is the worst, but they wouldn't all agree on which is the best. So it's these softer, fuzzier things.

Not that coding doesn't have bits of that too. There's a lot of sensibility and, and opinion.

Logan Kilpatrick: 00:11:07

Something interesting on this really quick, which I'm curious what your take is.

I, I feel like for myself as somebody who loves AI technology and spends all my time building and using it, I actually don't use AI for writing at all.

I do use it for coding all the time and, and some of my worldview has been like, I'm less personally like I feel like my code doesn't represent who I am, but I feel like my words represent who I am and what I say and what I write represents who I am. Um, and I don't know if you've felt this as well or if that's.

If there's a, if there's a thread, there's a through line on this, but it almost is like even if AI was way better at writing, like would I want it to represent me in that way. And I feel like my default for. For anything that's externally or internally facing that, like, people who I work with or respect are going to read.

I'd almost always want to, I feel like, write some of that stuff by hand.

Adam Davidson: 00:12:00

Yeah, that. That's a great question. I mean, and I think of writing as a process.

So, you know, writing when I was a staff writer at the New Yorker or at the New York Times Magazine or when I wrote a book, like, there's. I think of a whole bunch of steps that go into writing, you know, from what's a good idea?

To, you know, doing some research into, you know, is that a good idea? Or what are. You know, what is this kind of space of that idea?

You know, when I ran Planet Money, like, you'd run into someone doing something weird, and you'd be like, is this a trend? Is this something, you know? Or the Fed would make some announcement, and you're trying to think, like, how significant is that? Or.

And how would I animate that? So there's the whole idea generation and then. Which to me is a. Is a process in and of itself.

Like, you have a vague idea, then you have to kind of experience the world. If you're doing nonfiction storytelling, learn a bunch of stuff, kind of explore the whole idea space, then sort of like, all right, but what.

All right, now I've got to, like, tell my editor, here's the story I'm working on. So there's. Refining that. Then there's a lot about structure. Like, I. There's some saying about, like, professional writers talk about structure.

And then the actual, like, typing, you know, actual, like, what words go in what order is. To me, it's.

Logan Kilpatrick: 00:13:15

It's.

Adam Davidson: 00:13:15

I mean, sometimes you. I do, like, early drafts. Early. And then I do late drafts. Late. But it's. Yes, I think I would only do that.

Or for the most part, like, AI is not very good at. Or not as good as me. It is very good, but it's not as good as me. But it's better than a lot of other people. I mean, that's obnoxious to say.

I don't mean. But every one of those steps, it's really good at brainstorming ideas. It's really good at kind of researching them.

Something I love to do is like, I have.

I have this, like, agentic flow where I have an idea and I have it decompose that idea into all the assumptions within that idea and then have it go off and find who are the academic disciplines that have thought about this or who has written about this. And then I have it summarize all of that. And then I like, it's insane what I'm able to do stuff that some of it would have taken me months and stuff.

It's just literally impossible. And if I think of that whole process of writing in a weird way. Yes. The actual typing of words, it's hard for me to give that up. I agree with you.

I'll be honest, there have been some times where I didn't. It didn't matter that much to me and I just did let AI just generate it or I would just blather into it and then say, clean this up.

And it, you know, some formal letter I have to write or something like my wife and I had to write, you know, for financial aid, we had to write some letter. And I feel like, great.

It knows it's better at figuring out what a financial aid office at a university wants to hear than I am every single other step of that process, especially the crucial part, which is just brainstorming about what does this mean? How should I think about this? It's amazing at. I also think, and this is in the conversation about writing and AI is.

I see clear communication as like a major constraint on personal career advancement and on like economic development. Like you, you know, this is something like Matt Bean from UC Santa Barbara, who's one of our feed forward experts, has studied, you know, you.

When you go deep into any workplace, you'll find people who are adding enormous value through their ideas. But maybe they're not native English speakers or they just don't. They're not particularly good at communication and they often.

Their ideas are often either actually stolen or effectively taken over by higher ups with more power. Maybe this is a fantasy, but I love the idea of.

And just any workplace, even if, you know, I worked at the New Yorker, it's like the best storytellers in America. We're constantly having like miscommunication or misunderstanding. Like, I'm sure like any organization, you have that all the time.

So if it can help like level up, you know, every email and every, you know, I find it generally is really good at diagnostics. It's not always as good at solutions. But I have sent, put emails in and said, I'm really mad at this person, but I don't want it to be too harsh.

And it'll be like, it actually sounds pretty passive aggressive. Like tone that down, you know, I mean, in the AI way. And it's right, you know, so I don't know I have a million thoughts, but does that make.

I agree with you, though, that.

Yeah, I mean, obviously if I got an email from my wife about, like, some really meaningful thing in our marriage and I found out it was AI Actually, my son had that. He wrote a very heartfelt email to a girlfriend, and she said, was this chatgpt? And he was like, no, it wasn't.

Like, it was really hurtful to him to even think. All right, so.

So I want to go in a. I basically want to talk to you about the future, and then I want to talk about what it's like working within these places. I think, you know, basically the whole world is wondering what's going on in there. So I know you.

There's things you can't tell me, or there's a lot of things you can't tell me. But one question I find myself having is, how far out are you from, like, if we had. In knowing what's coming, like, my.

My general very crude understanding is you today have access to models that are probably one day going to be whatever, Gemini 3.5 and then Gemini 4. And so you have an instinct, and I'm not going to ask you to give me any secrets.

I just literally like, are you pretty solid on what we'll be doing in June with these models and where they are, or is it March or is it February? How far in the future does someone like you at one of the foundational labs know what's coming?

Logan Kilpatrick: 00:17:44

Yeah, this is an interesting question.

I think there's some perception that all the labs are just, like, sitting on all this space technology, and we're just waiting competitively for the day that we have to release it. I saw something about someone was like, oh, Google released Gemini 3 Pro, and it's really good.

So now they don't have to release anything for a long time, and they'll just wait until there's, like, a lot more competitive pressure, which is just fundamentally not how it works. I think there is, like, you know, in certain cases, the real nuance to this is like, the final buttoned up version of a model.

Like, we, like, there's. There's urgency to get it into the hands. Like, if we think this model is going to be so great and be amazing for the world.

Like, the impetus we have is like, get it in front of customers and, like, start that iteration cycle and, like, see how it performs and see how it, like, stacks up against our. The sort of ecosystem and where it skews and where it doesn't. And that Happens pretty rapidly.

Like basically as soon as we're like able to safely bring the model to customers and able to have the infrastructure set up to do it. Like it goes out into the hands of our customers.

Oftentimes actually it's like testing and certain customer use cases like before it's like widely available across the rest of Google and that's like, you know, that's a very, very common occurrence. Knowing what the future is going to look like is also an interesting scenario. So I think what we get to know is we get to know the inputs.

Like we get to know what is the research bets that we're making. You know, how is the, the sort of like recipe is, is what folks call it.

Like how is the recipe from a model perspective changing going into the next iteration. Gemini 3 flash in the future we don't know what the outcome is going to be. And that's the part that you do.

As you sort of train these incremental iterative checkpoints you do get a semblance of oh some early capability. The model's skewing in this way it's performing better than what we would have expected or worse than what we would have expected.

But because of how the pre training process is still on the order of many months and you don't until the end and until you do post training like you can't really, the models aren't super usable. So like a sort of partially pre trained model, even though maybe is somewhat, you know, is three months in the future versus what's available.

Like you can't use it, you can't do anything with it.

Adam Davidson: 00:20:11

I mean I've seen things that say they you whatever question you ask, it's essentially nonsense that comes out or incoherent.

Logan Kilpatrick: 00:20:17

Exactly. Yeah. Yeah. Because it's like a completion model. It's just sort of like emulating the pattern of the text that's coming in.

It's not actually post training again is like what makes the models actually useful.

And if it doesn't have post training it doesn't, it's not going to do, it's not going to interact like how we would expect the models of today's era to interact. So I think that's one of the biggest misconceptions as far as what's happening.

I do think there's like obviously like random research bet or not, maybe not even random but there's like a slew of other research bets that are happening and that's sometimes where you see like out of distribution relative to what's Available.

What's what we have internally and like, maybe like Genie or something like that is an example of this where it's like, you know, we have a great world model that really does a bunch of really cool things and like, it's not widely available externally and yet there's like a bunch of active research happening. But I think part of that story is too, like, it's very much like a research project still.

Like, it's not like a prime time thing that's like ready for production and.

Adam Davidson: 00:21:19

Like, and a lot of those fail. They just turn out not to like, pay out in the way.

Logan Kilpatrick: 00:21:23

Yeah, it's, it's research. Like, it truly is research.

So I think all in all, like, the, and this is actually one of the fun things of this moment, in my opinion, is like the gap between research and reality, like, continues to shrink.

And the goal is, if you're sort of doing this right, you put pressure on what is research and what is reality, and you sort of get as many things as you can safely and sort of reliably out the door into the hands of people and sort of let them see the light of the day. Because again, you don't. There's use cases that our customers have that we don't have.

And so you don't know until you sort of work with, work with customers and find out which is what, which is what makes it fun. So I'm not sitting on any crazy.

Adam Davidson: 00:22:05

Unreleased models and, or some crazy capability that I can't even imagine.

Logan Kilpatrick: 00:22:12

Yeah, no, not, not, not yet. I mean, I think again, there's like, you know, there's research bets on crazy capabilities.

I'm sure that we, that we can't yet imagine, but, but they're not like fully baked things that work reliably. And actually often what happens is a lot of the early scaffolding for those capabilities is actually in previous models.

And a good example of this is we're releasing Gemini 3 flash. And Gemini 3 flash is out the door. And it's an incredibly capable and powerful model.

And one of the new capabilities that we mentioned is this idea of visual thinking.

And visual thinking allows the model to actually, when you put multimodal inputs, it allows the model to use code to manipulate the images to further understand.

If you move it here to better see something or you add labels to the image or you change the contrast, basically a human would have access to all these tools, let the model do that, and it's actually been trained to do that. And the early version of the training for that happened in 2.5, it didn't work super reliably. It wasn't great and it wasn't a slam dunk.

And now you try it with 3 Pro with 3 flash and I think actually it works in 3 Pro and it like works out of the box. Like you can basically do like full anything you want to do to an image. You could almost do using, using this like visual thinking paradigm.

And it dramatically improves performance. You're looking at like in the order of like a 10% plus increase in a lot of cases, which is like very.

Adam Davidson: 00:23:47

Can you give an example of something I would to do? I want to go do it.

Logan Kilpatrick: 00:23:51

Yeah, that's good. That's a good question. I think like the contrast is one of these things. So like the model can basically be like, oh, like I'm, it's like clear.

There's like two images here and like I can't, like I'm actually having a hard time seeing because it's too dark or the foreground is too bright or whatever.

It can actually just go in and like use an image manipulation library in Python and change that and then relook at that image and be like, did that actually make the effect okay, maybe not. Maybe there's still something there. What if I just like slice the image this way so that I don't have to see those other objects?

Like, does that help me understand what's happening here? Or maybe I can understand what one of those objects is. Let me like add labels and arrows so that it's easier to depict sort of what's happening.

So that sort of process of like self exploration of what does the model need to do in order to get the most out of this image, I think is a super unique capability. And again like we laid the groundwork for that 12 months ago or eight, nine months ago or something like that.

And it's just becoming useful as a capability now. So I think we see a bunch of these types of things where the hooks are there.

We just haven't scaled and climbed model quality enough to like really make it useful, which is, which is cool. So like you can, you can actually find some of the clues hidden around if you're exploring the model capabilities.

Adam Davidson: 00:25:08

Yeah, that's really cool. That is, that would be very fun. So my next question is like looking further out, what are some of the big kind of horizon issues?

Like I'm working on a project with some academics using AI to translate cuneiform texts like from ancient, you know, from four or 5,000 years ago.

And there's Like a million of them in the world, which can sound like a lot, but to an LLM, you know, that's not a really, that's a pretty tiny training set and it's mostly in two unrelated languages and like half of them are kind of receipts, like with just numbers, you know, some of them are pretty amazing. So this whole issue of low resource languages I'm really interested in. And you know what, what if you have very small training data?

So that, that's an unsolved problem, right? That's like a big problem.

They're working on that by trying to simulate, see if they can create rules, kind of using non generative AI to just generate hundreds of millions of essentially cuneiform texts. And then you could imagine doing that like I'm studying Tibetan right now and AI is terrible at translating Tibetan.

It's just a very low resource language. There's not a lot of use cases.

So that's like one example of a thing that feels like that would unlock a lot if you could have fairly small data sets that could then actually be used for pre training. What are your, I don't know what's on, what are your fantasies of things that may or may not come to, to pass?

Logan Kilpatrick: 00:26:33

Yeah, well I think actually that example, this like low resource example is super interesting and like this is basically happening already today.

Like if you look and it's, it's not unique to LLMs actually if you look at what DeepMind did five years ago with AlphaFold, it was actually the same, it's the same thing. It's like there wasn't enough like human verified protein folding data.

And that was the problem of building sort of with other, with other sort of methods to like solve that problem. It didn't work.

There wasn't enough data, you couldn't actually and the sort of distribution was wrong and all that stuff and to do it like human, to like get the right human label data would have taken like four years. It was like four years per protein to be folded or something which just like was untenable.

So they built a bunch of synthetic data and that's how they got to the level of quality.

So there is something interesting where like as the model like capability sort of like generally the floor increases, you see all these things which like weren't possible before and like maybe Tibetan will be the next one that gets tackled.

It like has this capability transfer across everything else and enables all of these things just because the raw intelligence like I think robotics like a part of the reason the robotic story is going to work. Historically, it was like a data problem. Like, you couldn't get enough data in order to make this happen.

And then actually labeling all the data was exceptionally difficult because you, like, the human world is exceptionally visually complex and there's all these things happening. And, like, doing that at scale with humans is super resource intensive.

And if you've ever done human data stuff before, there's like, you know, you have to do all this work to make sure the quality is good enough and the sort of model is performing uniformly and all these things.

And I think this partially gets solved because the models have such great multimodal understanding that people are just, like, feeding all the training data through, you know, LLMs these days and getting all the right labeling, and it's, like, accurate enough that it just works. So I think you'll see more and more of these, like, unsolved domains get solved.

Not because there's some, like, crazy breakthrough transformational moment, but it's just like the model is now good enough. It's fundamentally a data problem. And you can basically use these systems to, like, get the right kind of data in order to make the.

In order to make the solution more tenable. So that's exciting.

going to see robotics work in: 2026

Like, I think people will have home robots doing the sort of, like, things we've all joked that robots were going to be doing. And maybe it's not going to be perfect.

g to be the case. But I think: 2026

And it's fundamentally because both, there's been a bunch of hardware advancements in robotics, but also, like, the LLMs image understanding has made it so that, like, the data process has just, like, accelerated so dramatically for these companies, which is really cool.

Adam Davidson: 00:29:45

That's really cool. And you could imagine that other, the visual thinking, like, yeah, you know, it helps a ton. Helps a ton.

Logan Kilpatrick: 00:29:53

Yeah.

Adam Davidson: 00:29:53

If there's suddenly, like, a blazing sunlight, just as it was carrying something heavy or whatever, like, it's able to process that, tone the. Adjust the ISO or whatever it is it does, and figure out not to drop a brick on your head by accident. Now that's really cool.

Are there, like, pet obsessions of yours that are, you know, just beyond our near grasp? Are there research areas you're fascinated by? Robotics, Obviously one of them.

Logan Kilpatrick: 00:30:19

Yeah. Robotics is definitely one, because I don't want to do the dishes, and this is my. Or my laundry, and this is me being lazy. So I'm sorry.

Adam Davidson: 00:30:26

Yeah, I would love an office cleanup robot. I will pay a lot of money for that, if that's available.

Logan Kilpatrick: 00:30:32

Yeah, I'm sure it will be, which is exciting. I think the other one, and this is more of a foundational capacity that I think the models are finally, again, it's a.

So many of these things we say are, like, model problems, and really, it's a much more complicated story than that. But proactivity is, I think, one of the biggest ones.

Like, I think this is my qualm with the current era of AI products in general is that I don't want to have to be the one in the driver. Like, I'm in the driver's seat of so much stuff in my life, and I'm like, in a lot of cases, I need to be pulled.

Like, there needs to be some balance of this, like, push and pull. And I feel like all of today's AI products, like, you're pushing. And I think for.

For people who aren't sufficiently motivated, which I think is, like, the default state and many things we do in life is, like, we don't want to spend the time pushing, which is very natural.

Like, the products aren't useful, so you can have this super advanced, capable thing, but if I have to be the one to put it through the paces, it's not going to work. And so I think we'll see that page turn hopefully in the next 12 to 18 months where the models really start to pull the.

The opportunity out of you in a. In a really positive way. I think, like, I would love that. And. And this is like, what do you.

From the best thought partner or the best, like, partner in real life or, like, the best teammate? Like, that's what they're doing. Like, they're. They're sort of pulling the potential out of you and pushing you to go and do this stuff.

And it's, like, very much like, showing up with, like, I did this thing. Like, let's collaborate all those things. Like, the models don't do that today.

And I think that the biggest gap that, like, I feel in this question of, like, what would I need to feel like I had, like, a super Advanced AI teammate that could, like, help me get more done. Like, it just doesn't happen right now, today for me because the model doesn't have that level of proactivity.

It's, like, exceptionally reactive to what's happening. And even if you look at, like, the.

So many of these product experiences, even the ones that are not, like, deterministically reactive, they're like, oh, set up this flow that if this type of thing happens, go and kick this thing off and then, like, do this type of response. It's like the. It's like everyone very much thinking about these products in a reactive way. And I think things will change when it.

When it moves to be proactive.

Adam Davidson: 00:32:50

Every few months, I build another version of my personal AI assistant, and the last one got farther it. It. I mean, I use cron jobs to make it run a few times a day, and it has my emails and my calendar and it's.

And it has a bunch of different prompts to come up with suggestions of things I should work on, identify things I've neglected, but it's definitely not there. And it gives me way too many things and they're the wrong thing. But yes, I keep thinking I want to wake up in the morning and have it.

Just tell me, with all your requirements, with your energy level, with your. Your personality, here's a great, exciting thing to work on.

I was actually telling my chief of staff the other day, I think that I'll be able to just spend almost all my time doing the thing I most want to do. That's also the most impactful thing. I don't think we're there, but it feels like I can almost taste it.

Logan Kilpatrick: 00:33:54

Yeah. And I think part of why this hasn't happened is there's just so much nuance to this.

As I think we're having this conversation on just reflecting in real time. It's like. Like, you know, part of the reason this works for humans and in some sense is because, like, we have lanes.

Like, you know, my, you know, incredible co workers, like, aren't pushing me in my personal life.

And like, my, you know, the incredible people in my personal life aren't telling me, hey, Logan, you should spend more time in meetings or you should say this thing to these types of people you work with. Like, there is this interesting, like, separation of responsibility, and I think it lets you compartmentalize these things in an interesting way.

And I think from. Again, from, like, a product direction, I think my worry would be, like, it.

It sounds appealing to build this, like, universal assistant that can do anything. And yeah, I wonder if it's like, actually you sort of need some of these, like specialization in different domains and like all of that context.

Adam Davidson: 00:34:50

Yeah.

And maybe it's a whole bunch of them, but maybe there's a master assistant who's taking in input from all these, you know, your work assistant and your assistant who's only focused on making more money and the other assistant who's only focused on optimizing happiness. And a psychologist would talk about immediate happiness versus long term life satisfaction. You could imagine.

I mean, I kind of built a crappy version of this where I have a bunch of different agents with different concerns about me arguing over what I should do. Again, it doesn't quite work, but it feels like, I don't know, it feels. I mean, there's a whole other question which is like, do we want to be happy?

Is that like, I sort of find myself thinking of these more existential questions of like, is it possible for the human being to just be like, yeah, I'm pretty, like everything's working good, you know, or are we just going to mess it up? That's who we are. Those are different questions.

Logan Kilpatrick: 00:35:51

Hey, I'm happy. Life is good.

Adam Davidson: 00:35:53

Your life, you're very happy.

Obviously I'm loving talking about these bigger questions, but we, you know, our core membership really is focused on obviously enterprise deployment. And so I do want to shift to that.

I want to walk through some of the core problems that we keep seeing and just sort of where you think of enterprise and AI.

I do feel like, like less than a few months ago, but we still do have a lack of alignment at senior teams, at senior, like C suite board level, you know, not everywhere, but we do see, I think we take a pretty strong point of view that this is maybe the most transformative thing to organizations ever, certainly on the level of like electricity or something. And that it's going to completely rework organizations. But I'm not quite sure everyone knows that, but let's just leave that part of it aside.

Logan Kilpatrick: 00:36:44

So.

Adam Davidson: 00:36:45

So one really simple thing, simple in explaining not simple in solving, is building deep confidence in a probabilistic machine. And so you know, this man, I noticed this manifesting itself in different ways. Like there's certain stories people tell about AI causing problems.

of the stories are from like: 2023

There might be many more than two dimensions, but one dimension is just. It's hard for our brains to think of a computer doing something probabilistic as opposed to deterministic.

And then the second thing is, how do you get enterprises confident in a tool that the exact same input has different outputs?

Logan Kilpatrick: 00:37:48

I think it's a tough situation. I think so. A couple of meta comments. Like, one, I think back to that story of like, you know, enterprises not having successful AI deployments.

I think some of this is like, how. I think you.

You sort of turned the question on its head a little bit, which is like, how successful did enterprises even want to be with these deployments? A lot of this is just like fact finding.

It's like, hey, like, again, we know the direction of travel is that these things are going to be good enough over time, but maybe we're not ready and we want to sort of do a bunch of these experiments and like, inherently, like, what even does success of some of these experiments look like? Like, would we really have rolled this out? Is the models good enough? Do we have the sort of money set aside to scale this thing up into production?

So I think there is a lot of which I think is the right thing.

I like a part of this enterprise story is like doing this experimentation, building this muscle building, start understanding actually that these systems are not deterministic and that they're inherently probabilistic and that they're actually at the sort of macro level. They're always changing.

So, like, there is, like, you know, what worked well for this one model in this one era is not going to be the thing that works well for the next model. And hopefully a lot of the stuff carries over, but some of it has to be retuned. So I think it's been this, like, entire.

How do you educate this, like, mass set of millions of people who are business decision makers and sort of builders of this technology and product leaders to, like, be able to make educated decisions about what's happening? Like, that takes time and that's the inherent reality. And again, I also think in this world where you have to.

Where you historically had to extend disbelief that this thing could actually provide value for your company and you were sort of doing it because, like, you're trying to be on the cutting edge and sort of follow the trend but maybe it's 12 months away, maybe it's 36 months away. You don't really for sure know. I think a lot of people have been hedging, which I think is part of the challenge.

So I mean the advice that I give is always like a pretty standard set of advice.

I think like you should find the thing that is like actually potentially economic transformative for your business and ideally one that's like going to be enough to like get folks excited about wanting to make the level investment because it is an investment and like figuring out how to have the organizational capability to do this type of work and go make that happen. I think there's like evidence of like very clearly beneficial things. Low hanging fruit like coding is perhaps one of the easiest ones.

Like if you're a large employer of humans that write code, like you should be making a massive investment in making sure that those humans have AI tools to support them as they develop software, deploy software, maintain software.

And I feel like the benefit of that is pretty cut and dry and I think gives you actually a lot of shots on goal of like the challenges of using these tools in a production setting and environment before you even start to like how do we put this in front of our customers and how do we products around these things? I think being a customer of the technology before you're like a deployer of the technology I think is like the correct order of operations.

And thankfully I think there's other domains now too. Like I think research is one of the big ones, like doing deep research type queries.

Adam Davidson: 00:41:09

It is utterly true. If that's all AI did, it would be like the biggest thing that happened in my life.

Logan Kilpatrick: 00:41:15

Like it's, yeah, yeah, it's, it's so massively impactful and it just works.

And like it takes something that would have been, you know, many, many human hours, hundreds of human hours perhaps in some cases and like distills it down into something that like I can consume very quickly is incredible. So I would go after the things that are like very much working and get the value from the fact that they're working.

And again it will expose you to all of the intricacies and all the complexity of AI and then go find those use cases.

I still do think having this North Star of assuming everything works, what will your company look like in 48 months from now if the trend continues exactly how it's going right now? I think keeping that in the back of your head like as a leader making a bunch of these decisions I think is important.

But the reality is you're going to iteratively get there. And the twists and turns along the way, the things you don't know.

Adam Davidson: 00:42:14

I would say, I mean, I would agree with everything you said, but I would add you should be spending at least some amount of time doing things that can't do yet. Because if, if you only do things it can do well, you're always going to be a few steps behind.

First of all, you might find something it can do that no one else knows it can do. But also, we were talking about this personal AI tool I've been building. Every three, four months, a new version of.

And it's getting way better, it's getting way closer. And with writing, a year ago I said it's nowhere near as good a writer as I am. I now say it's not as good as, as a writer as I am.

I'm expecting a year or two from now I'll be like, it's pretty much as good a writer as I am. And then I'll have different choices to make. So I think it is like the business school professors talk about exploration versus exploitation.

Like, to what extent are you learning new stuff? To what extent are you doing existing stuff? And you really need to balance those. I don't know what the optimal balance is.

Logan Kilpatrick: 00:43:15

There's something interesting too about assuming you find this has been some of the challenge that I think conversations I've had with enterprise customers is you sort of maybe do actually sometimes doing that exploration, find some things that work that you wouldn't have expected. And I think some of this is you then feel the pressure to actually go and build that thing, which is really interesting.

And I've like heard, I've heard this many times. And then you end up with this, you know, this capability is not yet mainline.

And you have to do all of this product and engineering work to like, sort of build the harness to make the thing useful. And it still is like, not, not exactly what you want, but like, has a real cost to doing that.

And I think if I look at like, again, what that takes is like three to four months, six months of like engineering resourcing all, you know, human capital going into this. And then by the time that thing is ready, the sort of frontier has caught up then. So there is some interesting.

So I think there's like, you, you really, to be intentional about, like where, when you find one of these things, do you sort of go all in? And how do you have that, like, innovation, that research going into reality? Pipeline.

I think the pipeline actually matters a lot because if your pipeline is it's six months of engineering and product work for a capability that like is actually pretty close to where the horizon is going to be. Like you, you know you want to do that in the most nimble way possible and I think otherwise you.

Then again there's like this massive of human capital software maintenance burden of like now you have this thing that needs to be completely rewritten to make the next thing work. So like that pipeline matters a lot. And I feel like folks aren't investing enough in the pipeline.

They're just like going after it like the any other problem that they would have historically. And then you just set yourself up for failure when inevitably the frontier catches up.

Adam Davidson: 00:45:07

anyway. And I think early on: 2023

That was with our knowledge and it was just total way worse than you know, 1.5 Gemini 1.5 or whatever.

I will say trying and screwing up or trying and failing because there's a bunch of things going on here, one of which is just learning this is something, something Ethan and Lilac's research has shown that there is an AI X factor. There are people who consistently produce better outcomes with AI than other people. And it's not training and software development. It's not.

They haven't been able to find what that thing is. Which means we don't yet know how to give that to people. If someone doesn't have it. We don't know how to say here's how you get it.

I feel like though it's like, like the more I use AI tools, the more they drive me crazy but also the more I'm getting a sense of who they are. Like I have to just accept they think it's last year.

Like that's just the thing they think and hopefully one I will say if I have one request, just make it on every call, find out what today is because that, you know, we have this now. We have this now you do in 3.0 it.

Logan Kilpatrick: 00:46:29

Oh, I think it's, I think it's rolled out for everyone already, so.

Adam Davidson: 00:46:33

Oh, okay.

Logan Kilpatrick: 00:46:34

Just I was just talking to folks about this. Yeah. Oh good.

care about information up to: 2023

So you do need to, if you're like overly prescriptive for a bunch of the consumer products though, where like it is truly like a, of in the moment information retrieval, whatever it is, knowing what the correct date is is exceptionally helpful. So.

Adam Davidson: 00:47:06

to say latest version, Python: 2024

But I hear what you're saying. There might be different use cases. We're almost out of time. I did want. What do you want to know from our corporate clients?

Like one of the coolest things about this job or this opportunity is like Ethan and Lilac and Jessica and I, it feels like we're sitting on, we're sitting on the front lines of something. Like we're seeing, you know, a third of Fortune 100 companies figuring out AI in real time.

What do you feel like you want to know about what's going through their minds, what they're struggling with?

Logan Kilpatrick: 00:47:51

Yeah, that's a good question. I think there's maybe two threads to this. Something that I really enjoy doing that we spend time with customers doing is like what.

And this goes back to that, like what are the short term exploration things versus the exploitation things. I feel like being able to articulate like what that North Star actually looks like.

So that as we're building this technology, we sort of know like ultimately like we're building it with, with our customers. And that is like the foundational point is that like we're not building it in a vacuum. We're not like building it, you know, in some silo.

Like we're co building this product with both, all of, at least at Google, all of the sort of folks internally at Google in our product areas and a bunch of our external customers.

So being able to know that North Star I think is actually really helpful and what capabilities folks need to make that Northstar vision of like what you want your, your company to be in three to five years. So I think it's a, it's a fun exercise.

Sometimes it's difficult because it's a little bit abstract, but I think it sort of like helps us make sure that we're sort of of have the right angle as we're approaching some of these problems. And like if we're way off then I think it's helpful to know like hey, we should probably be really skewing in these other areas.

So I think that exercise is oftentimes fun and I think if there's folks that you know or other folks who want to do.

Adam Davidson: 00:49:11

Yeah, no, that is actually we had a whole member session today with Claudine Gutenberg at Wharton who really walked us through like a really helpful framework to think about that. And yeah, if that's of interest, I would just quickly say it would be very heterogeneous within any one company. You know, I talk about four levels.

Like level one is just AI is personal aid. Level two is like workflow, like a specific workflow automation.

Level three is when you really see organizational like the whole organization being reworked around AI. And level four is cross company, entire industry wide changes.

You know, and this is a standard, you know, you can think of containerization or cell phones or whatever. And I find it's interesting, people can get 1 and 2 and 4.

They really struggle with 3, which is like how my company is going to be different and what they really can't picture that that's been an obsession. I actually created an AI tool that just describe your company and it narrates what your company might be like under AI. But it's.

I did that a while ago and it wasn't that great. But I find that people really struggle to even picture that and it. And it gets really polluted by will I have a job?

That question or I don't even know if polluted is the right word. It gets conflated with these kind of existential questions.

So there's a great study of a big company that found that they used pre generative AI but they used a machine learning algorithm to reallocate decision making.

And it was definitely more efficient but it conflicted terribly with the existing power structures and flow of information and that basically doomed the whole. They really struggled with that. So, so I think these are deep, deep questions. Yeah, we should talk more about that. Yeah, yeah.

Logan Kilpatrick: 00:51:03

Not easy to answer, but definitely something that is worth spending cycles.

On the, the other, the only other last one, which I talk about this a lot like, like we're through that lens of like we're co building the, like co building AGI with our customers. Like I love the brutal and honest feedback. So there's things that you're like, hey, we tried Gemini and it was horrible.

Or, you know, we've used these products and it didn't work, or, you know, maybe people love it as well, which is. Would also be great. Would love that feedback. So our. Our team is.

Our team is everywhere and online, and please email me or whomever it is, and we would love to, like, fix these problems and keep hill climbing. And it's. It's fun. It is very much fun to, like, be in that process.

Like, I think if we didn't have this iteration loop with our customers, it would be a lot less fun. Like, it's super cool to see the iteration loop actually work.

And back to that point, the comment of, like, all of a sudden, the last few weeks and months, people are, like, really starting to understand how important this moment is, and a lot of their products are, like, starting to really work. It's an exciting moment. I think that's a.

That is a byproduct of all of this iteration that has happened in the last three years, which is really exciting.

Adam Davidson: 00:52:11

Yeah, it's thrilling. All right. Logan Kilpatrick, thank you so much. This was a phenomenal conversation. I want to have it all the time.

Logan Kilpatrick: 00:52:18

Thank you, Adam. Thanks for having me.