Does AI Change Our Standard For Our Main Branch?

Transcript is auto-generated and may contain errors.

Hey folks, I'm just driving to CrossFit. We're gonna catch up on some AI stuff for the week. And like one of the things that I was thinking about is that in some recent conversations kind of talking about how, you know, traditionally we when we're building things as software engineers, we work like there's design work, you know, we get a well maybe not for every single pull request, but like you get something put together, right? there's some work leading up. Then you get a pull request put together and then people are reviewing the pull request, right? Signing off on it, approving and then things are are getting shipped, right? And that's a generalization obviously because maybe it's a service rolling out, maybe it's, you know, just part of the mainline code and that's going to get released later. Whatever point is that generally speaking, I would say that we get things merged into the main line of code and then the expectation is that if it's Oh, these wipers, man.

The expectation is if uh it's on the main line of code, then it's, you know, it's good, right? People are are happy with it. It should be something that could ship the next time there's some type of release made. And um when I was thinking well sorry when we've been talking about this more one of the ideas I've been kind of throwing out there is like given the pace at which we can correct things follow up on things have code fixes right um is is this changing right is this expectation that when we put something into mainline it must be the most pristine thing and I I know that's a bit of a controversial thing because even as I say that I'm like that doesn't feel good. Like I want I want mainline to be the perfect thing. That's the whole point, right? Like once it's there, it's perfect.

But I think my my point is less about or I I want it to sound less like uh a debate between should we just be pushing code that sucks and like it's like it's busted and whatever just because AI can make more busted right after. Like no, that's not that's not my point. My point is that like uh yeah, we want to keep the main line good, but if you had um I'm trying to think what a good like I have a good example I want to talk about. I want to talk about more generally first though, but like are there situations where a PR could go through and you know it doesn't need uh extremely close inspection because there's enough confidence in like uh the agentic rules for how it was written, the tests that were put in place in terms of the uh you know monitoring and alerting and uh automatic mitigations that exist as as a service is rolling out or obviously this going to depend on what you're building.

Um like do you have enough confidence in the other pieces that it should be okay that nothing bad could actually happen or if something bad happened it would be minimized significantly like so that's maybe one thing to think about. Um, do you need to have uh like if AI was building something and you have whatever from either watching what it built, having some other type of like release evidence, um, if there was some type of like user interaction, like would you have enough confidence shipping it without having clicked through everything yourself or having a human have gone through that, right? which I'm using this one as an example because I think when it comes to things like user interactions like arguably there isn't a perfect user interaction right I'm trying to use this as a specific case to say hey like if perfect doesn't exist

what is you know what's your your base threshold for accepting here right so is that an example even today where you can merge things And sorry I say today uh like we're just uh about to to start using AI but instead of having AI write your you know your userf facing feature if you did why is this thing beeping? What is going on this morning? Um, is it possible that, you know, even with all of the effort that went into it before when there wasn't AI, that it wasn't a perfect user experience, right? I I'm it's a, you know, leading example. I think the answer is yes. Like, you know, we try our best, we do as best as we can, and if there's some things that we can improve in the UX, then we will. But of course, as much upfront as we can, we we would.

it would just make sense to to try and do that. So, one example that I found for myself recently was like uh was trying to make progress with uh clawed code. It's doing one of the the video game ideas I have and I think one of the important things to note here is like this is not shipping to users, right? Like at least not now it's not. So as it's iterating the the risk is extremely low that like there's going to be unhappy customers because there are zero customers and like there is currently no opportunity for there to be customers. So, you know, the it it's going to make it seem like the bar is set super low on something like this one because the worst that could happen is the build breaks and then I have two clawed agents that are working on this.

So, the worst thing that happens is like not only does it impact itself and it's like, "Oh I have to go fix the build or I, you know, I made something in the game look stupid." the the worst case is that it affects the other one, right? The other developer that's doing this. So, um again, this is a it's a real example because I am building this, but the constraints that we're operating in are different than like a production system. So, uh the example that I have been seeing over the past week or so is around like art assets and even level design. So, uh I don't like some folks might not know this, uh because this is one of my YouTube channels, but I have other YouTube channels and one of them is called the Dev Leader Podcast. And so, I live stream, uh Mondays at 7:00 p.m.

Pacific. And so, come check that out. And the last stream we were going over at the very end, I was showing what Claude had put together for this game. And like obviously, it's it's rough, right? It's not Uh, it's not like a perfect shippable game by any means. I know some people are were showing like as soon as Astra came out, they were like, "Look at this game that AI made and it's like a perfect clone of something." And I'm like, "Okay." But the difference is that like it didn't make all of the art from scratch. Like, not by by any means. And uh and also like to those people's credit, they're probably not a dummy like me who like doesn't put video games together. The only video games I've like built are just like hobby things over the years that have never shipped. I have like no professional or like good experience doing this.

I just like making game systems. So when I was showing this on live stream, it was rough, right? But one of the things I was trying to highlight in that talk was that if we think about our own experiences and how we like can push AI to do better things like I can have AI build software for me because I have a lot of experience building software. I'm not saying I'm the best software developer. Uh but what I am saying is I've been doing it for a long time. So when I am getting AI to to build software for me, there's a lot of experience that I can channel for that. When it comes to like game development in a particular art, like what's really nice is that for you know the first time in my life it feels like I can have uh something's getting put together and art is being generated right there.

There is art for the game. before I would have had to go buy art, find someone to go make it for me. Uh but certainly like there was no chance I'm gonna go drawing my own art assets. Like I am I'm so bad at it. Why would I do that? Right. Also friendly reminder that if you're saying I'm so bad at something, it's probably because you never practice it. But um I got to get to this other lane, man. I'm trapped. I'm trapped in the slow lane. So, for the first time, like using AI, I'm able to have art that doesn't look terrible. Well, sort of, because right now in the game, it does look pretty terrible, but it's iterative. And I think that's the really cool thing here. So, the artwork and assets that have been going into the game, um, you know, it's not blocking for the most part the rest of the game development.

it can proceed. The stuff that it gets blocked on, the other developer will sometimes put like transparent things in place. So like the feature itself is kind of useless. You can't even see what you're clicking on. But it's building the logic behind something. So, you know, the two developers can work in parallel even if the art assets aren't landed, which to me is nice because like I said, right now there's no paying customers. The art itself and assets coming together being an iterative process has been super cool because uh I have it snapshot sort of what it's delivered every few hours and it gives me like a screenshot report. Um and there's sometimes where like between two screenshots I'm like I can't really tell what's changing in the game. like it will list the things out and I'm like visually I'm not, you know, I don't see much, whatever, but but cool.

Um, and then the next update I get sometimes I'm like, "Holy crap." Like I I could definitely see what was going on here. And what's nice about this is that not only is it continuing to like deliver things into the game, but each each time this is an opportunity where uh if I if I let it get a little bit too far, right, and I see like, oh, okay, like uh object placement, for example, there's been uh the water system that it's using, it was not placing the water properly. So, it would put water in the spots that needed it, but the edges of the water weren't right. Uh, it was putting in some waterfalls that weren't anchored like at all to the right spots. Um, and so that's just even object placement, not even the art style. for the art style.

It was making um sort of like it's a side view game, so it was making areas where you'd have like textured rock and then all of a sudden it would just be like this really flat hard um like surface that was supposed to be the walkable part. And I'm like, hey, like, no, from like an art perspective, you you kind of had something that was working and then you you made it look bad with, you know, with this awkward flat surface. So, I can give it the feedback and then, like I said, when I get the screenshot updates later, not only is it correcting it where it did it, but it's also doing it for the other areas that I didn't tell it about, and then going forward, it's not making that same mistake.

So it's not that it's just like oh it learned on its own and it's just getting better and better like no like I give it some feedback and then it's improving right so this iterative process I think has been a really cool fit because everything is getting better how it's um you know doing the next feature is getting better which is great but it never had to be perfect right if I had to find like the perfect art to go ship first. It would have, you know, we wouldn't have even been having this conversation. So, I'm saying that because I understand that probably lots of you aren't making games. Um, that's not what I do for my day job, but I thought it was an interesting example where, you know, if we think about the constraints that we have, is there a way that we could be shipping and iterating Right.

The alternative I mean like there's there's different things like this that have existed, right? We could we could put things onto long live feature branches and if you set that online you'd probably get flamed by half the internet. You know, lots of people are extremely against that. You know, get it into the main line. Don't even use branches. What are branches for? Um trunkbased development, right? there's only one way to do things. But I wanted to even for myself kind of think about this because outside of art for a game that's not shipping, are there things where, you know, I could I could relax on it getting integrated? And furthermore, I think one of the more important questions that comes from that is like if not, why not? Right? And I don't mean like philosophically.

Um, you know, you're Nick, you're just saying to make the main line of code a pile of Like, don't if that's what you're hearing, I'm sorry that I'm doing a a poor job explaining this, but um my my sort of meta point is like if you're not comfortable with having everything try to get perfected and you're proving it to yourself like whether that's through review whether that's through whatever you're doing before something integrates I'm curious what it is that's not giving you that confidence right so if just to give you another example what I mean by this say you started a brand new app right and you have something working on your computer and you're like, "Heck yeah, I feel pretty good about this." Like, I'm going to, you know, push it to the main line. And then you haven't written tests yet, right?

Like you don't have like, I don't know, continuous integration or regression tests or anything, but you have something working and you're you're like, "This is pretty cool." And then the next feature, someone else does it, right? Whether it's AI or another person, and they're like, "I want to push this through, but you can't like you have to trust me that it that it's the right thing. Right? Like, pretend it's a friend of yours, too. Like, not someone who's totally a malicious but pretend it's a friend of yours. You can't can't look at the code, but I want to merge this in. You probably would feel a little uncomfortable, right? Maybe not, depending on who you are. Maybe this is also a very bad example, but I'm assuming some people would be like, "Yeah, that makes me a little uneasy. I want to know what the hell was built.

like I want to trust my friend but like I I want to know what was built. Um but like if you if you go through this thought exercise like what is it about that discomfort that's not giving you confidence in that change right does for you does confidence come from I understand every single line of code that's in the PR and somehow that gets you to a point where you understand every single line of code in the codebase at all times is it because you have unit test pass is it because you have some type if it's a visual based thing like you have you know screenshots that you can look through to make sure the UI at least looks okay like what is it that gives you confidence for a PR to go through because my challenge to you is like how do you

make sure as much of that as possible is done automatically right cuz even if you did, you know, this say this project matures and you have regression tests and those CI running and okay, all 10,000 tests pass like okay, you have probably more confidence like at least all the shit's not busted. Like that's good. Um, but like did you get to see the code like the design patterns? Like is it following the right things? like right there there's always I think there's always going to be some level of like that makes me uneasy to not know. So I'm not saying to to go solve this in a way that you know you never even have to think about this going forward. But my point is like if you're going through this thought exercise and you're like well here's a laundry list of that like um you know I need to know about before I have confidence.

Great. like from your laundry list, how much of that happens automatically? Because I would encourage you for all the things that you say are not happening automatically, go spend time on those. Uh, by the way, this has nothing to do with AI. This uh this conf this part of the conversation is like you could, you know, AI could not even be a thing and I would still be suggesting this. I think it's actually significantly more important with AI because historically even if you were the one it's not your friend you're the one making the change right how do you have confidence in your own damn change you're going to want you know your continuous integration system running so you're getting your tests run you have all these regression tests you might have UI based things you might have whatever right but we put these things

in place so we can get confidence and we want that even when we are making the feature cuz we don't want to break So if you extend that to a best friend, right? You want to trust your best friend but you're not exactly sure what they were coding. And now one more level of indirection. You have AI doing it. Oh, it's it's not a human, right? Well, how how can you get to a point where you have so much confidence that even if it broke something, you could have it fixed very quick? Right. Um, just as I park at CrossFit, one of the sort of philosophies I have is like not necessarily to do this, but the framing is I want to have things in place in my codebase that anyone should be able to come in and make a change. and they're kind of forced into doing the right things with the right test coverage.

Um the right signals are showing that things are safe to integrate or not, right? They have the feedback they need to even know if it's ready to submit and that if something were to break that I already have all the visibility I need, right? I that is what I want to have in my codebase. I'm not saying I invite any random person to go, you know, try to screw it up. My point is that I aspire to be in a position where anyone could. And if if you go through that thought exercise and you're like, "Wow, that makes me wildly uncomfortable because A B C" and you go through your list, go spend some time on those because I think that will help tremendously because those are clearly gaps where you have confidence in being able to deliver. So, thank you for coming to my TED talk.

Uh, if you have questions about software engineering, career development, leave them below and I'll see you in the next one. Take care.