Vibe Debugging Went Exactly As You Might Have Expected

Transcript is auto-generated and may contain errors.

Hey folks, I'm just driving to CrossFit. I'm gonna talk about AI stuff today. Uh we'll talk about a sort of a recent thing I was trying out. Um kind of started off with a biddan kind of just like give some AI the crappiest prompts ever and see what's up. But the context is I was trying to look at some I don't know like what might have seemed like memory leaks in brand ghost which is a social media scheduling and cross pl cross crossposting platform that I build and uh one of the things that I had been trying to tackle recently was doing automatic uh image resizing across platforms and we've already supported like scaling resolutions, but not every social media platform has resolution requirements. But a lot of them do because there's an API have like a file size requirement. And that means that uh you know for for platforms that weren't enforcing a resolution, if someone uploaded a really big file, it would like just fail because platform won't accept it.

So, started building this out and then when I was looking at some of the server metrics, I'm like, I don't I don't feel good about this. Like, something's making me feel like it's using too much memory to do the task at hand. And so, I went down this rabbit hole of, okay, I got a code base and like there's many, many spots cuz there's like I'm trying to think how many plugins there are for posting to social media. there's a lot and I'm like when I see myself post for example if I post like a short form video or a a blog post link it's going to like 12 platforms so you know where exactly if there is a leak where is it or if we're being very inefficient with uh you know memory and it's all happening at once like where is that happening so started going down this path And the couple things that I like using for exploring I guess are GitHub copilot spaces I think is the what they call it.

So in GitHub you can go to uh co-pilot where the chat is and you can like select spaces there and it's basically like a it kind of feels like a chat GPT project if you're familiar with that. And um you can add different repositories as sources. So the really cool thing about that is that I don't have to um have particular things cloned. I can go cross repo. Uh if I'm on the move and I'm not at my computer, I can still, you know, talk to C-Pilot about my codebase and it has, you know, a full view of the whole thing, uh including like my front end and backend repositories. So pretty cool. Um, I like using that a lot and I I'll flip between a planning model or like something that goes like does a lot deeper reasoning and then like a mini model uh once the conversation's kind of flowing just so that it's a little bit more smooth.

So between that and then like co-pilot itself in Visual Studio if I know like um I want to be a little bit more a little bit more pointed and a little bit less like I don't know cast a net across the entire across the entire world. So, I'm trying to remember what the very first thing was, but I used a um a library called magic.net that was doing image conversions. And um that was my first sort of like check because I'm going okay and like yes I have a bunch of social media platforms but they all have to go through this code path through magic.net to do some resizing. So like what's going on with this? And I can't remember what the trigger was for me, but um and sorry, I apologize because this is going back like a little bit over a week, but one of the first things I realized was like I got to get offmagic.net.

I just can't remember what that trigger was, but um I think the other tool it's like I always go back to chatgbt when I want to like uh do some internet searching. Um I don't know like I'm not saying Claude or Gemini or whatever don't work. I just happened to like gravitate towards chat GBT for this kind of thing. And I think that at that point I was getting uh ideas from it around what alternatives there were. Right. I needed to be free um cuz I'm not paying for that kind of stuff. There's lots of options that I know will be free and I need to make sure the licensing's appropriate. And so there's a another library called uh Lib Vix. I don't know what VIPS stands for, VIPs. And so then there's a Nougat package for fornet to uh to integrate with it. And then I was poking around and it's like they show the benchmarks are like significantly faster for for NetVIPS.

Uh and then then the claim is that also they are more memory efficient, but like I don't have data to prove the memory efficiency part. So I said, "What have I got to lose? Pay co-pilot, go convert my go convert my image compression logic from using this library to net vibs." And this is the kind of thing where a prompt like that, you know, I gave it a little bit more detail, but honestly not much. A prompt like that I know is not going to be uh super successful. But the reality is it was able to do it with a little bit more handh holding especially around like test coverage. So it was able to do it but um it kind of put me into this situation where I don't I don't know that code that's been written now. So that's kind of a scary feeling.

Um also things didn't map one to one all the time. Of course it's a different package. So there were some things where I'm looking at the pattern and I'm like I'm not I'm not really sure about that. But I did have test coverage that was showing me functionally like I should feel pretty good about the conversions and resizing. So I said okay and then I'm going to do um locally. So in Visual Studio I was running benchmarks and uh this is where things started to get a little bit kind of like I the word I want to use is uncomfortable. Uh started getting uncomfortable for me. Not because something bad was happening, I guess, but more that I'm starting to feel like I don't know if I have full control of this thing in terms of like my understanding because I was running benchmarks and noticing that like memory seemed pretty good.

Right? If I'm just to give you an example, if I'm loading in a 20 megab picture, I'm not expecting that I only use exactly 20 megabytes. Like I know that there's going to be some extra copies. There's going to be some extra overhead. So, I'm doing this looking at the benchmarks and going I I feel like that's kind of what I would expect. Like it's over 20 megabytes, but it's okay. And then I'm using uh in Visual Studio there's a profiler agent. So, I'm asking the profiler agent to run the benchmarks and profile and um I think I think I was really hopeful that it was going to be awesome and it was just pretty terrible. So, and you know, like not not just giving me like stupid answers or not being helpful, but uh oh man, I'm hydroplaning an awful lot. That's not good.

um kind of doing the thing where you probably experienced this before where it kind of feels like the agent is stuck between it doesn't know if it's an ask or agent mode and it wants you to approve like every thought it has. So you're sitting there like why are we doing this? Like just go do the thing and it's like please approve and you're like I've been approving the last 20 messages. So, kind of just like a shitty experience and not helpful. So, I'm going, okay, I'm these benchmarks seem fine, but like I I don't feel like I've done a good analysis on this. And I tried running it and sure enough, like it was working in production, but there's there's a butt. Um, memory patterns still didn't look good. So, like didn't address my original concern. Uh, and I ended up breaking something. And the scenario that I broke is that there were situations now where some regular posts that were going out that were using large images were suddenly just breaking now.

And when I say breaking, um, I don't mean like crashing. I mean, uh, we we're rejecting them. And so the sort of the lesson here is that uh because I had AI go refactor this big bunch of code for me and I saw that it was functionally doing what I expected. There are some behaviors that uh that were kind of baked into that original code that I that I I didn't realize and kind of stupid. Oh my god, this highway is terrible. I don't think I've hydroplaned this much in like years. And I have to switch a bunch of lanes right now. So, I'm kind of nervous. Give me one sec. I want to finish my thought. But there's a lot of water on this highway. Come on. Let me over one more. Apparently in some nearby areas in Washington, it's like flooded like crazy. Uh but not related to rain is my understanding, but just this morning is pretty pretty awful on this highway.

I've lived here like in this area for a few years and I've never experienced this. Anyway, if I die, you know why. But um the the reality is it refactored some behavior away that I didn't even realize I had, but in hindsight I probably should have realized. And that behavior is that um it was not greedily loading data. What I mean by that is like uh if we if we knew that there was a large image, right? Um there's some things we can do to check do we need to go resize this thing? And so we could check those without loading the whole image in because if we don't need to resize it, don't load the image in because you can post a lot of images to uh to social media just by the URL. So like we don't need to download that thing. Don't bother.

Um so what was happening was we would load the whole thing in. That's number one. So this like nonfunctional beh uh requirement where it's just now loading it in. But then the other thing is uh it was doing some size checks after and being like oh that's that's too big. Like we'll just reject it. So, I broke the like the resizing logic worked, but the part that was trying to figure out whether or not we needed to resize that kind of sat a level uh away from that, I broke it in a couple ways. And that's primarily because I was so focused on memory. I was so focused on uh did like my conversions of pictures actually work and I kind of uh let AI do uh more than I probably should have to be totally honest, but kind of put me down this path and I said, "Okay, well, I have a couple of options.

I can go revert, but I actually I actually think that uh we're in a better direction now. It's just that I broke a behavior." So I started looking into this code and realizing that uh yeah there was a lot of like extra extra memory allocations, extra copying because what co-pilot was doing was getting the thing to work. It did in fact get the thing to work. It just did it in really stupid ways. So just to give you an example of what I mean by this and some of you might not be familiar with this and that's totally cool. So I'll try to talk about it in in more general terms. In uh inn net you can work with binary data in a lot of different ways. One is like a a bite array uh and streams are the other thing.

So if you're trying to download something from the internet, you would use something that gives you a stream so that you can stream in the data uh you know chunk by chunk or there's other you know kind of streaming mechanics. And so what was happening was that there are different methods that exist for uh this image library. And in some cases it was like okay I have a network stream. Oh, but uh to work with this part of the picture library, I need a bite array. So, like let me just copy the whole picture into a bite array. Do something like something trivial. And then it would kind of continue along and be like, oh, like I need a I need a stream, but I also need a stream where all of the data is loaded so that I can seek across that stream. Because if you have a network stream, you can't go backwards on the data.

it just that doesn't make sense. You can only stream forward. So it was like okay let me go take this this whole copy of the image. Let me go recopy it again into another stream. And again it's just so that it can meet the API spec of this library. So I'm going through this and going oh my god first of all this is terrifying. Uh, and now I'm going I'm not about to go ask AI to to optimize this because I don't really trust that it can do it now. So, I personally spent my my time hands-on coding with that to go eliminate those patterns. But, I'm not exaggerating. It probably took me five iterations or so of like I would get it improved and then I would look at it again and like, nope, there's still another pattern.

But to to pick up on those uh what I was doing was asking co-pilot in Visual Studio, look at this code and find patterns like this, find extra allocations, find these things, not not go fix them because I didn't trust that it could do the fixing, but go find them. And it did a good job of that. So it would say like hey this uh this method call that you're using here like that's going to make a copy of the image and like by the way this method does that three times. So it would find things like that that I wasn't catching. It took me a few iterations. Um then I got to this point where I'm like everything feels good. I feel like I'm not doing extra copies, but I'm watching my memory graph and I'm like I still don't think I don't want to call it a leak, but I don't think uh I don't think I'm dis this is another net thing.

I'm not disposing of resources uh at the right time. And so I was doing this kind of like ridiculous oneshot prompt across my codebase and just literally uh the the context is the entire solution in co-pilot in Visual Studio and it's like uh I'm trying to post images and videos to social media and I think there's a lot of um a lot of memory allocations like find inefficient memory allocations or uh leaks like that. Just silly uh super generic. And what was happening was that I I'm not exaggerating when I say this. Probably somewhere between 10 to 15 times I did a prompt like this and every time it would find something. Hey, you have a stream here. You're not disposing of it. or you're disposing of it, but you have like an early return for an error situation and you're not disposing of it and just ran through this loop like 10 to 15 times.

Um, and some of these were like oneline fixes, commit um, and then run it again and keep doing that. And I had like a a screen full of commits that were just like really simple things that uh, that Copilot kept finding. So, the idea there was that I wasn't relying on AI to fix it, but I was relying on AI to find it. And it could do so with a really generic prompt. I'm sure if I crafted it better that I could go narrow down. Um, but at the end of it, you know, I was, this is over a a good few days, but I was able to like watch my memory chart and see things getting released and, you know, I see it getting near the container limit where we're running and then I can see it drop back down uh, you know, a few hundred megabytes and I'm like, okay, like it's it's finally finally doing what I need to see.

Uh, so overall just a lot of back and forth with AI, but that's been a hands-on one for me recently. So, thanks for watching. If you got questions about software engineering, your career, leave them below in the comments. You can go to codecommute.com, submit them anonymously, or message me Devle leader on any social media platform. I will see you in the next one. Take care.