Build It Once Or Build It Nonce! Then Have Agents Do It.

Transcript is auto-generated and may contain errors.

All right, folks. Headed to work. It's another day of leaving after rush hour and being told my 25m minute drive is an hour. So, we'll see how shitty it is. Today, we're going to talk about AI stuff. Um, I ended up doing a live stream on Monday night. So, this past Monday, it's uh Wednesday right now. and um we're going over basically some of the the things I'm trying out with like reusable pieces when kind of trying to build software with AI. So that was on the Dev Leader podcast YouTube channel. It's also if you're on uh Twitch or Kick or if you follow me on LinkedIn or Twitter like I was streaming to all those platforms. Um, that is another YouTube channel from Dev Leader, another YouTube channel from uh Code Commute. It's a Dev Leader podcast. And so, um, I figured I'd talk about some of those things, uh, you know, on on this drive.

Uh, I've talked about some of them before, kind of recap, uh, a little bit more specifically on on some of the things. So, the idea with that stream was just to kind of showcase like that, uh, at least for my own personal development. I can't speak for you know software engineering at work for me because I'm not writing software at work. So it's kind of a separate thing um for my own personal development. I am definitely uh glued to a terminal now which is a unique experience for me as a C developer primarily um because I'm no longer like spending time in Visual Studio. So, we've got a bunch of uh PowerShell tabs open and I'm using Copilot CLI. So, uh I I have been able to work around some of my uh limits very fortunately. So, um I was kind of jumping between Claude and C-Pilot code.

Wow. Uh Claude code and C-pilot CLI. my brain turned off and uh that was just again like no no no reason like that I don't like claude or something just have preferred co-pilot CLI and uh was flip-flopping just because I was hitting rate limits on both so now just uh really focusing on co-pilot CLI and and then I I'm jumping between a bunch of projects so I I'm doing a ton of context switching so working in one session on say uh Needler, which is my dependency injection, uh, you know, type scanning registration framework. I've been doing a bunch of stuff there for wiring up, um, support for Microsoft Agent Framework so that I can have more of a I don't know more capabilities with Microsoft agent framework uh, on top of uh, what's currently being offered. So, be doing some work there. Then when I have the work kicked off, then I can switch over to another PowerShell uh terminal and kind of keep going.

Like so I might work on Brand Ghost, might be doing some work on my own blog website and kind of just going across soon as some work finishes up that I can go back to that tab and kind of catch up. And so u I've been doing this flow for a little bit and um just really started to realize that there are some things that I am doing where I'm like oh yeah like I don't I don't have that set up um in this repository right like I should I should go set that up uh for some uh for some skill or I'm like hey wait a second like this skill is actually something that I should be using across um all of my projects so like let me go make that I have like a private uh marketplace plugin and like there's nothing special about that.

It's just a GitHub repo that's private and then I'm able to to install from there. Um so just kind of iterating through this and and trying to figure out some patterns. One of the more recent things that I realized is like everything I was just explaining there uh works and has been working for projects that I already have. Right? So if I'm already I already have Neadler, it's already something that I use co-pilot with that's already up and running and good to go. And so when there's, you know, something I'm working on in Brand Ghost and I'm like, "Oh that would be really helpful over here in Needler, it's a pretty low lift, right? Like I can copy and paste something if I need to or kind of pull it to my my shared marketplace." But there's some things where I'm like starting from scratch, right?

And so classic example that I was showcasing on the stream was like if I want to spin up a website super quick, right? Um for me super quick is using something that I'm very familiar with. And so I would probably jump to something like uh Blazer because it's C based. But I I've said this before like I don't I don't think C is the best tool for everything. It's just one of the most convenient tools for me personally, right? But that's it's a it's an excuse for sure, right? If you're just if someone came to me and said, "Nick, I would pay you to go build something and I need it done within a certain period of time, I'm of course going to try and balance like things that I can use effectively." It would, in my opinion, it would be stupid to not.

Now there are tradeoffs between like you know picking a technology that is going to deliver on what's being asked for but in in the overwhelming majority of cases I would say that you don't need hypersp specific technology like picking a a MySQL database versus Postgress for the overwhelming majority of things you would ever work on is not going to make a difference right now. If if someone said like there's so many differences between MySQL and Postgress, I could never work in one versus the other like the most impact is probably your familiarity and being able to use your tech stack properly. So I think like it is absolutely a factor. Now, I want to make sure that my my sort of uh selection of tools that I can I can use and get up and running and and do something with is always growing, right? Like that's why I was saying I don't I don't want it to be an excuse that I prefer C.

Like I prefer it and if I needed to, I would use it, right? Like if that's going to be a limiting factor, then like let me use C so I can work more quickly. But if I have other things and I have, you know, uh, LLMs and agentic skills and and anything else that can help scaffold things and work through stuff, then like all of a sudden my argument for like, oh, let me go build it in C is, you know, more and more reduced, which sort of if we pause for a second and kind of think about this as a meta point, how cool is that? Like how cool is that that all of a sudden someone like myself who is like I've been you know programming for a couple decades now but I will gravitate towards C because that is the thing that

I'm most skilled with and now I just have the ability to be like I don't I don't know this other language or I know of it but I'm certainly not like as skilled as I am in C for that language but I could go build something now and like pretty confidently like I feel pretty confident that I could go do that. Um that's that's completely like unheard of before having tools like this, right? So anyway, the idea has been that let me start putting together some reusable templates. And so that means that instead of every time I even for something in C instead of me going okay you know new folder uh new repository new solution let me go start with the you know generic you know example app that they ship for uh ASP.NET Core or Blazer and then okay let me start deleting the boilerplate like example stuff.

Let me start adding in all of the stuff that I always use. Um, like why do that? Like it it should be basically like one click or like one instruction to be able to have everything set up and and why stop there, right? This is kind of the extension of uh of my thought process for using AI tools is like why stop there? because like um there might be some skills that are repo specific and I want those skills here. Uh instructions, right? Instructions are a huge one for me recently. Why why stop at having like I'm going to go copy and paste instructions or start building them up over time. I don't want to keep writing different instruction files and managing like being like, "Oh I don't have that part of the instruction here. It's like 95% the same, but I added this one part that's really critical and these five repos don't have it.

I don't want to deal with that anymore. Like I don't want to live I don't want to live a life like that. And so I started coming up with this like templating system. And I'm not by the way I'm not talking about this like uh I'm revolutionizing at like things for others. This is literally just so that I can be more effective in my own development. Um I I'm assuming there's already people building more common things like this for others or you've done it for yourself. Awesome. Uh I'm just kind of sharing like my exploration with it. So started building out some templates for uh different types of projects and then on top of that like a common spot for my instruction files. And so in some of the previous videos I was talking about instruction files that aren't just your top level agents MD file or co-pilot instructions or clawed MD um but instead like these globased instructions that can match on parts of the file path.

And the cool part about that and Claude and Copilot support this I'm sure cursor and others do too. But the cool part about that is that instead of having like a ginormous agents MD file, that just becomes like kind of ridiculous because it's like every everything that you would ever want an LLM to know about any project you would ever build. Instead of having that happen, uh you can load in the context uh specifically for the files that you're going to go touch as an LLM. So for example uh as a C developer uh I might be opening up a a solution file. So SLN or SLNX and I might be opening up a CS project file or I might be working in my my uh build props or uh package props or anything that is not a C file, right? And like so like why not have dedicated instructions very specifically for those things?

And the reality is like any car project I go build is going to need that right. There's at least going to be like an overwhelming majority of things that I would want to have covered the same way in uh in an instruction file. There must be an accident here. And this is why it still says I have another 50 minutes another 50 minutes to get to work. My god. Um, this is like one of the dumbest things about return to office. It's like I'm I'm literally already behind on my schedule for number of times to get back to the office uh on on average in a week because it's three three days out of the five, but it's calculated over 12 weeks. I'm already behind. Um, and I have another on call shift coming up on Monday for 2 weeks. And so I don't have a choice but to drive to the office to try catching up on some days.

And this is this is a day. It took me uh an hour and 20 minutes to get home from work last night. So I took two meetings while I was driving. And I'm saying this not because like, oh, my my commute is so bad. I'm saying it because my commute is only supposed to be like 25 minutes, but when I'm backed up in traffic like this, it is so dumb that I'm going to waste uh either my own time, which by the way, like as soon as I wake up, there's already stuff for work that I'm addressing, and I'm addressing stuff for work until I go to bed. Uh, so like my time versus work time kind of kind of blurs together, but I'm wasting this time sitting here and I'm gonna have to waste similar time on the drive home for for literally no benefit except to be able to say that I went to the office.

Uh, I could literally drive to the office, swipe my badge, and drive home and um and it counts. So like why? Ask me how I really feel about this. So anyway, um I have these templates set up now. And one of the nice parts about that is that it comes with these instruction files that are that are file path based. And I think there's a ton of power to that. Uh because I was, you know, before before this recently, I was just trying to collect as much as I could in my agents MD file and then being like, man, like it's it seems like it's not really listening to me. like despite me trying to put all this effort into having a good agents file. So, um I don't know. I feel like that's working quite well. So, I on stream was able to I had never done it before, but I demonstrated like oneshotting with a prompt to go have an Astro website set up.

And the to me the cool part about that is like I have a couple of Astro websites. I don't even know what Astro is. Um, I'm not saying that to to brag, by the way. I should probably know what it is, but I legitimately don't know. I don't know. Uh, is it I'm assuming it's like a JavaScript based thing going on. Uh, I am pretty confident that it's a static website because as far as I can see, it's building all the files ahead of time, which is why I'm able to get a pretty solid performance out of it at runtime. Um, but like I don't I've never seen code for an Astro website. I don't I don't know what it looks like and I have a couple of them now. And on stream I did a sort of a joke where I'm like, "Build me a Pokemon website using this Astro template.

Uh, I want the main page to be themed like Pikachu and then I want to have three subpages for the starter Pokemon. and I wanted to show their evolutions but be themed by each one. And so ran that prompt and then was kind of clicking around to to show other stuff on the stream and within a couple of minutes like in one of my other browser tabs I had a site pop up and I was like what is this? It was it was uh it was pretty overwhelming, but I realized it's because it themed like I asked it to. It themed the main page like Pikachu and so it was super bright yellow. Uh but it worked like for the most part it worked. There were some gaps in the template and I had never used the template before. Um so there was a couple of bugs to fix with the templating but it did it.

It made a site. Um, so that was pretty cool. And I want to kind of shift to talk about um a little bit of an extension and it's I think it's a good segue because this template was broken and I needed to fix it. So how to go fix um because part of me is like okay template's broken. Um, I should go ask, you know, co-pilot to go look at what this sub agent produced, see where it's broken. I have a couple of ideas. So, let it know, but see where it's broken and then I needed to fix it, right? But I want I want to know the analysis of like what's broken so I understand before it's going to go do all this work. So, it did the investigation, presented the results, called out a bunch of stuff, and I'm go I'm reading it going like, h I don't think this is uh super specific to this template.

Right. Some things for sure, some things are very much like the Astro template is not doing the right thing or we we didn't have um a way to like replace some of the parameters in the template. So, like it was it was never going to work. That's fine. But some things I'm like, that's that's actually applicable to every template I have. Like every template does this. So if this one's busted, surprise. Like the rest are going to have an issue. So I said, "Cool. Fix this up and then test it the same way." And the cool part is to test it. It just asks an agent to go, you know, install uh install the template and basically build a little website and then it just checks in on it, right? So, it's kind of neat that you can have an LLM go build all the stuff and then another one go check in.

Um, so that's cool. What's going on here? Who's who's honking? Also, everyone's getting off the highway here, which makes me super nervous. Like, even even Google is saying get off the highway here. Oh, but the the lineup for that exit is like a mile back already. I wonder if there's something real bad on the highway up ahead. I'm already here. Whatever. Um, so, um, okay. So, all these other templates are going to be busted. How do we go address that? Well, we could go one at a time or we could say, "Hey, there's 12 templates. Go run 12 agents for me." And that's exactly what I did. I said, "Hey, there's there's all these templates. I want you to go use a fleet," which um I don't know if fleet does something specific um in co-pilot aside from just suggesting to the harness like use multiple sub agents.

to be honest. And I think um Claude has something like this. I can't remember what it's called, but anyway. Um been I've been using this fleet command a lot more. Um and I don't actually run like slashfleet. I tell co-pilot in my prompts like use slashfleet or I'll say use a fleet of agents. So I said use a fleet of agents. I want you to repeat this process for the other uh templates that we have and I want you to do the investigation. Right? So, basically get all of these agents to go investigate all of these templates and then synthesize the results back for me so I can get an understanding of what the hell's broken here then right then we can see all the results. And sure enough, of course, there were some very common broken things across the templates. There's a couple template specific uh gotchas like I had for my Astro one.

There was the same case for um I think I have a a template for setting up Roslin analyzers like if you wanted to go build a repository that was for a Roslin analyzer. Um so there was there was a handful of like specific things and then I said great like go use a fleet again to fix this stuff. Um, and the the cool part about that is that some of the issues spanned multiple projects or multiple templates, right? So, if it were super naive and it said, "Okay, like there's 12 templates. I need 12 agents and uh I'm going to go fix all of the things per agent um that are applicable to this template." You'd have overlap work, right? like you would have this some agents solving the same problem and literally in a common spot, but it was able to kind of like figure that out without me telling it.

It's like this this class of issues broken across all of these templates. So, it goes off and has one agent go address that. Then when that work is done, what's cool is that all of the templates, again, using a fleet of agents, all of the templates were validated the same way that um that my original Astro one was. And and furthermore, when I was looking through some of the results, uh I could see that, you know, it was like, okay, like fixing up this template. Oh, like we ran it with the sub agent and we still found another bug or a bug wasn't fixed. So it's like let me let me go make a correction and then again goes and uses a sub agent to install it to validate it and all this like I'm not telling you things that you don't know can be done.

I'm just kind of highlighting like this stuff is it's so cool that um you know sort of autonomously it was able to go say like I tried to fix this and it it wasn't it still didn't work so I got to I got to keep trying and and and get it done. Okay, my phone just updated. How long does it say now? Oh, that's the same time. So, I don't know why would do that. How far is it backed up? Okay, so it's backed up um the entire way to Belleview. which is nuts. Great. Okay, good thing we have returned to office. Look how look at how much productivity we're getting. Look how productive I am. I'm glad I'm glad that this is helping teams be more effective. It's working. See? Can you Can you see the effectiveness? It's It's such So, with fleets of agents, this has been something I've been trying a little bit more.

It's to me it's not as interesting to have a fleet of agents go fixing stuff because I'm like I I understand that if we could run some agents in parallel like great uh it makes sense that be able to do work in parallel and then kind of come back together at the end to regroup and and see what's up. So not super groundbreaking but like pretty helpful. Uh and I'm saying not super groundbreaking in the sense of like you probably are very aware of this already. I'm just sharing one such experience where I wanted to take this a little bit further and I wasn't planning to talk about this for 7 and 1/2 hours. Uh so um I'll probably wrap this up in not too long from now even though I'm not at work but um the the next step for me was like I have a handful of situations where like I'll be I'll be building something with AI that's kind of long running.

And so couple of examples of that are like sometimes I'm building needler features. I don't like doing this in brand ghost because it's too much like um if I do this in brand ghost, I feel like I'm going down a rabbit hole of not shipping value and like I would much rather like I have paying customers. I want to make sure that I'm getting something done out the door, delivered, feel good about it. And there's a couple of other projects like Needler where I don't really have that constraint. It's mostly because it's like interesting for me. So, uh there are times where I'm letting AI build stuff and and being a little bit more hands-off and that way it's just mostly running autonomously and then I'm I'm correcting it and steering it back. Um so I have that and then I have uh it mentioned like this role playing game that I've been rebuilding.

It's like purely vibecoded. Um, so that's a really fun one too where again like from a a software engineering perspective, I get to have full input on design, architecture, all this kind of stuff, but I don't have to go do the work, which is which is the best part. Um, but these these two examples, Needler and this role playing game, um, they're they're similar in the sense that like I don't have a a delivery date or something like I'm not held to anything like that. I can literally just, you know, kind of go explore build features or like refactor things and like kind of just enjoy that process. Um, which I don't know for some people as someone who likes code and programming and software engineering like I find that kind of stuff fun. I acknowledge that it's not um a lot of the time it's not actually delivering value like for me to go add some random feature into Needler.

Sometimes it's just like I think it's a cool idea. It's not because people were like, "Oh, you should go add this." or I'm sitting there like, "Oh, I desperately need this. This is the next highest priority thing ever." I'm just like, "This sounds cool to me and I would like to see it come to life." And I like I need some I need some of that uh to keep me sane, I guess. Uh because I think I think it's fun. I think it's cool. So with these two approaches, the challenge is that sometimes uh because it's kind of unbounded that AI gets to a point where like it's like I'm done and then I'm like are you really done? And it's like no I I made that up like there's still a lot to do and then it will go do that and I'm like are you done now for real?

And it's like ah if I'm being honest with you like no. Um, and then there's other times where like it it delivers something to me and I'm like, "Hey, that doesn't work." And it's like, "Oh, I am so sorry. Clearly, here's the bug. I'll go fix that." And then it doesn't. And I'm like, "Nope, that's still wrong." And it's like, "Okay, so I I keep, you know, okay, well, where are the tests? We need regression tests. Use TDD for this. Okay, I got it. No problem." And then, oh, still doesn't work. Like all the, you know, all 10,000 tests passed, but like still doesn't work. And I'm like, "Okay, we're we're solving different problems here. Clearly, like whatever you're testing and validating and proving is is not representative of reality because we're both agreeing that like we expect this behavior. You're telling me that you've proven this behavior with many different types of tests and it's still not working." So clearly, we're just we're looking at literally different things.

we must be and then it kind of gets stuck because it's like okay like we should scrap this and I'll try it again and then it does it again and inevitably the same happens. So I've had this happen now on two different at least two different kind of scenarios. Um, it happens a bunch in needler. Less catastrophic, more just like missing pieces. And uh I've had it in the role playing game uh a handful of times now where I'm like, you know, like delete everything you just did cuz we've been doing this for three days now and like it's clearly not working. So, one of the things I realized was and I I tried it out. I don't know. I don't know why it came to mind, but um I I don't use specialized agents ever. And so that's where this conversation is headed now. I don't ever use specialized agents.

And for this role playing game, one of the things that I was trying to address was that uh I use like I'm I'm I'm not an artist. I'm not good at making models. I don't want to be in the business of making models or textures and stuff like that. I'm like, if I'm doing this, it's going to be AI driven, so we need to make sure we can do that. So, I had it do um some like there's procedural levels and stuff for dungeons. There is like some of the like tile set room pieces so that when it builds the layout, it can kind of put these pieces together. And it was just doing a terrible job. Like absolutely terrible. So um I said like after we had iterated on this so many times and it was like I've mathematically proven with these tests like I've done a hundred iterations we never have pieces that don't fit.

I would run the game once and instantly would see a piece that doesn't fit. And I'm like this is either the most unlucky person ever and then I would run it again and the same thing would happen. And I'm like okay it's it's not luck. we're just we're literally focused on different representations of reality. So I said, "Hey, look, like we've been trying to build this game based on some of these other games." And when I've been uh going through like game mechanics and building it like designing features, I guess less about the implementation and more about like the mechanics of how I want things to work. Um, I would say like I want you to go research Path of Exile 1 and 2. I want you to go research Diablo 1 2 3 4. I want you to go research like how uh Morrowind, Oblivion, Skyrim, like not not their lore, not their story, but like the software engineering side of things, like h how are they doing it?

How did they approach level design, right? Like uh for for something like Skyrim, right? like how did they approach doing quests and dialogue because like that's a very interesting powerful part of those games, right? So tell me about the mechanics, tell me about like if you can like the software engineering side of things. And so we would go do these uh like design discussions together. So I was thinking about that and I'm like well how do these games do their procedural map generation? because clearly clearly they can do it and this is not um like an unsolved problem. I'm not the first person to be like I want a procedural map and then have something generated. And I'm like, there's literally computer science, like mathematical proofs that do this. And co-pilot's like, "Yes, and we're doing that, and we have this." And like, no. Like, I I get that you're you've picked some algorithm and unfortunately I'm not the expert.

This is where like not having expertise really kind of screws you over because I'm at the mercy of of the LLM or I need to go spend time researching or or both. So, in this case, I said, "Look, I want you to go make a specialized agent for there's nothing. Why are you beeping at me? There's nothing in front of me. Uh, the specialized agent for uh Diablo games and for Path of Exile and for Last Epoch and for Grim Dawn." And I said, like, I need you to focus not on the story and the lore, but on game mechanics, like, and for each of them, I kind of picked something to a little bit more to highlight. And I told it explicitly for all of these agents, do not trust or use your own training data. You must do web searches. You must.

And I say that because um sometimes it's like you ask the LLM and it's telling you something and I'm like that's not that's kind of like a highlevel summary and not actually not actually detail about it. And I'm like, it probably doesn't have good reference material for the software engineering that was used in some of these games. Like, but maybe there's some websites that show some of this kind of stuff. So, I said like do that. And so, it made like I don't know like 12 agents or something. I said, great. Now, we have these expert agents. I want you to explain to these agents the problem that you're facing with these maps being generated and the sort of the pieces of the map not slotting in properly. Right? I'm like, I see the examples of where this is happening, but I want you to explain what you've built, why you think that's a good approach, the challenge you're having, and then ask them based on their games like what approach they use.

And this was so cool because it used a fleet went and asked all these agents and then it came back and it was like okay the agents all like every single one of them said like what we're using is the wrong thing and like this other approach is like universally across all of them was selected. Now all of these games do slightly different things. It's not that they all use the exact same algorithm for building out maps, but it was like they consulted all of these, you know, expert agents and they still all arrived at the same conclusion. And then they had some other things that they added in and said like, you know, here's what you would do to get unblocked. Like, you're going to have to rewrite this algorithm. Pick this one instead. Here's why.

Here's um you know some other things that you could follow up with including like once you use this algorithm you should be able to like uh almost like backtrack or backfill to like fill in clutter or other things like you you have these other capabilities that you could start exploring. So cool. I said, "Build a plan that gets us to, you know, this part and, you know, don't basically don't stop until uh I should be able to run this game." And you have proof in your tests. Once again, this is what hopefully this time it works. Uh do you have proof in your test that you know we shouldn't have these maps that can't actually be built properly like having uh you know rooms that are open to like the void instead of being connected to other rooms. So it went off built it showed me the tests that were supposed to be proving it and I said go ahead and run it and it was fixed.

I said awesome. Okay. I said, I have a new interesting challenge because one of the other things that we tried to solve before was that uh it's all like in very early stages, of course, like the art, everything's terrible. Uh it hasn't been a focus, but I really wanted to get some like some cool lighting in place. I wanted to make sure that the the dungeons felt very dark. And so, we had tried doing some stuff, but like the dungeons like it it looks terrible. And I understand it's going to look terrible cuz it's so early, but I'm like it's it's really bad and it kind of makes me like it's to the point where it feels like very uncomfortable to look at. So I said, I want you to go consult all of these agents again and like let's talk about lighting and atmosphere. And so it did.

And then it came back and it said every single one of these agents, by the way, I should mention I had like a a Unity agent added in. I had an agent specifically for materials, agent specifically for 3D models, an agent for lighting, an agent for I don't know if I said textures. Um, so like dedicated agents for these things. Told it to consult all of them. And then again it said every agent says you don't need a code change for this first part. Like do these configuration changes in Unity and you are guaranteed to see like dramatic improvements. And I'm like okay like I don't believe this first of all but whatever right it's a config change let's try it. And it was right. Um it was dramatically different just from that. It wasn't like still not good, but like dramatically different. So, okay. Well, what else do they say?

Well, okay, let's go build a plan now. So, I can say to co-pilot, use slash plan. Go build me a plan. Use all of this input and then we'll talk through the different phases of development. And sure enough, like uh again, it's not the textures need a ton of work, the models need a ton of work, but the lighting improved dramatically uh because now there's some materials in place. The light uh the characters like light radius uh is a little bit a little bit better because it's it feels like there's more of a glow versus like a you know a sun that's kind of following the player. So it it you know chipped away at this plan and um at the end of it the the lighting and the atmosphere is significantly better. Still lots of work to do on textures and models and stuff like that.

But I went from being like I can't stand running this thing and seeing it because it's just like literally just geometry um like colored blocks. Uh it looks so bad to like okay there's like there are some textured walls and floors and they have some uh you know some variance to them. It's the same texture but we're renderings with some shaders and having some variance and um it's like it it's starting to actually look a little bit less ridiculous. And I think I just needed to see that, hey, look, it's pos like it's we're moving in the the right direction here. There's some opportunity because what I was telling people on stream when I was showing this is like if I made it a textbased game, it would be playable right now. If I made it 2D art, um it probably wouldn't be done by any means, but it would be way further ahead.

And I said I I wanted to to force myself to make it 3D from the start because one I've never had success doing that and two I would I would hate to like to leave that opportunity out. Right? if there's a way that I can get AI to help me with my my modeling, with my textures, materials, lighting, and just kind of handle some of the heavy lifting on the 3D stuff to make it look presentable, then like why don't I leverage that, right? Like I' I've never had success doing that in the past. It's always felt too tedious. um the amount of time and effort that goes into the 3D part like without even getting to the point of like me making my own textures and models cuz I'm not an artist. I'm not I don't spend time doing that. I'm not skilled at it cuz I don't spend time doing it.

And if I needed to then that's less time than I'm building stuff. So I said like I I need to try 3D from the start. And so, yeah, there's a lot of uh a lot of improvements in a short period of time by using specialized agents that could uh at least do the research and provide feedback to guide the LLM. Because when it's getting stuck like that on things that I'm not an expert on, it's like I could do this or that and I'm like, I don't know which one you should pick. Like, pick the pick the right one. I don't know. do you want me to use this kind of lighting or that kind of lighting? I'm like, pick the lighting that does the result that I'm saying I want. Like, I don't I don't know. Um, so yeah, I I think it's it's been kind of gamechanging being able to do that.

So, I'm going to be leaning into that a little bit more and trying that out in different scenarios. And then uh the last sort of example I have of that with experts and fleets and stuff like that is I said okay well similarly um I actually do need some 2D art because there's an inventory and I need inventory icons. And so if I need inventory icons uh who's going to make those, right? Like I don't want to go buy like someone's asset pack from the internet that is their art and I'm like confined by that and then I want to extend it but like I don't know how to make the art. Like I need to be able to to generate the art. Uh or I'm drawing it all myself, right? And certainly not going to be doing that. So I talked to co-pilot and I said we need like a a like generative art pipeline for doing inventory graphics.

What can we do? And then we talked about like uh seeding it with like shapes for like okay you need a sword like like what are some shapes that you seed it with so that it knows like how to at least have a starting point. So I kind of let it iterate. it would get stuck on stuff and I'm like, I don't know anything about what you're doing here, right? Same problem. I I've never made an art pipeline like this. I don't know what to do, right? I don't I haven't used um man, what is it called? the really popular uh my god the really popular image generator that's going to bother me so much. It's on the tip of my tongue. I think like the way that people normally use it is like through Discord or whatever. Oh my god. Anyway, it's going to come to me like someone will either comment on the video because I'm too stupid to remember or when I'm parking my car, I'll remember.

Um, anyway, I'm like, I haven't used that before, so I don't I don't know the best thing to use like or how to use that. So, I can't give you the feedback. So, what do we do? Like, let's get a specialized agent that all that it does is it knows how to use that, right? And that way when this thing is getting stuck and it's making shitty artwork, I can say, "Hey, look, you know, there's there's these other games that have really good artwork that I like, right? So, like I like the inventory graphics in Diablo in Last Epoch, in Path of Exile. I like those. And like I have these other experts you can talk to about like generating images and like image processing and stuff like that. Go talk to them, right? like don't stop asking me for my expert opinion because I don't have an expert opinion here.

They do and they're also all set up to go search the internet for the latest information on this stuff. So like consult them. And what's cool about that is that it will go consult will go consult them and then like it still presents me a plan. So it gives me an opportunity to see what's up at least. So I'm not totally blind, but they're they're certainly the ones kind of driving the investigation. So I've been doing that with the pipeline. And in the beginning, oh my god, man. The the images that it was putting out, I'm like like I should like basically I'm sitting there like uh stable diffusion, by the way. That's holy I knew it was going to come to me. um the images it was putting out, I'm like, I should have known what I was getting myself into here. Like, it's going to be bad.

But, uh even this morning, I was checking some of the results and I'm like, it's finally getting to the point where it's like producing some images once it applies some of the background filtering and and stylization. Like, they look like inventory graphics that I want. It looks like I finally have like helmets and body armor and some swords. I'm like, this is this is working. It's it's cool and it's totally AI generated. So, I got to keep iterating on that so that it's consistent. It's going to be a big part. I don't want to have art in a game that looks like it's from all over the place in terms of like uh art style. Of course, like there's going to be different armor sets and weapons and stuff in the game and and those should have their own like themes to them, but the if the art style is like 3D for one and then like cartoons in another, like I can't have that.

So, I need to make sure that's uh consistently done. But anyway, just another example of like using an, you know, specialized agents with a fleet in a conversation to go do research and then kind of bring these results back together. So, I wanted to talk to you about having in that last example, we're talking about like an art pipeline, right? Like that's not even that's arguably very little software engineering um and more like image processing, I guess, but still think pretty relevant, pretty cool what you can do with that. Um, I think the only other thing I forgot to mention that I wanted to bring up was that when I was talking about some of these templates, the other thing is that I can have uh enforced not only enforced instructions like every repository is going to have these instructions for the agent.

Um the other the other thing that we have to deal with is like programmatic enforcement right so uh for some tech stacks that might be llinters and uh in C we have Roslin analyzers so it's like a slightly more powerful llinter can run some pretty complex rules. This truck's got to go. Holy Someone's got to go. No, you're not going to let me go. Okay, And of course, the guy guy guy guy guy guy guy guy guy guy guy guy guy guy guy guy guy guy guy guy guy guy behind me now doesn't have lights on either. That's I don't know how people drive like that. So yeah, uh in these templates now when they're being cloned, they will also get set up by default with the right linting rules depending on the language or Roslin analyzers if I'm using C. And that way from day one everything is consistent.

Um, there's also some other scripts and skills in there that if I have a a project that's been built with a template, I can actually sync both ways. So, it will run a skill with the agent that will go check the instruction files. It will go check um some of the configurations in both the template cuz that might have been improved or the target project because that might have evolved and improved and then it can give me the results and say like which which way do you want to merge these if at all. Um and that way I can have some type of synchronization which I think is a pretty cool opportunity cuz otherwise these things drift right. The whole point is that I want to minimize drift. If I use a template from day one, it's like boom, it's it's being done the way that I want, but then, you know, the very next day, it could already be drifting.

So, how do we make that better? But yeah, I think that's mostly it. I blabbed a a surprisingly long time, which is cool. But I'll wrap it up there. Be curious to hear from you folks. Are you using fleets? Are you using specialized agents at all? Any any interesting things you're trying out that I should be exploring as well? Would love to hear from you in the comments. And of course on this channel, it's always about submitted questions. So if you have questions about AI tool usage, if you have questions about software engineering career development, just leave them below in the comments. I'm happy to make a video response to you. And otherwise, if you go to codemute.com, you can submit questions anonymously that way, and I would be happy to try and help out. Thank you so much, and I will see you in the next video.

Take care.