Some Of The Stuff I'm Building With AI

Transcript is auto-generated and may contain errors.

Hey folks, I'm just going to drive to the office here. Uh last video was a little short. Sorry. Was uh struggling a bit with finishing up CrossFit and driving home. So uh a little more recovered. Now I'm going to do a little AI recap um just to kind of kick things off now that I'm back home uh after vacation. and then I'll get back into more like uh Reddit threads and stuff like that and submitted questions. So, um lots to catch up on just because had some time off. Um when I was traveling in Africa, I was uh leveraging a little bit of downtime in the evening to be able to do um some I guess like co-pilot submitted work on GitHub. So before I've talked about you know using cloud agents and stuff like that like last year I spent a lot of time de delegating work to cloud agents.

Um this is in my opinion before like a lot of the CLI and terminal agents were um I don't know like operating at a level that I thought was helpful and I just wanted to be able to delegate the work. But this is a a really good example for me where I was literally physically away from my computer. I didn't bring a computer with me to to Africa, but I had my phone. And so, you know, during the travel there's a lot of downtime where like we'd be going from one part in Africa to another. So, it'd be on these little planes. um the you know I'm I'm in the United States so going to Africa coming back from Africa there's like the travels around like 24 hours in total um to go to and from so like there was periods where I'm I'm like

I'm not actually you know doing the exciting part of the trip so I had downtime and so it was really nice to be able to to lean on uh you know submitting stuff to co-pilot to work that way uh I I think something that has helped tremendously is just like putting more effort into targeted instruction files. I've talked about this a lot. So again, if you're not totally familiar with this concept, instead of blasting tons and tons of stuff into your your clawed MD file, your co-pilot instructions file, or your agents MD file, um, instead of doing that, because that top level file is sort of like the the agent context that's always loaded, instead of doing that, I keep that more lightweight and sort of uh high level tell the agent how I want it to operate. And then there's uh basically other instruction files that when the harness is editing files in particular file paths, um those instruction files get loaded up.

So you can have it match on file extensions, sub paths, all sorts of different things. And then I've been filling that out heavily um with a lot more structured guidance. And so what's nice about that is that whether it's, you know, a terminal running on my desktop, whether that's a co-pilot agent in the cloud or um even some of the other developers I work with on things like Brand Ghost if they're using Claude instead of uh you know, C-pilot CLI because like I don't care. I use Copilot CLI because I like it. I have nothing against Claude. I think it's a great product. Um, and if they want to use Claude, then the instruction files that I've made that work with Copilot CLI, I have a little like I'm going to it's a bit of a stretch to call it like a compiler, like a

converter that basically takes the the C-pilot instruction files and like you know, you run one command and it maps them all to a copy to have the clawed uh clawed code ones that work the same way. So, um, yeah, it made it really nice to be able to kick off, you know, tasks to go have an agent build stuff, and I'm like, I know it's going to follow all of the guidance that I've been building up, right? So, that was pretty cool. I was able to get um, couple of small enhancements for brand ghost, couple fixes that way. Um, what else was I delivering some stuff for? I I'm going to talk about some of the other side projects and things I've been working on. Um, but I, you know, I was, uh, wrapping up some pull requests and stuff that way. Um, I had a couple of issues that were filed, like one, for example, I have a an open-source uh, like C client for something uh, called Eve from Verscell.

If you've watched my other videos recently, I've been talking about how we move some brand code over to use Eve. And Eve is just like a like a an agent harness that you can run in the cloud. So if you want to do an agentic workflow, you can kick it off to go run in Eve, it's sandboxed. Um, and then the nice thing is that if you think about if you're using clouded code or copilot CLI or whatever other terminal you use, when you're building skills and you have things that like your agent is running, it it's a pretty nice map from that over to to Eve. So, if you're like, "Hey, cool. I built this skill that works really well how I want and I wish I could just run this in the cloud. Uh it's a pretty good parallel to port over. Um so I have a an open-source C client for Eve and all that it does is map the upstream fixes.

So Versel owns the the JavaScript TypeScript uh client for that. And instead of me trying to, you know, create my own and maintain it, like I don't care. I don't need another um I don't need another side project like that that I necessarily want to go, you know, invest creativity into. So I just have this repository that literally I have a scheduled worker that runs and checks the upstream commits and then it files GitHub issues on my C repo and basically I just try to maintain parody with whatever they're doing. Um, so you know if they add new functionality to their SD like their client SDK uh or you know Eve has done an upgrade so they're enhancing the the their TypeScript client or you know there's bug fixes whatever I just try to maintain parity and so those issues get filed on my repository and then I just have co-pilot go uh go to work on them.

So, I was able to catch up on on some of those uh while I was gone because I was gone for two weeks, you know, it's not like the Versel dev stop. So, it was nice just to be able to like keep that going and not have a a huge backlog when I got back. So, yeah. Uh while I was gone, it's not like, you know, all the coding stopped. It was kind of nice to have that um capability. Um, I think while I was gone, I don't know why this happened, but um, while I was gone, I was thinking about um, some video game development stuff. So, for those of you that don't know, like one of the things that that got me into programming, you know, 20 plus years ago, which is, I think, actually the case for for many developers, is like I was like, I want to make my own games, right?

Like, how cool is that? So, um, you know, I' I've never I've never truly finished a building a game because I just like building it. I don't necessarily care to to play it. So, um, because I had a lot of downtime when I was traveling, I had a bunch of time to think about some stuff. So, um, there are two games and I'm going to talk about both of them. Um, one of them is, you know, a game that I've worked on literally since, like, you know, was one of the first things I ever started programming. And if you've watched my other videos or seen my other content, I always uh bring this one up where basically it was like a textbased role playing game. At one point, it was literally just like uh, you know, a windform application.

you would click a button to attack a skeleton and then I had other monsters that you could attack and get gold and buy like items from a shop and like literally there there was no there was no movement or animation or anything, right? It was just like uh really a glorified calculator if anything. It's kind of funny. And so over the years, I just keep extending this game and I've had it where you can like walk around in 2D almost like Pokemon, uh, do like turn-based battles and stuff. And, um, more recently, I just had co-pilot basically start building this thing from scratch again. So, the all the code is is vibe coded now. Um, and it's been fun because I get to talk about, you know, the the RPG systems that I want to have built and it goes and builds them for me. And that's the enjoyable part for me is like coming up with these creative systems.

So, um, in my downtime when I was traveling, one of the things that's been bothering me is like, you know, I I know that I'm not going to build some role playing game that's like, you know, the next I don't know if people watching this play role playing games, but it's not going to be the next Diablo or Path of Exile or Skyrim. Like, I understand that. Um, but you know, one of the things that I wanted to be able to do was like I I like the idea of, you know, investing time into building a character for a role playing game. And so before, like when I was younger playing role playing games, I remember playing Diablo II and you know, putting a ton of time into getting a character up to uh if you're not familiar with the game, the max level was 99.

Um, and I never had a character that reached 99, but I did have a character that that reached uh level 98, and I just I never did it. And at that time there was no like shortcut right there. Like you know in Diablo I 4 now like a new season starts and there are people that are like you know max level within a couple of hours and they go through their paragon levels and like you know another day or something ridiculous. We just we didn't have that. So part of the part of it was really just like grinding to get that like that max level. And so I I kind of miss that. I kind of miss investing a ton of time into a character. But I get it, too.

Like, you know, if you do that and then there's a new, you know, new season for a role playing game or whatever and everything restarts, it's like a character's kind of it's almost like it's not it's no longer meant to survive a long time, right? It's like that's just not how games work now. And so my thinking is like if I'm playing a role playing game, I do want to invest time and I want my character to be something that perpetuates, if that makes sense. I I don't want it to be, say, if you're playing Diablo or Path of Exile and New Season, like yeah, your character sticks around, but like you would never use it again. like you're only going to play the new season with a new character. And so when I'm building my role playing game, my thinking was like a couple different things.

One, I want my characters to to mean something after I'm done playing with them. So they they should be part of, you know, if I'm playing a new character or doing another playthrough or whatever, I want my character to be like previous characters to a progression to mean something. Um, and then the other thing is like I like this idea of I don't necessarily just want to have to be at my computer as the only way that I can like sort of spend time in my my role playing game. And so, um, you know, that doesn't just mean cool, okay? Like make it work on, you know, a console like a portable device or your phone. Um, I don't just want to to play, you know, a game made for a computer on another device, but I do want to be able to interact with my game somehow.

So, I've had this idea of like, well, in role playing games, there's crafting systems and stuff like that, right? Okay. So, what if I could have a companion app that when I'm not at my computer and I wanted to spend time playing my game, instead of being able to walk around and like, you know, kill enemies and stuff like that, um, and like playing the full game like on my phone, instead of doing that, what if I had a companion app where I could use my materials and items and basically just have like a crafting companion app? Um, and I really like the idea of, you know, some games have done this, like Crimson Desert, if you've played this, where you have an encampment and you can send off NPCs to go do quests, right? Like there's a bit of a passive mechanic. While you're playing the game, they're going off and doing some other work for you.

I loved this in Final Fantasy Tactics. You could send off some of your team members to go do stuff. So again, like what if I have a companion app that lets me go take um you know, manage my my party uh or my guild, whatever it is, and send off NPCs to go do work. So I had this sort of realization while I was traveling like how I wanted to start structuring some of this. Now, beyond just what I said, there was a couple of other pieces to this that came up which were essentially like um I building role playing games like I've done textbased ones. I've done like you know turnbased textbased uh so you kind of have like Pokemon style battles. I've done a little bit of like real time action, but because I suck at, you know, doing uh art assets and stuff like that, I never really spend a lot of time.

So, when I'm building it now, I I do want it to be animated. I do want it to be, you know, liveaction, but there's something about turnbased games that are really cool. Um, there's something about tactical turn-based games that are really cool. If you've played Balders's Gate uh 3 or you've played uh the Larian Studios games for uh Divinity, uh you've you probably have seen or experienced they have a combination of live action and then you can turn the game into turnbased. And so I think this idea is super cool. And so instead of trying to, you know, to remake that kind of thing, my thinking was like, well, if I'm already going down this path of like companion apps, what if if you're playing my game normally on a computer, you play it like a isometric, you know, uh, role playing game, action role

playing game, and then you can have a companion app that you could play on your phone, uh, or, you but whether it's in a web browser on your phone or whatever that's basically like tactical turnbased and and that way you could like take your party of characters or your guild, your guild members, and you could do like tactical turn-based combat. And really the idea is that you still get to invest time into building up your game world, but you get to play a different way. And so I thought for me I thought that would be really cool. So I started writing down a bunch of ideas for like how I want to start creating some of these systems. Um so that was one thing that I was kind of thinking about in my downtime. The other uh is there was a game I don't I I really don't know why this one came up for me but there was a game I used to play when I was a kid.

My sister and I both really like this game. The game was called Creatures. And I'm not sure if anyone knows what this game is, but um this was like I think the easiest way to explain this is for very casual players, it was kind of like having a Tamagotchi if anyone remembers what that is. You had these characters that were essentially they looked like a almost like a Furby if anyone remembers those. I don't know how much I'm dating myself here, but um they were called Norns and you'd have these little characters and basically it was like you would you were their babysitter and their caregiver, right? If you know what a Tamagotchi is, you feed it, you play with it, and that was it. But um there was a lot of really cool stuff in this game that if you went looking for it or tried to understand it, like there was a lot of computer science and biology that was put into this game.

So if you're a casual player, you may have never even noticed it, right? You're a kid, you got some, you know, some furry animals that you're babysitting and that's, you know, you're playing that game. But this game had biology and genetics like mapped into it. So these creatures that you were babysitting, they knew how to interact with the world. Like they would learn based on their genetics how to interact with the world. Um you could breed them. So when they'd have offspring, literally genetic traits would um get altered and you'd have these offspring. And so like you could literally breed these things to have different genetic tendencies. And I again I don't know why I was thinking about this but I was like man like that's such a fascinating thing that they put into this game so many years ago. So many years ago. And they had like a whole you know genomics kind of system.

I was like, that's so underappreciated because it feels like the casual player was just like, I'm babysitting some fuzzy things. And so I was thinking like, I actually want to to remake a game like this. Um, so I spent a bunch of time actually with chat GBT. Um, basically just going through a a a whole game design that I want to do uh for essentially like a you know, my take on what a game like this would look like. Um, so I I basically sat with chat GPT and just went through like all of these different game mechanics that I want to be able to introduce. Um, I want to have, you know, um, progression. So, the way that this worked is that, and let me kind of explain like my my thinking here is that if you're I want to be able to support just super casual players.

Like, if you just like the idea of babysitting a couple of, you know, fuzzy things, great. Like, you can just play the game that way. And it has some puzzle mechanics. So if you want to get more functionality, right, you unlock more functionality by basically uh you as a player solve some puzzle mechanics. And the more complex things actually require that your creatures, the things that you're babysitting, um they have to actually solve the puzzle mechanics. And so that might mean that you have to breed some some creatures that can solve those puzzle mechanics. You might have to find ways to incentivize them to go solve the puzzle mechanics. And as you do that, you unlock more things. And so there would be this play through where, you know, if you were a little bit more of a serious player, you'd have a bunch of things to unlock and you could have more interesting creatures.

And so sort of designing that system and then if you're like really really into it there would be a whole system on top of that where instead of doing these unlocks uh you know you would do that of course then you could also explore planets and so I don't mean like fullon 3D I'm going to go procedurally generate like you know an entire universe with super in-depth planets, but there'd be these procedurally generated planets that you could have you could show up at and basically interact with the other creatures there because the whole idea of this is like is genetics, right? So, you might see other creatures and you're like, "Wow, like that creature has um like scales, right? Like my creatures are all fuzzy. I want to have scaled creatures. So, like how do I get that creature to bring on? and I can use the genetics for that.

Um, I don't know if people remember a game called Spore. Like I remember when Spore was being developed and I was like this is going to be the coolest game like has so much potential and I remember playing it when it came out and I was just so disappointed. Um, sorry if you're the who who made that? Was it like a Maxis game or something? Sidmer? I don't know. I can't remember. I was I was just so disappointed with how that game worked, but I thought the concept was going to be so cool that you could take the genetics of other things and then sort of make your own life form. In practice, it felt like it was so nerfed in terms of the combinations of things. I'm like, why like why even bother? Um it felt terrible. So anyway, um what I want to be able to do is like this idea of if you're really into it, you can go explore planets.

you can go get genetic material from, you know, creatures and life forms on these planets, bring it back, and then um whether it's through breeding, other more advanced like genetic um manipulation, you have new creatures that you can create and continue on because there's different habitats and biomes. Like you might need different types of creatures to be able to go explore those. So it might be such that you know you go to a spot and you happen to identify a creature that can breathe underwater or has better lung capacity kind of thing. And then then you're able to go to a planet that's all water because you've bred a creature that can breathe longer and there's creatures on that planet that don't even need air. They just permanently can breathe underwater. So then you get some of their genetics. this kind of concept.

And then finally, um, if you're like just completely nutty about it and you're like, "This is the coolest thing." What I want to be able to do is have, you know, the final endgame sort of unlock is that as you have all this genetic material, um, you'd essentially be able to like in-game edit the genetic code because it's literally going to be like code. So, you'd be able to edit the genetic structure. Um, and that means that you'd be able to create genes for things and like when you go turn it on, maybe it doesn't work at all, right? Like maybe you've created an incompatible uh set of genes and so like whatever creature you try to like create out of that that genome like just simply, you know, doesn't doesn't survive like instantly. It's uh incompatible. doesn't get born. Um, and then you might have successes where the creature lives and maybe it doesn't have the right effects that you were looking for.

But basically giving you full control in the very end game to be able to edit all of the genetics of a creature that you want to create. Um, so anyway, I don't know why I was thinking about this, but I when I had downtime, I was like, I just think this would be such a cool game to go recreate. And because the graphics and art assets don't have to be that insane, it's really about like complex systems and then some puzzles. I was like, I think that I can I can build this. So, um, that's another side project that I'll be I'll be working on. So, I'll share more about that if it goes anywhere, but um seemed pretty exciting. Otherwise, um I know I said this was going to be about AI, but those are more of just the side projects I'm I'm looking at.

I still have uh a handful of things that are being developed. So, um I have I don't know if I've shared the details on some of these things, but I might as well. So like um one of the side projects that I have so for those of you that don't know uh my prior life before Microsoft was working in digital forensics. So, um I am extending a a picture and video recovery tool that I made to help a friend out to basically build a digital forensics platform because um because I can because I know how to build that and I think I can do a pretty good job. So, I'm building that on the side. I am building an agentto agent communication platform. So the use case for me is like if I'm running co-pilot sessions, I want to make sure that I give you an example.

if I have like a front-end dev session and a back-end dev session because they're working in different repositories. Um my experience has been that yes you know I can have one agent session you know working across two repositories but it almost seems like kind of like how it might with humans where if I'm Nick and I'm the backend dev and I just work in the back end if I was like I know I could change some stuff in the front end like I'm just going to go I'm going to go do that. It seems like in my experience the the cross repo kind of interactions there. It's like the developer who isn't actively working in that spot seems to make bad decisions like didn't totally get the architectural patterns whatever. Um, or I've I've seen this where, you know, I'll have a a front-end agent who's like, I want to build, you know, this feature in the front end and like I'm going to make a a handoff document to give to a back-end developer.

And it's like it's prescribing what the database storage and should be like. And it's like, dude, that is literally not your responsibility. And like you don't know, you don't know what the database storage looks like and you shouldn't care. Like why why would you care? That has nothing to do with your part of the system. And so what happens is these handoffs get created and then if you just blindly give that to an agent, they're now implementing something that may not make literally any sense architecturally in um you know in that part of the codebase because they're just blindly following some spec that some agent made that also doesn't have any idea what the hell's going on. So, um, what's been interesting is if I run sessions like this, if I act as the man in the middle and I'm like, "Hey, like you want to hand this off to the backend developer?

No problem." I'm going to have them review it. And so then I would, you know, copy paste some stuff over to another session. I'm like, "Hey, review this, give feedback." And I'm literally now just like sitting in between two agents chitchatting. And it's ridiculous. like what am I doing? I'm just copying and pasting text back and forth. So, my idea was like what if I just make um a system that is basically an extension to co-pilot and this will work with other agents, but you can have secure communication channels. Like, I don't want to be copying and pasting back and forth between the agents. Like, let them just talk, right? So, I could set up a chat basically with uh two agents that are working in these different areas and if they need to do handoffs and get, you know, reviews and feedback, then they can.

Now, I realize that some of you might be saying, "Well, that sounds stupid. Why don't you just have like a sub agent do a code review or whatever else?" And like, yes, that can totally work, but I'm just trying this out. Um because like I said, it's worked really well if I manually copy and paste the handoff and facilitate the chat. It's worked really really well, but I don't want to be the one doing that. So, I'm hoping that I can try this out and uh and see what it's going to look like. So, and that's being built in rust. Oh my goodness, rust. The best part is I haven't seen the code. Um I think Rust code looks terrible personally. I know everyone loves Rust. So sorry if I hurt anyone's feelings, but um when I read Rust code, I want to die and that's my opinion and I'm allowed to have my own opinion.

So anyway, that's what's going on. Thanks for watching. Um we'll get back into questions and stuff like that in the next video. So if you got questions, go to codemute.com. You can submit stuff anonymously. I think these videos are going to look pretty good based on on this video.