How To Build An Opinionated Dependency Scanning Framework

Transcript is auto-generated and may contain errors.

Hey folks, I'm just headed to work. Um, I figured I'm going to talk a little bit about code. Um, I don't have topics for pending questions and I didn't see anything on experience devs that I didn't feel like I've already talked about 100 times. So, we'll talk a little bit about code today. Um, I will start this video off by saying for the first time in 13 years, 15 if you include my internships, but for the first time in 13 years, I officially received a promotion, which is super cool. I've never actually in my entire career received a promotion. Uh that doesn't mean that I haven't gained more responsibilities or changed my compensation, that kind of thing. But I have uh never actually formally received a promotion. Uh this morning was the first time ever. So pretty cool. Pretty excited about that. Um for context, that is for me that was it took me five years at Microsoft to go from level 65 to 66.

Um, I feel like uh I'm not I'm not complaining about being promoted. I feel like that took significantly longer than I was expecting. Um, I changed managers three times uh across three organizations. Well, sorry, two organizations, but one of them was reorged in the middle there. So, um I know that when people change managers, the goal is supposed to be that you're not resetting your progress, but I 100% feel like every time I switch managers, I reset my progress. Um which sucks, but is what it is. So, today's a good day. Good way to start the week. Happy about that. Let's talk code, though. Um, I've been uh for Brand Ghost. So, Brand Ghost, for folks that don't know, Brand Ghost is the social media and crossosting platform that I'm building, but it's built in built-in.net. I program basically everything in C. Um, it's my favorite thing to code in.

Uh, I'm just very very comfortable in it. So if I have to build anything then using C is just it's kind of like I don't know a good way to come up with a an example of this but I am so comfortable using C sharp that if I had to go code in some new territory that even if C was absolutely not the good language choice I could still be more effective than learning a new language to do it. So, um it's not that I don't know other languages, it's just like it's uh it's just far more efficient for me to stay with C. Um so anyway, that's what I program in. And over the past little bit, so past few weeks, I guess, especially playing with AI tools and stuff and spinning up some new small applications to play around with, um, I found

myself getting to a point where, you know, if especially if you are a C developer, but I'm sure this is, you know, comes up in other languages as well, but, uh, if you're familiar with the concept of dependency injection and there's dependency injection frameworks, um, this is something that all of these things end up having to use because if they are web services, they're using ASP.NET Core. Um, if you're using uh I wanted to play around with semantic kernel again and semantic kernel doesn't use ASP.NET Core, but it uses the um the service collection building and all of that kind of stuff. So, it's all like wired up to to work with Microsoft's dependency injection framework. And so a couple things going on for me. One is that like over the past few weeks, maybe it's been a couple months now, I've spent a lot of time um transitioning from a dependency injection framework called AutoFac, which is something that I used for years just out of comfort.

And at the time it was far more advanced than Microsoft's. Um but Microsoft's dependency injection framework has since caught up and it's basically the standard. So, it's gotten to the point where I was like kind of fighting against it. So, I switched over to using the builtin stuff. Um, but I'm at this point, especially with a bunch of like little side AI projects and stuff where they're all setting up dependency injection and they're all starting to grow a little bit and I'm like, there's some things that I cannot stand with the Microsoft dependency injection framework. Not because that it like sucks or it has a problem but like sort of the the recommended patterns that everyone follows.

I just do not agree with uh one of them primarily and that pattern is essentially that as you are adding new parts to your application so you're adding a new feature you're building out new parts of the codebase what you need to do to make your stuff work with dependency injection is that every new class you make you have to go add that class onto the dependency container so that it can be automatically resolved. So the reason that this sucks is that every time I write some code, I have to go back to like basically a central spot in my application and add a new line that says go register this thing and I add a lot of stuff. I keep adding all these lines, but this is just like an accepted pattern and I don't understand why. It drives me nuts. Um, and then so people go, "Hey, well, you don't want all this stuff sitting in your entry point.

It's like it's chaotic." Um, you know, you get hundreds or thousands of lines of just registering dependencies. So people group these things up into extension methods and then they can say like collection add cool feature dependencies and that calls an extension method where a bunch of that stuff gets registered. Cool. Um, the problem is that like it just moves the problem. You still have this big list of dependencies. So, you're either adding it to one list or another list, but these files keep growing. And like I said, I just can't stand it. And it's not that it's wrong. And I don't want to I never want to come across in a video like I am telling people that something is, you know, 100% right or 100% wrong. It's just not the way that I program. So I find myself fighting against it.

So especially with these AI tools kind of scaffolding out some of these really simple you know applications that are getting built like if I want to keep building on these I don't like the patterns that are starting to emerge and like I said the one common pattern is like how all these dependencies are being registered on this uh dependency container. So um I said you know what Brand Ghost is mature at this point. Um it's a plugin I build a lot of plug-in based uh applications. So Brand Ghost is entirely a plug-in based architecture. The entry point's very very light. Um and I said I'm sick of it. So I pulled out uh a bunch of the pieces for Brando which is an ASP.NET Core application. I pulled a bunch of that code out into a a new library. And the idea behind that library is that I can sort of have a generic way to go sort of scaffold the dependencies um and new applications I make.

Now, it's very opinionated because it's following my opinion. The goal is not to go make the most robust, you know, do it any way you want. It's it's not it's here's a way that I do it. If you like the way that I code, too, then like you can take advantage of this. So, it's a uh open-source package that's called Neler. Kind of like playing on the dependency injection part. It's a little It's got a little stupid logo that's a little syringe and he's got a happy face. It's It's funny. Um, so I've been building out Needler and like I said, oh my god, buddy. Cannot drive behind people that are that stupid. Holy crap. Um, this guy's hitting the brakes to switch lanes between me and a car. There's no one in front of him. He's trying to get between me and the car in front of me.

And there's no space, but he's like hitting his brakes so that he can merge over. There's not even an exit coming up. Like, there's no there's literally no reason to do that. Um, so Needler basically started off with me copying and pasting a bunch of brand ghost code over um, and then trimming out the pieces that are like very brand ghost specific, which is not a lot, right? Because this is just the part that wires up my lightweight entry point to all of these plugins. So there's only a handful of things that like over time as I'm building brand ghost I'm like ah there's not really a good spot for this like hm I don't have a good plugin or it's going to be a bit of a hack but this is like the easiest spot to make it work and it works perfectly so let me just do it.

Um there's a handful of things like that like 5 to 10 kind of smallalish things. So what I did was built out Needler by copying and pasting that code over starting to remove those things and then while I'm doing that kind of going like is there a way that I can see like as I'm doing this that makes it a little bit more generic where I can see a way to get that thing hooked up. For most of those things the answer was no. But that's okay. So I built this framework out. I put a bunch of little tiny little sample applications together like building a console application and it's cool because like you know you can build an ASP.NET Core application and it's two lines long and you might say well yeah like the sample application like the weather one that comes with uh you know the sample template project it's only a couple of lines long too.

So like hey Nick that's not really much better. Um, but the point is that if you wanted to keep adding more to that weather one, you have to keep adding extra lines to go register the dependencies. On mine, you don't. You just add the new routes. You just add new things and you don't have to go back to the dependency container to register stuff. So, there's a handful of like I don't know like maybe five applications and then had that working. started putting some tests together. Um, once I had the basics in place, I used uh Claude and Co-Pilot a little bit. I'll have some YouTube videos coming out this week on my main channel, which is Dev Leader, that show that. Um, they were done last week, but uh I think my editor uh was he has some family stuff going on, so I think he has like seven videos to edit for me right now, so that's okay.

Um, so I'll have some videos on that coming out this week. But then what I had to do now that I'm like, "Okay, Needler's kind of shaping up like it's it looks like it's working the way I want. I need to go reintegrate it into Brand Ghost, right?" Because all that I did I I told you this already. I just copied the code out of Brand Ghost into Needler. Now that it's been copied out, I need to go basically remove the code from Brand Ghost and and call Needler because Needler has already done the work. So, um, this is where it kind of put things to the test cuz I'm like, okay, I got to make it work. And I missed a ton of stuff. So, I told you like five to 10 things like, yeah, and I basically had to go find a way to make all of those work.

So it was good though because once I had the package in place and I could see the limitations because I'm trying to reintegrate it. Um that's where I could say hey now now that I see both sides like the consumer of it and uh sort of as the package maintainer defining the APIs for it that's where I can say hey I actually have a better idea for how I want to call this because that mindset I find is like super valuable when you're defining APIs. So for example, if you are on the producing side of the API, you might sit there and go, "Oh, like I bet you it would feel good to call this API this way or like these are the parameters I need." So if someone gives me these parameters, like you know, this API will work. But if you put a whole bunch of that together, you might find that when you go to start calling those things, you're like, why does this feel kind of shitty?

like it it just doesn't really flow that well. And um so I I thought it was super helpful to like have the the foundation in place and then see these opportunities to go improve it. So there'd be times where like in Brand Ghost I started calling into Needler to set it up and then I'm like I had to go write like some pretty ugly code even though Needler would support it. I'm like, "This code like kind of sucks." But then I'm like, "Wait a second." Like, I own both sides of this. So, I can go back to Needler. I can add some like fluent builder syntax. Uh, I'm not sure for folks that aren't really familiar with that. It lets you write code that kind of looks like you're chaining some method calls together. So, it starts to read almost like a sentence.

So like make a uh you have a builder that so you make a new builder and then you say like using these pieces using those pieces configured this way without that and it kind of reads like a sentence like I said and then at the end you say build and then it makes you the thing that you wanted. Um, if you're familiar with link inside of C, link is also like fluent syntax where you can say like I have something and I want to select these things where these conditions are met and then I want to order them by something and then I want to call to array on it to make it an array. Um, that link syntax. What is this van doing? Swerving all over. My goodness, that link syntax is like fluent like builder syntax. Um, so Needler has a bunch of that and by reintegrating it into brand ghost, I found some more opportunities to clean that up.

Um, and then yeah, I think at like at the end of the day yesterday, I finally had like the green light on everything. It's kind of scary because there's like 1,500 tests in Brand Ghost. There's a lot of test coverage and when you're switching the entire dependency injection like type scanning system and registration system, it's supposed to be the same, right? That's how it started. But technically, I'm replacing it. I'm like I I'm not hopeful this is all going to pass on the first first try. Um and uh some parts did work uh first try. I think the last gaps I had were um one was ridiculous. So, uh any tests that I had that were starting up the application, not pieces of it, but like actually running the application, it was crashing on startup. And I'm like, I can't understand why. Um, so as I'm digging into it, we have integration with Discord, like the cuz it's a social media posting platform, so we can post messages to Discord.

And it kept saying like this thing from Discord is is trying to be registered. And I'm like, that's not supposed to be registered. And then I go to the code and it it literally has the attribute that I made that says don't register. Like it has that on the class. And I'm like, what the heck's going on here? How do like, you know, thousands of classes all work perfectly and this one class doesn't? Like, what the heck is going on here? And it turns out the most ridiculous coincidence is that I have an attribute that says do not autoregister. Do not autoregister. and Discord has one built in for whatever library I'm using that's called don't autoregister. So I had accidentally replaced it with don't autoregister instead of do not. So uh Needler was trying to automatically register it. So yeah, that was a pretty stupid like one character typo.

Um, and it only affected Discord because that's the only spot in the codebase that has a library that has a very very similar attribute. So once that was out of the way, um, the only other problem was, um, one of those things that I set on my list of like 5 to 10 things that was left over. And what was happening was that the routes of the application were being registered twice. And you can't do that in ASP.NET Core. will throw an exception. So basically right on startup it was like hey you've already registered this route for like the first set of routes it was doing it again. So I just had to add another simple exclusion into the registration and it cleaned everything up. Um, but that, you know, that was surfacing in a really weird way in the tests because it went from being like, you know, not authenticated, but it's originally because the routes weren't even being registered.

Then it was a 500 because now the app is crashing on startup. Um, or it's having an error on startup. It's running, but then the the routes are are throwing exceptions. So anyway, um kind of a bit of a chaotic thing to have 1500 tests and like you know just hope that they all work, but most of them did. Honestly, it was uh the only ones that weren't really working were the the ones that started up the core application. I mean, arguably some of the most important ones, but uh pretty cool. There's architecture tests in place. So they do things like checking that my types uh are registered the way I expect. Um there's some things like the job framework. I use quartz for that. Um quartz is the my preferred job scheduling framework innet. So I have to make sure that those types are registered properly.

Bunch of stuff like that. But these architecture tests, they you can hook them up to your dependency container. They'll scan for types as well and then make sure that things are are organized in a good way. So those all worked pretty quick uh in the beginning, which is great. That's a pretty good indicator for me that like at a high level things are going to work. I'm not saying it's going to be perfect, but if those tests are already failing, then I should expect a lot of really weird side effects in the other tests. We got to get over one more lane. How are we going to do that? Oh, just like that. Okay. Um, so yeah, I'm going to do my live stream tonight. If you're watching this, this will be already probably Wednesday or something when I post this. Maybe Thursday. It's Monday right now.

Um, my live stream tonight, I'm going to code live and we're going to play with Needler a little bit. Um, I didn't over the weekend I I took Friday off. I did a month of on call. I was on call for a month and then I had some other work stuff come up and uh you know my my boss was like, "Hey, like take a day, take two. I might I might do a short week again this week just to catch up a little bit more." Um I have a feeling I'm going to go on call very briefly next week. Um we'll see. Um anyway, I like took Friday off. I was It was just like I'm playing video games on Friday and played video games a lot on Saturday. And um I remember it was like Friday night and I'm like honestly I'm not I'm just like a little bit too burnt out to uh to to write a newsletter.

But I was thinking cuz like it's a kind of a cascading thing, right? It's a chain of events. If I don't write my newsletter, then I don't have my live stream topic. So, when I was committing to not writing a newsletter for this past Saturday, I said, "You got to make sure you have a live stream topic, though." And um I said, "Hey, like we haven't coded live in a little while. Like, let's do that." So, my plan for this live stream, um, I want to do this more, but because we did it on a couple couple live streams ago, I want to start the live stream off asking AI to build us something and then we can revisit it at the end of the stream. Um, so I want to, it's funny because if you're if you watch the live stream tonight and then you watch this video, they'll be out of order, but I want to ask probably use Claude code.

We'll see if Claude code can build a simple uh, rag solution. All that I would like to do is I'm practicing my prompt right now. I want to build a simple rag solution probably using Postgress uh or SQLite to be honest I don't really care um it'll I'll have to check what C packages are available but um let's say Postgress for now because I'm pretty sure there's PG vector nougat package but I want it to use semantic kernel I want it to use Postgress and then I want to build a like a rag engine because I haven't I haven't yet built like locally um like a mini rag framework. So, I wanted to go through and I have a like a wiki, I guess you'd call it. It's just a bunch of uh markdown files for game content. So, for those of you that know, I have this role playing game that I've been putting together for like the past 20 years.

But, uh, a few years ago, I actually started trying to put some content together for the game world. And I haven't touched it in like since 2022 or something. And then I recently just kind of moved the uh repository around and I said, "Hey, like maybe I can get AI to help generate some stories and stuff for me and just try to build out the lore, right? So, um it's kind of like if I don't dedicate time to it, it will never get done. But I might be able to have AI crank out a bunch of stuff and then I can review it and be like, "This story is awesome. This story sucks. Like, make this canon. Make this just cut it out." Whatever. I can have it review timelines and stuff.

So I wanted to use um or build a little rag system that would go through the markdown files and um essentially index them and then that way I can use semantic kernel or whatever like I can write C code that talks to semantic kernel and says like tell me tell me about this game world like tell me something about this period of time and it will have indexed the markdown files and then like the LLM will be able to respond back to me in a way that kind of knows about the game world. And then periodically I'll have to ask it like a good way to do this, but like how do I regenerate the the indices, that kind of stuff. Um, yeah, I'm going to I'll ask Claude to do that. Conceptually, that one's not too hard. I think there's a bunch of like tuning to make that kind of stuff better.

right? Like how how big are the chunks? Like is there other metadata that goes along with the chunks? Um I think honestly that's probably the the trickiest part, but like how much do you clean up the text before you you dump it into the the vectorzation, that kind of stuff. Um so we'll see. But I think the nice part is that I should be able to like rerun it and rebuild the uh the index. So it's not like if I index it once and then I change my mind about how to index it, the whole thing's broken. So we'll have AI start that off and then I'll probably spend the hour on stream with Neler just building some really simple stuff just to show people like, okay, we could do uh WPF, we could do ASP.NET net. ASP.NET's probably the most obvious cuz if we want to add more routes and stuff, I can demonstrate just adding a little new plugin and then not having to wire stuff up to the dependency container.

So, we'll see. I might uh might ask the audience members to what they want to build, but that's the plan. I don't like skipping the newsletter. That's an important thing to me. But uh also my mental health is important too. So I I've done over a hundred issues and I've only taken a couple breaks. One of my breaks was for my wedding. Uh I've written newsletter articles like while I'm in Hawaii and stuff. So yeah, I don't uh I don't take it lightly, but that way when it comes up I'm pretty honest with myself if I need it. And I think it needed it. So that's mostly it. Just getting to work here. Oh, this person is making a left turn from not the turning lane, which is very interesting, but that's okay. And then I get to do hopefully have some good conversations today. I know when my manager was talking to me, he was very excited.

Um, I think as managers when we're able to deliver good news to people that like is a really feel-good moment. Um, you know, we can offer all the guidance and structure and push people in the directions and try to set them up on projects and but at the end of the day it's like it's all of their hard work, right? All of their hard work. And uh it feels good to be able to I don't know like kind of present people something that it's a little bit more quantitative around how much you know their impact has been recognized. So hopefully we'll start the week off on a positive note that way. Thanks for watching folks. If you have questions that you want answered, leave them below in the comments. You can go to codemute.com. You can submit questions anonymously that way. And then I've mentioned a couple of YouTube channels already, but I have Devleer, which is my main channel.

That's where I have C.NET and AI programming tutorials. Check that out. I have the Devleer podcast. That's where the live stream is going to be that I mentioned. But there's also a podcast there where I interview other software engineers. You can hear about their career journeys because there's a lot of really cool stuff that people have done and not every journey looks the same. Which brings us to the last channel, which is Dev Leader Path to Tech. Especially if you're just getting started or you're switching careers, you can check out that channel for resume reviews and I'll have some other content related to getting into the tech industry. Thank you so much for watching. I will see you next time.