Dependency Injection, Semantic Kernel, and Upcoming Coding Projects

Transcript is auto-generated and may contain errors.

Hey folks, I am just driving to work here. Um, it's Wednesday morning. I'm going to talk a little bit about we're going to talk about code um and some stuff that I'll be having more tutorials and kind of things on coming up on my main channel, Dev Leader. So, just to give you a little bit of insight so you can see kind of what I'm playing around with and then if you're interested, you can check out those tutorials. So um so recently I released a like an opinionated dependency registration kind of framework. Um the concepts like aren't like I didn't like invent them or anything. It's borrowing a lot of things from two packages screwor and uh autofac. It really combines like how I like to build plug-in systems, I guess, primarily. So, the the idea being that we can keep our entry points to our programs very very lightweight.

Uh I on the live stream that I did the other day, I demonstrated that we can get down to like you know two lines of code or one line even for web application and then you can add functionality in without having to go back to the entry point uh or you know tacking more things onto extension methods to to register your your dependencies. And so this was sort of uh I don't know the the product of of Brand Ghost, right? So Brand Ghost is this social media content uh scheduling and cross-osting platform that I'm building and I use this for all of my social media content. And so Brand Ghost is at a mature enough point now where like I have I've proven the pattern. It's something that I use uh over time in other applications as well, but I like especially with the stuff I build, I end up repeating patterns many many times.

And I kind of got to the point with brand ghost where I was like, look, especially with some of these AI projects I'm doing on the side, I really need something that's going to um be a common ground for getting my dependencies wired up because I keep getting these things built like these little AI programs that I'm dabbling with and then I have to go overhaul them because the dependency injection I just I can't live with it. So Neler was this package I built. That is what Brand Ghost uses for dependency injection and now I have that available so that I can start building other things that use it, but that's not really new. I kind of talked about this already in some previous videos. So, I just wanted to give you a heads up that I have a handful of videos coming out talking about needler.

Um, one of them that will be coming up is that I have this um, well, this next little project I'm going to talk through, I have it uh, going from what AI created. So, it's like this 200line entry point. And of course, you don't need needler to reduce this entry point, but it's just a good example of like it's it's pretty nasty. There's a lot of code there. Oh, come on, buddy. Turn faster. I'm going to be stuck behind you. Um, and I get it down to like two lines of code in the video and then convert everything to being uh dynamically registered plugins. So, there's a handful of videos coming out on Needler and I'm going to continue trying to showcase uh the usage of it. Not because my goal is to get you to use Needler. Like I actually I mean totally transparent. I don't care.

Uh I use it. It's opinionated. It's the way that I do things. So, in general, when I'm talking to to you and other people about things like my my own experiences, my goal is not to like convince you to do it my way. That's not the goal. My goal is just to share with you my experience and then you have more information and you can make decisions. Um, like I mean this honestly. If you listen to the things I say and you're like, "Wow, like that's the dumbest thing. I would never do that. At least you have more information to go make decisions. And if that still results in you being like, "Nope, I don't agree with that." That's fine. That's totally cool. Um, and I I mean that again with uh with Needler. Um, so what I'm trying to do is use that to to show how you can build plug-in architectures.

And if it's a good fit for you, great. If uh plug-in architecture is a good fit for you, but you don't like how Needler does it, totally cool, too. And if plug-in architecture never seems like it's a good fit for you, also totally fine. It's just, you know, more information for you to be armed with, and you get to see it through a particular lens. So, I'm going to make more videos showcasing plug-in architectures, uh, scanning for things, and then, uh, you can you can do with that what you will with that. Uh but um this leads me to the next project um that I'm going to be spending more time on and it's uh semantic kernel based. So the project that I put together this itself doesn't need like tons and tons of work but it's going to be a cool opportunity for me to play with semantic kernel more.

And what we built this actually comes from the live stream that I did on Monday. It's Wednesday right now, so a couple days ago, um, at the beginning of the stream, I basically gave a a PRD, like a spec from chat GPT to Claude, and I said, "Go build this." And what it is is a rag system for uh a bunch of markdown files. And the markdown files are basically a wiki for uh like game content. So, uh, for those of you that have watched, uh, enough code commute videos, uh, I have mentioned many times that I have a video game that like was like one of the first things I started programming like over 20 years ago. And, uh, it's like it's something that'll never get finished, but it's like a a bit of a playground for building stuff. And here's just another good example.

like I can use this role playing game as a mechanism to go play with semantic kernel, right? It's the purpose that it serves is for learning, not necessarily that it's going to be a uh complete game. Uh maybe if I retire someday that'll happen. But um what it does is this this tool will basically scan through recursively the directory structure for all these markdown files which form the wiki for all of this game lore. So over the many years when I play with this this code base is someone honking? Oh, it's the other side of the road. um when I play with this code base um I just keep building features and functionality and then I like rewrite it and whatever.

Uh, but in 2022, I was like, "Hey, like for some of the stuff I'm building, I actually need to like start putting together some of the game world." And I realized like I have spent all of these years kind of building systems in the game cuz they're fun for me to build, but never actually describing like what what does this world look like, right? It's supposed to be a fantasy role playing game. So things like, you know, Diablo, like it's kind of inspired by by that. Um or, you know, like any Bethesda games, like, you know, just fantasy role playing games. And I'm like, I haven't actually written down anything that describes what this game world is like. So in 2022, I started just trying to do that. And um I think I was like kind of winding down because it comes in comes and goes, right?

like in cycles where I spend some time on this and I was winding down and I remember like this is when chat GPT was just kind of coming out when I started winding down and I remember thinking like man if this thing could do a better job then I wonder if I could have my my game world like the lore for it get written by AI and I don't mean exclusively but because it's a ton of writing and story telling ing like it would be great if this thing can kick off a lot of that work for me and then I can review it, I can curate it and build the game world sort of a little bit more passively, right? Cuz if I have to sit down and just like dedicate time to writing lore for a game world, like I got I got other things to do.

But if uh you know, if I could spend a few minutes here and there reviewing stories and stuff like that, I think that would be kind of fun. But that's going back a couple years and only recently did I convert this code base over uh like this wiki sorry over to markdown files. So this lorebot is actually a semantic kernel powered tool that will use rag and uh so retrieval augmented generation. It will go do embeddings on chunked con oops I spat everywhere. embeddings on chunked content for the the wiki information. It then stores that in the uh the vector columns in Postgress and then from there we have some APIs in semantic kernel and in um in ASP.NET net to to basically query and ask the lorebot questions about the lore. And it's pretty cool. It works. But the idea is that, you know, I got to convert this whole thing from whatever whatever Claude decided to put together that was pretty nasty into uh like a needlerbased plug-in system.

So, I started doing that. The codebase, I should mention early enough in this video that uh the codebase is like it's totally public. So, I pushed that up this morning. It's on my GitHub, which I think is just Nazantino if you want to go check it out. And um so, it has this lorebot there, and I just split it up into like a core library, a web app entry point, and then Oh, there's a cop behind me. It's awkward. Um, and then a console application, but the console application's not done at all. So, um, what I'll be doing is building out features in this and this is going to let me explore semantic kernel. So, I wanted to share with you because I wanted to spend more time with Semantic Colonel and I haven't really sort of carved out the time to do so, but now I have a bit of a a playground to kind of go do that with.

So, I'm kind of excited to to go building tutorials. Um, I really want to explore the dependency injection around semantic kernel because it's a little bit unique and I'm kind of learning more about this. So, still very early for me. I got to be very transparent about that, right? Like I am not a semantic kernel expert by any means. And um it's interesting to me because it seems like the semantic kernel itself when you make an instance of it uh they actually really want it to be like transient or scoped per request if you're doing like a web API. And this is kind of confusing to me because um I would imagine that building and I don't know the details, but I would imagine building a semantic kernel instance because you have to configure it with plugins and all this stuff. I would just imagine that's like a a heavyweight operation, right?

Like that's my assumption is that it's a heavyweight operation. So if you have to go do this per request, right? request comes in on the API, you have to go build a kernel, then use the kernel to go answer questions. I'm like, that seems like that's pretty heavy-handed, and like surely there's a better way, but like I don't I didn't build this, so I don't know. I trust the people that did, so it's probably just my lack of understanding. So, I'm really trying to understand um sort of the the life cycle of of semantic kernel. Oh, man. Sorry. There's a there's a cop driving behind me and we just passed a guy who's pulled over on the side of the road, but his entire like back end of his car is like completely smashed up. So, I was fully expecting that this police officer was going to stop for this guy and just No.

Anyway, I guess we're ignoring the uh massive crash scene, but that's cool. So, um, the thing that I was playing with this morning in particular was, um, again, if you're not familiar with semantic kernel, maybe this stuff isn't making sense, but I will try to explain it, uh, kind of how I perceive it and why I think this is kind of a a pretty fascinating thing to explore. So um semantic kernel itself has a lot of like APIs on it that are very much like a web application builder for ASP.NET. So when you're adding semantic kernel onto a web application, it almost feels like you're adding an entire web application onto a web application in terms of the how the dependency injection works. like it's really weird. Semantic kernel has its own dependency container, right? And so does the web application that you're going to be attaching semantic kernel to.

But also I mentioned that we don't it's like not recommended to have a singleton of the kernel. You want to recreate them. So it's this really bizarre situation where we build our uh web application builder once we build the web application once and then we have this kernel that we need to keep recreating per request but also I am really interested in having that kernel share the dependencies of the core application. So how do I in a efficient way make sure that I can always provide those dependencies into uh the semantic kernel plugins. So um the pattern that I was exploring this morning and I haven't benchmarked this. I haven't uh really spent time exploring what my alternatives are. This is kind of my first approach with it.

But um in C and the dependency uh containers so it's called like a I service collection we have these service descriptors and the service descriptors sort of define how um services will be resolved and created and we can actually like copy those descriptors from one collection to another. So you're basically saying however this service collection says it will register these things, let's just copy how we say we will register those things to this other collection. And uh so I'm exploring a pattern where I'm copying over those service descriptors from the root service provider. So, the one that comes from the web application, I'm copying those over to the semantic kernel builder. And this feels kind of gross to me because I've never I've personally never had like a use case for doing that. Um, so I kind of feel like I'm, you know, breaking some rules or doing something I shouldn't be doing, but so far I'm pretty sure it works.

And like it's just like one for loop that copies the descriptors over from one collection to another. So I'm uh I don't know. I I'm kind of on the fence about it because it kind of looks like it works. It seems like that like might be allowed or okay or else why would they let you do it? But I still don't know if I'm like violating some, you know, uh some best practices or something like that. I I just don't yet know how to sort of create a I don't know if you could do like a scope of the uh dependency provider itself and then create a new collection from that to go make a new provider from. Like it's just really weird. So what I'm going to be doing is um iterating on all of this, right? So, if everything I just said you're like, "Dude, I have no idea what you're talking about." Like, welcome to the club.

And uh I'm I'm just looking forward to making more videos where we can try to add more kernel plugins. We can see how that looks with like how I traditionally do plugins. Like, can I make them work in a similar way? kind of want to let this this cop pass me because then it won't feel like he's We got to be quiet as he's passing. I'm He's like tailgating the next car. So, he was really trying to get around me, but very interesting. It's got to be a cop. It's got like everything's tinted. It's got the cop wheels. Interesting. Maybe it's not. Maybe it's not even a cop. Um I should have looked a little closer. But anyway, um now that I can think straight cuz this guy's not tailgating me. Uh I think a couple things I'm going to explore are going to be one Oh, buddy.

You can't do that. Come on, man. I hope that cop pulls you over. This guy's in a huge van. You can't see the road, obviously. Um, we get these double solid lines and you're not supposed to cross over them because you're basically going into the the paid lanes. And so it's a it's like I don't know what the fine is. It's like a few hundred if you cross them. This guy literally just did it in front of me and the cop is still right in front of me and he just like swerved in to go past someone and then go back. So he crossed twice. But anyway, not my problem. Apparently not his problem either cuz the cop didn't do anything. But um yeah, we're going to look at uh number one, seeing if there's like is this pattern that I just talked about, is that going to be a mechanism that works that's efficient?

Um what are the pros and cons of that? So I'm going to explore that a little bit more as I add in more semantic kernel uh functions and plugins. And what I'm hoping to do is see that evolve and then I'll reinccorporate that back into Needler. So if you're seeing what I'm building and you're like, hey, I want to build in semantic kernel 2. Hey, I don't want to have to go do all this stuff manually, then hopefully Needler will give you some of those options. And again, if you don't like how I'm doing it, then at least you can learn about it and then you can come up with your own patterns uh that that you enjoy doing for setting up your dependencies. So that's one thing that we're going to explore.

Um the next is that right now I had this whole thing gets set up like I said by claude and it's using Azure Open AAI and that works great because I have Azure Open AAI access but I also want to go port all of this to run off of a local LLM and I have not yet done that. So I have not run my own local LLM. I've never set up C code to do it. Um, I think Hassan Hhabib has been posting a bunch of tutorials and guidelines for how to do that. So, I'm going to go check out some of his content to learn more and then uh once I get familiar and comfortable with it, I'm going to put some tutorials together and uh kind of share my experience on that. So, we'll go from having this semantic kernel setup with Azure Open AI to running your own LLM.

Uh, it's probably, as far as I know, like Olasharp is like the way to go. There's like even Microsoft learn articles on it. So, um, if you're interested, and I haven't made my videos yet, like definitely go check that out. But uh I'm I'm very interested in that because like I said, I haven't spent much time with Semantic Kernel as much as I'd like to and I've never run my own local LLM and I I'm just really curious to see what that's going to be like. Um I think for some people, you know, motivations and and stuff are different for everyone. And I know some people are like very much of the mindset like, "Oh, I don't want to share stuff with uh tech companies or if I'm using some, you know, third party LLM, like I'm at their mercy." Like I I guess I have less of a concern about that personally.

Um I don't know like just where how I view things like it's just it's just not really a a concern. I guess I'm not dismissing sort of the risks or the, you know, the potential to be locked in or the potential to be locked out or the potential for my information to be used for training. Like I see that. Um I guess like I just don't really care that much to be honest. But what is cool to me is like if I can run a local LLM, if it's performing, you know, not like a ridiculous um discrepancy where it's like, you know, 10 minutes to go answer something or the accuracy is so that like I might as well not use it. If it's performing reasonably well, like I think that could be super interesting just as an alternative, right? Like I paid 200 bucks a month for the the Clawude Code Max plan or whatever.

I don't know. Like for me that's worth it because of how much I use it, I'm okay to pay that. But if this thing can run locally and it does a comparable job, like I don't know. Then yeah, maybe I can save some money. That's cool. I'm not going to I'm not going to say no to that. But my motivation is perhaps different than other people because I think some people are just very concerned about, you know, having this reliance on on service providers, which is totally fair, just not sort of the same concerns I have. Um, I think one more thing to call out with this video series that I'll be doing is that or I guess two more things.

one is that I want to explore a little bit more how again with dependency injection and registering services and things like that how we do this kind of thing outside of just web applications because I think we see so many examples where um you know you're building a web app and how do we hook these things up and it's like it's just the same thing over and over which is actually very convenient if you're building a lot of web apps that when you move away from that, like if you wanted to build a command line tool, which I'm going to be doing with the semantic kernel thing and now you want to register dependencies, like should that look dramatically different? Like I would hope it doesn't, but if you've never done it before because all you've done is web applications, then that might be kind of weird.

So we're going to explore that and see like you know how can we make common ground and then you kind of layer on the web application layer on the console application you could layer on a desktop application um and then like what do we need to do with those dependencies so we'll explore that a little bit and then the last part that I wanted to mention is like uh like this lowerbot it does rag right like so it is chunking things up it's running the embeddings It's then doing uh like a semantic search by using uh vectors. So it does an embedding on the thing that you're asking about and then it does the calculation to figure out like the uh I want to call it like the distance score but like sort of the the confidence level for the how related the chunk is to the thing that you're you're inquiring about.

So it does all of this. It works. But this is the first rag system that I built and and AI put it together for me. So, kind of channeling some of the other things that we talk about when it comes to learning with AI is like, yeah, it's awesome. It built it, but I know some of the concepts, but I have no idea what it's doing right now. No idea. High level. Yes, I understand chunking. I understand some of the overlap in the chunks that you're getting. I understand what it means to go give the LLM the content to generate the embeddings. I know what it means to store stuff in the database. Like I know how to query a database, but there's some details in there that I'm like, I don't know what that's doing and or I don't understand it enough to know why it's making that decision and I want to spend more time on that.

Right? So, for example, why is the chunk size 400 characters or 400 tokens? I don't know how it's measuring that actually, but why is it that why is an overlap of 75 characters or 75 tokens good? Like what happens if that was 50? What happens if that was a,000? Like I don't know these details. So I want to understand um like how this works more effectively because right now it's a bit of a black box to me. So, we'll spend some more time on that. Again, like my goal with some of these videos is not to to try and show you like look, I'm pretending to be an expert. I'm like, hey, like this is me exploring this thing. And if uh if you're interested in learning alongside me, great. So, the dependency injection and stuff in general, I feel much better about semantic kernel for that's very new to me.

Um, but yeah, hopefully something for everyone. So, that's going to be a good chunk of the videos that I have coming up on my main channel, which is Dev Leader. So, if you're interested in learning about that, if you're a .NET or C developer, uh, definitely check that out. I recorded three videos yesterday. Um, my editor's a little bit behind. He had a hit a really big move with his family. So, uh, just trying to make sure he has all the time and space he needs to kind of take care of life cuz what's more important is that for him obviously. And sure, I would love my videos edited, but I would much rather that he's like, you know, he's just got his his stuff all together and he feels good to go. I don't want to stress him out with anything. So, um, I'm going to keep making the content and then when he's ready, I'll I'll have a lot to upload, which is fine.

Um, but if you haven't checked out the main channel, that's all, uh, C andnet tutorials. Uh, there's some AI programming tutorials there as well. So, that's just Dev Leader. And then worth mentioning cuz I have a couple other channels. I mentioned that I have a live stream and I did that this past Monday. My live stream is ideally every Monday at 700 p.m. Pacific. And generally it's not live coding. Usually it's based on a code commute topic. So if you enjoy these videos, usually every Monday I will pick a topic from the previous week and then do a live stream about it. And then that way because it's an AMA format, if you wanted to share your thoughts or ask questions, you're absolutely welcome to. that is done at the Devleer podcast. I think YouTube it's just @devleer podcast. So you can check that out. It's also a podcast channel.

So I interview other software engineers and you can hear about their career journeys and some of the things that they are focused on in their careers that uh you know uh are a big part of kind of what I might say is like their developer identity. So um like I I mentioned Hassan Hhabib already in this video. If you don't know who he is, he is a uh software engineer and content creator. Um really I think super awesome like philosophies on on software engineering. So interviewed him. I've interviewed Scott Hansselman if you know who that is from sort of the net community. Uh he's one of the VPs at Microsoft. He's super cool. Um I just interviewed Derek Co Martin of Code Opinion. So that'll go up soon. But um yeah, there's like it's for me it's super cool to get to talk to these people and sometimes there's people that aren't like they're not popular on like social media or anything, but so much to learn about people's different career journeys.

So again, that's just the Devleer podcast on YouTube. And then finally, I'll mention uh the other channel which is Devleer Path to Tech which is where I do resume reviews. So, if you're interested in having your resume reviewed or learning about, you know, maybe you're putting your resume together or you don't want it reviewed on a YouTube video, it's anonymous, by the way. Um, you can check out that channel and kind of see the things that I recommend uh on résumés, but like with all my content, it's my opinion, right? So, doesn't mean it's the right opinion or the only opinion, but it's an opinion at least. So, thank you so much for watching. I hope that was kind of interesting and I'm excited for some of the content coming out on the main channel. So, I will hopefully see you over there. Take care.