What AI Topics Should Senior Developers Focus On?

Transcript is auto-generated and may contain errors.

hey folks I'm going to LinkedIn for a question and this one started off great because they said and I totally missed the title but they said hey Nick sorry about the clickbait I hope it worked and it did because I missed it but it would have worked if I saw it anyway they said why was Razer abandoned by Microsoft so obviously they're trying to get me to be like what the hell man um but their questions awesome um so they were saying they don't uh feel that AI is going to be taking jobs at this point uh like sort of like replacing software right so um but what would my advice be to someone who is a senior developer that's not specifically like an AI engineer uh but they want to start incorporating some type of AI into their skill set right um and then

they went on to say I don't want to be behind but they also don't want to just like completely transition into like I'm going to go be uh like a you know working in machine learning or like specifically in AI and I I think this is probably the case for many people right I think there's a lot of people coming into the industry where they're like I see that this is an evolving space and I want to make sure that I am essentially like you know I I'm kind of starting out I'm going to jump onto this whether it's a trend or they see it as an opportunity and there's probably a lot of people who are already senior developers or they're already on this path or they don't have any interest in completely pivoting because they don't think that they need to and I

agree I don't think you need to completely change what you're doing but I think how we're building things is going to change and I've kind of made videos and stuff about that before so before I get into answering this person's question with my perspective and uh I have a little bit of AI assistance because why not leverage some AI to answer a question about AI there's probably an ice cube meme in there somewhere right um so just a reminder if you want your questions answered leave them Below in the comments or do as this person did message me on social media you can look for Dev leader which is my main YouTube channel or uh look for Nick centino on LinkedIn it's a premium profile you should be able to message me and uh if you message me I will keep an anonymous this person

said they didn't care but I'm I'll keep them Anonymous just because that's how I be doing it um okay so to kick things off I'm going to mention I have pulled up on the screen behind me um that I asked chat BT I said hey like what things would you recommend that someone focuses on and I had a couple of things that came to mind and this did a better job than I could think of but the couple of things that came to mind for me because I don't want this to be about like tooling because I've made other videos on that I think we're all kind of being bombarded with different tooling and stuff like that there's cursor there's co-pilot there's whatever llm you want to use to ask questions there's like I use VZ uh vzer Dev for front end development which I

do very little of but I use that if I have to do any because then I don't have to think um there's a million options and every day there's a million more which is super cool there's lots of new stuff coming out but I'm not going to make this video about that I wanted to try and make it about if you are a software developer um and you don't want to get left behind what should you be doing what should you be doing Sor I can't speak properly apparently now before I jump into some of the topics I will mention that I think trying to familiarize yourself with some of the tooling is one such thing I'm not going to focus on the different tools but I think that it's important that's something you're considering and if it's not feeling so good right now like

hey it's not really effective for me I can't find the right tool or they all seems to suck um I would say don't give up on it because the rest of the industry is investing significant resources into this and we might be experiencing a bit of a bubble effect but I don't think this is one of those things where like you know in a year from now we're like ah we're going to give up on AI if I had to bet it's probably not going to be that so I would at least not give up on trying to familiarize yourself with tooling now the couple things that came to mind for me cuz I'm thinking as a software developer that does not want to Pivot over to to being you know working specifically in machine learning if I'm going to build software that's leveraging AI

like what things would I consider and the three things that came to mind I think there was three um was rag so retrieval I think it's retrieval augmented generation it's on this list yeah that's what it is acronyms I work at Microsoft everything's an acronym literally and I still am not good with acronyms so retrieval augmented generation was one thing that came to mind for me uh Vector DBS uh so being able to sort of what's a good way to say this like being able to index things so you can do semantic search on them um that was the second thing and then the third was like this idea around like agents now all three of these are things that I have not personally spent significant time on so I'm not about to sit here and speak to you like oh I'm an expert on

these things in fact um after I get through a playlist on my main Dev leader channel that looks at like repository design patterns um because it's net Focus Entity framework core Dapper I'm going to do some caching and stuff I have a big playlist that I've written on my board that I have to film once I get through that I want to go back to looking at semantic kernel and trying to build some more things that are focused on things like rag that are focused on Vector databases and then being able to do some work with agents I want to make that video content so that I can do like kind of force myself to learn and then at least get to a point where I can showcase to other people here's how you set these things up in a simple way so that's up

and coming for me it'll be over the next few months that I'm kind of dedicating more time to that but these are at least things that I'm aware of that I think could make significant improvements in terms of the things that we're trying to build or at least significant opportunity so to to touch on these three things um because and I'm sure people watching this I'm sure there's plenty of you that have have way more experience so far in your development time than I do in these areas and that's awesome like I personally don't have time for this unless it's like my spare like spare time like right now right but I'm making videos instead so I'm either building my own things which will start to incorporate more of this stuff so I'm either doing that or I'm filming stuff for content in my day-to-day

work I'm not really doing this kind of thing you might get a chance to do that which is awesome or you have a you have other spare time for building projects and stuff so anyway um with retrieval augmented generation the idea is that and I think a lot of people have this misconception like if I want to use uh AI I basically I need to train it on all my data like that's what needs to happen or they'll say I need to fine tune it on all my data and that's not quite accurate and again for those of you that are you know more experien and I am in this area my understanding at least is that when we want to think about training or fine-tuning the best example I've heard of this is like you're trying to teach the AI like skills right so

the analogy I heard before was like imagine and it's a it's a stretch but hear me out cuz it seemed to click really well for me um imagine that you needed to train AI to be able to tie shoes right so the act of being able to take laces tie them together into a bow or a knot or whatever like that's teaching it to tie something but then if you said okay well now I need it to go tie uh a knot on a rope or something like it's already learned the skill of tying so you wouldn't have to go retrain it to go do that it it already knows the skill so you would give it some other type of data and say you already know this skill I will ask you to tie this and it would be able to do it obviously

I'm talking about like a a physical example but that's really the analogy that I was given and I thought that made a lot of sense because otherwise it's like for example I'm using co-pilot when I'm coding co-pilot is not retrained the entire model is not retrained on my codebase the entire model is not fine-tuned on my code base but I'm assuming and I don't know the details behind it but I'm assuming there's some type of uh retrieval augmented generation going on where it's able to get context and we're going to talk about semantic search in just a moment but it's able to get context from my code base this is why you'll hear people saying like oh there's a a meme currently going around where someone's complaining like what is like claw direct my my four months of progress or whatever it's because the the

whole um the whole code base doesn't fit into like the the token window and so that's kind of circulating I don't even know what the original context is I'm sure it's taken slightly out of context but the idea being that like to do that it's essentially taking context it's not even using in that example it's not even using rag but rag would allow you to go basically search for Relevant context provide that into the prompt and that way when you're asking the llm for for some you know some response you're giving it this extra context to work with that's relevant so again it's not retraining right you're not retraining the model you're not even fine-tuning it because it knows the skills maybe in some context you do have to retrain it I can't think of a good example um especially for what I'm talking about

right now especially if you're trying to do things on code or you have a a knowledge base or something like this is the kind of thing where it's like give it extra data to operate on not to train it so in order to make rag work and again based on my understanding of it we need a way to be able to go find the relevant information so again the whole point of this question right was like what things should people be focused on and like trying to learn about I would say rag is one and my understanding is like that's advancing more and more there's different ideas for like how you can pipeline this stuff and I don't know there like when I say rag I'm talking about probably something that's very primitive and there's a lot more built on top of that now which

I have no idea about but in order to find the relevant pieces of information that you want to go use for rag we would like to use generally something called sem search and that means that we're able to go look across a repository of data so that could be documents it's like if you're thinking about a database in general like usually if you're looking in a database um you'd be looking up things by ID or you'd be in this case maybe want to do like a keyword search but that means we have to go look for specific words but semantic search is kind of like the meaning of things which is super interesting right because when you go to search things on Google it's a keyword search right there's a little bit more to it it's not quite just an exact keyword but conceptually it's

a keyword search when we're doing things like semantic search you could take a concept so what's a good example like shoe okay the word shoe is different very different than the word boot however semantically they're very similar because they're Footwear and sandal is another word right these are all Footwear socks maybe semantically they're probably very similar they're used in similar context so with semantic search and Vector databases what we'd be able to do is again take an llm and there's different this is where like my knowledge is I don't know um different models that we would use to be able to create the vectors so they're much more effective at uh being able to create the vectors for us right and we could give it the data say create the vector for this and then we would store that in a database then we use

semantic search to be able to do well give a vector database my understanding by the way and I don't know this for sure but similar technology I've seen when I was working in forensics is something called photo DNA and just to touch on this photo DNA was a concept um that allowed you to do like almost like fuzzy matching for pictures so so if you wanted to go find a picture that was the same traditionally you would have a hash of the picture and the hash values would match you'd say that's the exact same picture but as soon as someone changes one bit literally one bit in the picture you have an entirely different hash and to your eye it might be identical but the hash is completely different so the idea with photo DNA was that it would basically normalize the picture and then

it would create a vector on that picture and when I say a vector if you again I I don't know my entire audience right so I want to explain things in a little bit more detail think about in math class if you have like a a three-dimensional Vector right you're going to have three numbers separated by commas and that'll give you your like XY and and Z axis so and the magnitude on those axes so with a vector what we're able to do is basically have um many more Dimensions so it gets kind of weird to think about in like physical dimensions so don't unless your brain is this big um but the idea being that you can still do a distance between vectors that aren't three dimensions they can be like you know 140 Dimensions you just you you use the same algorithm to

be able to calculate the distance between these so my understanding is that with a vector database and doing Vector search it's essentially doing a distance calculation um so how they create the index though is probably going to be highly optimize to be able to get this distance to work uh fast because I think if I had to go build this I wouldn't know the right algorithm to do to store that effectively but that's the general idea is that it's something like a distance search based on the vectors that were created and those vectors are created by a model so maybe that's not a good way to explain it but um we can use Vector databases and embeddings to be able to go support rag so this is another entire area if you're thinking about building software and you're like okay we're going to use rag

or retrieval augmented generation we need to be able to index large amounts of data because whatever we're building would leverage that then like these would be two things that I think would be awesome to learn more about because even as I'm trying to talk to you about it like it it's probably obvious I know a little bit about it conceptually I understand how some of these pieces work and how you could put them into a system but I don't know the details about them I'm absolutely not an expert and so for me this is an area I certainly need to spend more time on um one I'll I'll share an example like if I had infinite free time something I would love to do and I was I don't know if I've talked about this on on this channel but um I have like a

20-year-old project that's a role playing game and I would personally love to get more time back for that CU it's just fun to build and one of the things that I started to work on was some of the World building tools for that game and that's because I was getting to the point like I've spent so long building these systems and like there's no content in the game like you actually can't play anything so it's like okay well where's the game and I'd be like no no but you have to check out the inventory system and how the items work and how the skills work You' be like that's cool man but like it's just a console app basically right um it's just fun to build the other stuff so I started saying okay well let me start building some some World building tools and

one thing that I really wanted to do was essentially um create like a repository that's like the history of the world that that's for the game and that would have like um you know the geographical areas like literally describing the world and then would be like characters or you know people from history and events that happened and this is I stopped doing this really when like I don't know probably a couple months after like chat GPT and all this stuff was really becoming popular and I was thinking like man it would have been so cool to like basically use rag right so have this all this stuff indexed and now that there's agents which is the next topic like what if I had agents that could basically go start creating Theory of the world right I could tell them I could instruct them like you

are a you know a World Builder for this role playing game and give it some guidelines and then set it off and see if it can go start creating content for the world I would love to see what it's able to do and the cool thing is like given that I would have authority over it I could say like you know if they're building up some of the history and I'm like I don't like that I just remove it from the history and then I could update the instructions to say hey like Don't Make History that's like this or give it better directions but if I had infinite free time I would be having AI agents help me build my roleplaying game at least for the content I think it would be super cool so that brings us to agents though um this is again

something I don't have like much experience at all about I think I can't even remember what I use it was was it Lang chain or something else but basically I whipped up something that was like clone the repository and updated some of the the prompts to be able to have a couple of agents that could work together and I think for this it was kind of like to write an article right so like I had one agent that I could instruct that was like hey you're able to use a plugin to go to the Internet so you're going to go do the research and they're going to it's going to be the agent that can go collect information and then another agent was like the manager I heard in in some of these examples it's like good to help have a an agent that can

help coordinate between the other agents I still don't really know what value that adds like can't they already talk to each other like I said I don't have a lot of experience doing this I had another agent that was essentially the content writer right and then I think there was like a fourth agent I was trying to do that was like a fact Checker or or something else and I was trying to say like hey here's a topic I would like you to go write an article about this and the the results I had were quite lackluster and and it's really interesting because in some of the tooling you can see like basically it outputs the conversation and you can see how these agents are talking to each other it's kind of funny um and I'm not a very good you know prompt engineer but

there was sometimes I could see them talking to each other and I'm like based on what you just told that other agent like I really have a hard time believing it's going to do a good job so I could imagine you know for people that are really good at generating prompts like you need to give your AI agents enough sort of understanding to be able to almost like prompt each other more effectively so maybe that has to do with having um you know better roles toine and stuff for them but I think this is a territory that um is going to be incredible um I think this is the area where people are like this is what's going to replace all the software Engineers is just going to be a bunch of Agents um I I have a hard time believing that um you're not

going to replace whole teams of software Engineers with this at least no time soon but I think that if there's ways to build these types of systems um I think that could be super interesting to focus on what comes to mind for me though that's a little bit of a sort of a it's a bit of a conceptual jump and I'm not sure how to make it is like I'm thinking about how I build software right now like the different types of systems I build I think when I think about AI agents I'm thinking about them being the ones who are help building the system with me but I'm trying to shift my mindset around this of like if I'm building some software I have a software system it does whatever I I always use the example of a pokedex okay so if I were

building a pokedex and it was a web application there's a database to store the Pokemon I've seen like how would I put AI agents into that and what would they do like what would their purpose be so I'm trying to wrap my head around I don't just want agents for helping me write code or something I would like to find ways to build software that leverages agents and it's kind of the same thing like with llms in general it's like the use case that everyone goes to is I'm going to have chat like I'm going to have like an you know chat support on my website or chat support and whatever and it's like yeah like that's the use case we've seen a million times so how else can you go build stuff integrating an llm and in my case I'm saying how would I

put agents into a a software system so I think my perspective is like those three areas as a software developer I think are going to keep growing like crazy um I'm sure there's others too um but those are the ones that I'm personally very interested in and I would say if you're a developer ER probably learning and understanding and how to integrate those and leverage those would be a great opportunity to focus on I'm going to jump over to to chat gbt output here um and go through this list so Rag and Vector databases with embeddings are the first two that came up uh the third one they have is fine tuning and prompt engineering so yeah like I I don't have good examples of where I understand where fine tuning would be helpful the example I gave earlier I was talking about using rag

instead of fine-tuning I still haven't seen a good example where someone's like the the cost to go do the fine tuning makes it worth the result that it would have so I don't understand that so again that would be great for me to learn prompt Engineering in general I think is one of those things that like we should all continue to get better at uh I certainly have a ton of room there um especially when it comes to like structuring output like I've try if you think about it if you're building an API for example you have full control over like how you want to format the result object that you're sending back as a web request or whatever um or even in code and not over the web so when you start putting AI in the middle of that you're like by the way

this thing that speaks English format this data for computers um it's funny because if you give it examples like format data this way it can do a pretty good job um but I've seen like even with prompts at work most of the time I've seen it sometimes go you know like certainly I'll format my output properly for you and then also adds the the output and you're like okay but I can't parse that whole thing programmatically because you added in some extra sentence so the better you are at prompt engineering I believe the better that you can work around some of these weird like nuances uh certainly I've have not perfected it so um the next on their list number four was AI agents and then they say multi-step decision making so maybe that's like better framing when I was talking about like how do

you integrate agents into a software system the multi-step decision-making is probably part of it so they say autonomous AI agents they list out Lang chain Auto GPT and I haven't heard of baby AGI whatever that is something to go look up right so can perform complex workflows by breaking down tasks retrieving information reasoning through problems uh and then they say learning how to create agents that can act on apis databases or external Services expands the utility so again um some of those things that they said I'm like I could think about how I would leverage the agents to help me build but how do I make a piece of software that is already leveraging those um in semantic kernel one of the things and I've made a video about this on my main YouTube channel is like uh you can drop in plugins that allow

the agents so allow the when you're asking the llm to go perform or give you a result you can give it a plugin that enables some ability so it can call some code that you give it which is super cool because as soon as you can write code that's called by the llm now you can go wire it up to whatever you want so I think there's a lot of really cool stuff to build that way and I want to more time on that next up on their list is data pipelines and feature engineering for AI so they say well structured data pipelines are critical for feeding high quality data to AI models interesting okay so Engineers should learn how to pre-process clean and transform structured and unstructured data efficiently while understanding feature engineering Concepts that improve AI performance so yeah I guess in general

like I mean I've given AI some pretty unstructured data and it does an okay job but I guess when you're talking about in volume we want to make sure that we can give AI uh you know sort of transform data that's normalized um I also wonder if this means for especially like on the training side it's one thing where it's like it's already you already have a model and you're trying to interact with it but especially for training and all of that I can't imagine what has to go into that there's another item a little bit lower on list I'm just going to jump to it now but they say like AI ethics bias and explainability so I mean I think that's super interesting if you're spending more time building stuff with AI I think that could be really important to understand um they have

ai powered search and recommendation systems so this is kind of what I was talking about so they say search and recommendation engines powered by AI so then they say semantic search with embeddings they say hybrid search using keyword and Vector methods okay I I can imagine in this space there's like some really advanced stuff that is way over my head um yeah they say those things are increasingly common like yeah that's the normal stuff now so Engineers can explore different Technologies uh for this kind of thing but yeah I think um you know we have the concept of search already built into a lot of software and so AI powered search and recommendation I could see that being instead of just keep keyword search and then having fuzzy search like now it's like truly building in semantic search and having that done effectively I think

that could be super powerful um they talk about model deployment and optimization this is something I don't have any experience with so deploying AI models and production um requires knowledge of inference optimization this is like stuff way above my head so for example quantization distillation pruning okay and serving strategies like onx tensor RT and serverless AI inference I got I got some learning to do so Engineers should also learn about Frameworks such as Triton inference server hugging face inference endpoints and vertex AI so yeah like again but if you're Building Systems that are leveraging AI then I could imagine that this is probably something that you'll at least be more and more exposed to um this one's interesting mul modal Ai and vision language models so I don't have um like any experience at all aside from like using uh like Dolly to go generate

or I guess it's just built into chat GPT now to go generate pictures and stuff like I don't have any personal experience with leveraging llms to go like do anything in other uh modalities aside from text so I think there's a lot of interesting things that could be explored there because when I think about leveraging AI in software systems I I'm imagining that we have I don't know good way to explain this like when we're building programs it's like we have programmatic interfaces between things but suddenly we can cross different modalities when we have models that can do it right so like one of the earliest things ever like way before having llms and stuff was like text to speech and speech to text so I remember being a kid and like trying to build one of the first things I remember trying to build

I was like what I want to do is I want to have like voice commands for my computer so I want to be able to like you know speak things and have it do it I remember being able to get my driver's license and being like oh man I want to put a computer in my car and I can like talk to the car and have it do things and then I mean we have all this stuff built into our phones but you can use like that's a different sort of a mechanism for like interfacing with technology right and I just imagine that when we have more and more opportunities with like multimodal um models that we can start doing more and more interesting things so um tons to explore there and then the last thing they have on their list is just AI augmented

code generation which is kind of what I was saying earlier with the tooling um kind of staying on top of that so I'm not going to dive into that but um I thought that was a pretty interesting list list I like I said I had a couple that came to mind for me and there's definitely a handful in there that I'm like you know basically zero experience with so to this person that was asking how you know they don't want to switch to doing machine learning they still want to stay as a software developer but don't want to fall behind I'm hoping that that's a helpful list at least to give you an idea of different things you could go look at for me even just speaking to all of you going through that list and even refreshing my mind on okay like what's the

last thing I built that used Rag and like I bet you if I went to go reimplement something now the tools that we would have available like instead of me having to go Cobble something together there would be like just use this package and it would do it a million times better than you could you know even a year from uh a year ago so um lots of interesting opportunities and I certainly have to follow up on a bunch of what's uh been said in this video so hope that helps and just a reminder if you want questions answered leave them in the comments I'll try my best and then otherwise you can look for Dev leader on social media send me a message or Nick centino on LinkedIn and I'll keep it Anonymous and the more you write the more detail you add if

you have career problems or things you're trying to figure out you know feel free to elaborate and it'll be totally Anonymous so thanks so much and I'll see you next time take care