I Stopped Using AI Just for Coding (Here's What I Found)

Transcript is auto-generated and may contain errors.

Hey folks, we're going to do a little AI recap this morning. Um, last one of my last updates I was talking about how just trying to figure out a couple of different things other than just just code, right? Um, as an engineering manager, especially uh having to oversee a couple of teams. I'm trying to look at going to turn off these wipers cuz I keep being told they sound like farts over a couple of teams. I'm trying to look for ways to um kind of bring some of that I don't know the the optimization or uh the benefits that I've been seeing with coding to to other parts of my role. And it's primarily because uh outside of work like I code a lot um and I guess I AI codes a lot for me now. And so that's that's been going really well. I've been talking about lots of optimizations there but work has been a little bit light.

regarding that. Um, I use it a lot for for investigating for, you know, for uh helping with incidents, live site investigation, stuff like that for the platform we run. And so that's been super helpful, but like what about just as a manager? So, uh, I'm going to talk a little bit about that and then kind of bleed over to some to some of the stuff that I've been, uh, doing at home, which will bring us back to coding and all that. Um, so, uh, kind of maybe maybe boring for some people, but one of the things that, uh, I need to do is organize Azure DevOps. I feel like uh this is one of those things where it's easy to kind of go underwater as in if if no one's really staying on top of it then this tool like I personally I see a lot of benefit in Azure DevOps and Jira and things like that.

Um, but when they're not maintained, they just kind of fall apart. Like, you know, you have a sea of just hundreds and hundreds of things to do and no priority and like, well, why bother putting more stuff there? Like, you know, like, you actually have to, it's a tool, so you have to give items like work items, data points, things like that, and then actually do something with it. you have to prioritize it or else you have a whole bunch of stuff and then it's not doing anything for you. And I feel like uh I don't know since I've been at Microsoft in the areas that I'm in coming into teams that have already existed for a while. It's kind of like it's kind of at that point where there's a bunch of stuff and not a lot of organization and then the benefit of those types of tools is uh diminished.

And it's not that we don't use them. We use them heavily. And then we're still I personally not feeling like there's benefit like if I this is like one of the goals that I have with with teams that I manage is like uh if someone is wrapping up their work, right? they should be able to go look at the exact same spot that I would for like what's what's the next highest priority or as they're working on something and getting close to it if they're like hm like this has been a cool project like I wonder I wonder what else right of course they can come talk to me like and I would never want to discourage people from doing that of course they could talk with people on the team and like see if anyone else has you know insights as to what's coming up or what other big projects are, I would never want to discourage people from doing that.

It's just that if they were to come to me, I'm going to go to the same spot that I want to be able to give them, right? I want to give them a query or some other way to go look at what is in the priority list. Now, we don't follow super strict sprints. Um, where I used to work, we tried like we were uh, I would say almost like leaner in that sense where we could organize sprints, put things into them, uh, be more careful about what goes in and out. Um, since I've been at Microsoft, again, for the teams that I've been put on, we haven't been like hyper vigilant about that. It's like some of the some of the work streams, they like inherently must bleed over sprint boundaries. Uh because if we're doing like a I don't know if a sprint is anything less than a month, there's some things that just like truly you can't call them actually done for over a month, right?

They might be like most of the way there uh long before that, but truly just based on how some of the systems work, it's not it's not feasible. So instead of being strict about it, we just kind of make things work. So, we haven't been super strict on sprint stuff and like going to look at this board has been kind of like not that useful. But anyway, I'm I'm trying to get things set up in a way that people can actually get some benefit out of it. And so, I actually had some success over the past week and a bit. I was on call a lot over the past two weeks. So, that put a damper on things. Um, but with that said, I was able to spend time. I was able to get a bunch of stuff organized. It's not done, but I figured we'd talk about it.

Like I said, I realize maybe some of this stuff's not super exciting, but one of the big challenges with this kind of thing is like going to find all the stuff. And so it it's not only that it's challenging to do that cuz like maybe that's not such a difficult problem. which just is kind of boring and sucks. But the other thing is like actually clicking through like uh you to go find things I have to start basically creating an inventory of them. I have to keep expanding a query and and all this kind of thing and I don't want to do that. That sounds like not enjoyable. It sounds like it's going to be errorprone for me. Um and so using something like co-pilot again use whatever tools you want.

um using co-pilot to be able to do this, connect to Azure DevOps, start describing things, having it go explore, and then just like talking to it to to filter what needs to be, you know, looked at and then uh navigating things this way has been so tremendous. It's uh again one of the the things that I keep coming back to with AI tooling like this is that some of the stuff that it's doing is not necessarily so groundbreaking right like it's just querying Azure DevOps under the hood and that's not that groundbreaking itself just an API you call but the sort of the groundbreaking thing in from an operational sense is that I don't have to jump between thinking about querying and logic and syntax in this particular moment because that's not that's not the point of what I'm doing. The point of what I'm trying to do is organize work.

So every time I have to do a bit of a mini context switch to say okay like I'm trying to organize work but I need to have a query. I need to maintain this. I need you to think about the query logic to to and and or things. Again, not complex on its own. It's just one other like mental state that I have to put myself into to to do that. This is maybe seems less applicable for something like Azure DevOps work items but certainly when I talk about uh you know querying data sources like SQL or we use custoto a lot um you have to think about the query syntax all this kind of stuff the schema of your tables that you're pulling instead of doing that so hyper specifically I can just talk about what I'm trying to do so co-pilot made it really easy for me to filter to what I have to look at.

Um, I was able to get uh early on some queries put together to say like here's how I want to think about things and then I could have this I didn't what's a good way to say this? I've been trying to figure out how this stuff gets like so messy so fast in Azure DevOps because like it's not there's a lot of really smart people that I that I work with and that have come you know before my time at Microsoft and so I'm like how does this keep happening and like when I've been on the team I'm perpetuating it right I'm not I'm certainly not making it any better I'm trying to this moment um like why does this keep happening?

And so when I was trying to talk to co-pilot about here's what I want to do going over like um sprint over sprint and semester over semester like here's just like what I'm trying to accomplish just this act of going back and forth made me realize something that was pretty important and that is that Azure DevOps and this is uh you know Jira would be similar uh we follow objectives and key results as a as like a pattern for planning. And so if you're not familiar with this, it's a concept called OKRs. You can go read about this. I've talked about it before in other videos, but objectives and key results are by design like they're for a time period, right? You have a particular objective which is going to be for this period of time. You have key results which you are trying to make progress on over a period of time.

And so what happens is that you get to the end of that period of time and all of your key results, by the way, are supposed to be some measurement that you're trying to address uh that is on a scale, right? You're trying to move some metric from X to Y. And so you design them in a way that allows you to try to hit the target. I mean, make progress towards a target. You could end up overshooting the target. But a lot of the time we set more aggressive goals and a key result not to, you know, make ourselves, you know, punished because we're never going to hit them, but to try and make sure that if we set some aggressive goals, we have to get creative in how we solve the problems to reach that target. Right? If you make everything way overshot, it's like obviously that's not helpful.

But if you have everything under, then it's like it's it's too easy. like you're almost uh I don't mean too easy cuz I don't want people to I don't know to have enjoy work. I mean too easy in the sense of like coming up with creative solutions that push boundaries. So he said some more uh at least slightly aggressive objectives. The end result is that hopefully you can try to achieve them but if you're under it's also not like a fail, right? The whole goal is to move towards this target. So, what happens at the end of a a period, whatever that period is for you, there's likely going to be some leftover work items that sit in these objectives and key results. Now, what I realize is that there's no formal pattern or practice that people are following consistently, including me. So, I'm not blaming other people, right?

Um, you reach the end of this period and then it's like, cool, we have to do this again. a lot of the objectives and key results. My god, I need sunglasses. I don't think I have any in here. That'll help. Kind of. Holy cow. Um, so you end up having all these things that are still kind of left open and a lot of your objectives and key results the next period are going to be similar, right? Cuz if you didn't reach the goal, you might say, "Cool. Well, like now that we're in this position, now we want to set another goal, right? Like that's that's related. Sometimes they're different, too. Whatever. But what do you do with all these work items? So, do you just rename the objectives and key results? Do you make new ones? Do you uh do you close out the work items that didn't make it so that you have some historical record of that and then reopen new ones?

Like, what happens here? And you might have an opinion on that, but what I'm trying to tell you is like no one has formalized that. There's nothing consistent that's done. So why does that matter? Well, the problem is that we end up having all this that's left over. And so one one way to work around this is that if you talk about the work in terms of the functional area that it exists in, that's not something that uh changes with time, right? I mean, yes, it will slowly evolve over time as your product and service changes, but like I have functional areas that have existed for longer than I've been at Microsoft. So, if you allocate work to the functional area every period, you're not like trying to to maneuver things.

So when I had like I kind of made this realization by talking with co-pilot again not groundbreaking but I said okay for me to get organized here because it is all over the place let me actually take a step back I'm going to align all of the open work to functional areas so it's at least anchored to something because right now it's just lost in a sea of god knows what. So, let me anchor it to these functional areas and then once everything's organized, then I can start pulling it into these more dedicated work streams, right? Because the big challenge for me is that when everything's everywhere, if I have I think in on one team I had like almost items to go, you know, organize, there's no way I'm going to organize 400 work items into time slots. right? Like it doesn't it doesn't even make sense because a lot of those aren't even actually relevant anymore.

So like I need to be able to triage them. So I'm going to come back to that in a moment. Uh but basically had to organize everything by functional area so that it's inventoried and then start pulling them into some timebased uh objectives and key results. And then I'm going to have this challenge at the end of this period which is just under 6 months away where I go, okay, cool. I'm going to have these objectives and key results that are time based. What is the strategy? What do we want to follow? So, I'm really just buying myself some time right now to organize it by functional area, but I haven't solved this problem yet. The first step is organizing by functional area. on the triaging thing.

Uh, one of the things I wanted to do talking with co-pilot was like I I want to make sure that everyone on my team feels comfortable that uh, you know, if they if they're like, "Oh, yeah, like we should, you know, there's some tech debt or there's whatever thing that we should follow up on or even I have this idea for a cool new thing we should do." Anyone should be able to go propose work. That does not mean that all work gets done. That does not mean that all work gets prioritized instantly above everything else. It means that anyone should be able to have a say and express themselves. Uh whether that's like I said tech dev bug fixes or new features and ideas. So what's the minimum amount of work that someone needs to do if they want something like that documented? I don't want to people to have to fight the tools.

I don't want them to be like, I don't want to propose my idea or whatever because I'm going to have to go, you know, set 50 different fields just for it to be recognized. So, I worked with co-pilot just to kind of establish a pattern. Again, not groundbreaking, but whole point of this was like it it just took the the burden off of me to to think of exactly how to do this on my own. And uh we just came up with a a query that everyone can look at a pattern um and then some basically some some fallback. So you know here's the one pattern and uh if you're not doing that right and you tried to do some other stuff cuz you were hoping you'd get it right then I can also put into this needs triage bucket like misconfigured things. So, uh, long story short, like you can you can try really really hard and if you screw up, I can still catch it.

And, uh, otherwise, if you put in, you know, the least effort, which is like tag it with one thing, uh, if you tag it with one thing, it must show up on my on my triage board. Um, and I realize that sounds like overly simplified, but uh, I think a lot of the fallback cases make this uh, pretty handy. So basically I have this big triage query now and I can work with people on the team just to close stuff out or say like hey is this duplicated and let's get it lined up uh the real thing lined up with the right time slot. So all that made a lot of progress um and just wrapping it up with one team and then I'll just repeat the whole thing with the two others. So yeah I think that made something that was completely overwhelming clearly for a long time.

uh a lot more straightforward and uh could do it all right from the CLI which is great and uh I guess the other nice thing was that as queries and stuff were coming together I could run them in Azure DevOps and visualize what was going on because this is one of those things where Copilot could be saying it's doing something and not so at least I have some visibility into what's actually happening. Um, I think the only other thing I'll mention, sorry I spent so much time on that, but um, the only other thing I'll mention is that from an AI perspective, I've been trying to look more at uh, like recurring and scheduled workflows. So, if there's something I'm doing like, hey, that's a helpful call it like a skill, right? It's a helpful skill I'll use, but like that's actually something that I want to be running regularly.

And so, uh, one of the more meta things that I'm kind of doing right now, I think for folks that have watched some of the other videos, I talked about, uh, this setup I have where I'm trying to put together reusable, shared instructions and things like that so I can put them across my my repositories. Um, I just did a recurring I haven't run it uh on a schedule yet, but set it up to um run this recurring thing that will look at my repositories that exist. It will go look at the custom instructions that were put there. Um, and then co-pilot will analyze it and see if there are patterns and practices and enforcements I'm using with the instructions that should be pulled back into my uh my common set. And then that way I can reed them out to the others and they all get the benefit of it.

So that's my wife. Yeah, that's uh one of the things I'm looking at so that I don't have these tasks that I have to keep following up with and I get the benefit of them running in the background. So um it takes one huge step out of that like how do I get all this data? how does something do the initial analysis and then I can kind of look at it and sign off and if it's doing a bad job and being stupid then I can tune what it's doing so that it's this repeatable thing over and over. So looking at different opportunities like that. So that's that going to CrossFit. See you later.