How Are Engineering Managers Using AI Outside Of Coding?

Transcript is auto-generated and may contain errors.

Hey folks, we're going to talk about AI today, but I'm going to do a bit of a twist on it because I was thinking about this a little bit more recently. Um, so usually obviously in AI talks and stuff, it's always like how are we using co-pilot claw cursor to go crank out code to do code reviews? Like what? It's always code. Um, and uh, I'm thinking about this more because I spend a lot of time outside of work coding and using AI for coding and all this stuff, but uh, it's funny because in my actual role, I'm like, wait a second, I actually need to play catch-up with AI in terms of, uh, like using it as a manager.

Um, and it's I don't know, it's like kind of ironic, I guess, uh, for myself like thinking about this cuz I I know that some of the use cases I saw coming up for AI for managers was like, you know, uh, basically around uh, sort of performance review conversations and stuff and I'm like to me like this is the worst. Uh, personally, I think it's like the worst because you go from like having what could be, you know, uh, in some cases like really positive conversations for your employees where you're like, hey, like you're doing an awesome job. Here's all the great things. Um, and then the other parts that are more challenging where it's like, hey, like these are areas you need to work on for improvement. We're having difficulties here. Um, and I think in either of those cases, you don't want to lose the personal touch.

Like that's my perspective on it because they're already they're already something that's uh like not mechanical, I guess, is how I would put it. And I think that um I think that's an opportunity where the messaging is like critical. And so if it starts to not sound like you, I think the the impact that that has is like very significant. Um, I don't know, maybe I'm someone who like hyperfocuses on this, um, overindexing on it, but I would say if I if I were in a situation where my own manager was like needing to give me some critical feedback and I was reading it and I'm like, "Guy didn't even like I didn't even write this." Um, I would be I would be pretty bothered. That's how I would feel about it. Now, whether that's warranted or not, I don't know. I guess open for debate, but I I would feel bothered.

And so, I've kind of steered away from I don't know, like from using AI for a lot of like my day-to-day stuff as a manager. But when it comes to uh I mean there are there is day-to-day stuff where uh say like code reviews and stuff I'm not code reviewing every day but there are days where um I will spend more time doing code reviews and not and so there are situations even for that where like yes I've leaned on AI but again because it's code related and I can do that. So, just to give you a super quick example of that, um I remember an employee was putting together a a code change that was supposed to be pretty small. It was actually just updating a query for something for a recurring job, like having to query some data source and uh having to to fix a sort of a gap in this query.

And I remember seeing it and being like, okay, this is an interesting situation where the change they made on the surface looks like it makes sense to me, but I feel like I wasn't going to add any value aside from just saying like looks good to me because uh this is one of those things where say I approve it and say that is enough to pass the gate. like am I approving it just because I I don't know enough like sure I get I guess it's okay like that's kind of how it felt to me and so that wasn't like a good feeling so uh I remember pointing co-pilot at the review and just said like you know can you look at this query and can you find any you know any opportunities or any gaps here and uh it just allowed me to to

kind of have like a rubber duck, I guess, to go back and forth to discuss this uh this query and then I could see like uh you know it was calling a couple of things out and then I could make a decision like do I actually think that those are important? Uh or uh in some cases it's like maybe the importance isn't the the ma uh the point. It's like it was saying something like here's how here's how this actually works uh based on whatever and I'm going wait a second I actually don't know if that's this person's intention so maybe I should double check with them. So kind of the the style of code review comment that's not like hey go fix this. It's more like hey just checking like if uh if if this was intended or not cuz if it is great then you're doing the right thing apparently.

and if not then then you may want to reconsider. So, I've used, you know, co-pilot and stuff in situations like that on reviews, um, for better understanding. Um, but like, you know, uh, for performance conversations, for organizing tasks and stuff, uh, or like I I don't think some people realize like how much email comes in every day. And that's just from working in uh the uh Microsoft 365 side of the house. Like my my inbox doesn't make sense. Like I and I'm not saying this to like brag about my inbox. I mean there's so much email sent that uh I mean it genuinely. it would be impossible for me to try and read every email that came in to my inbox because there's so much automation um and trying to organize it into folders and stuff like it works until until it's a full-time job like trying to organize the automation so like I need something that's going to help keep up with that so I don't miss stuff.

So anyway, where this is bringing me to is like um we had a hackathon last week and so I was I was spending some more time um doing some like eval scenarios for some skills and things that we use at work and that's just because my my one of my feature crews, so like a sub team, we have some skills for working with some things and uh I think we've been using them and they've been pretty helpful. That's really cool. And you know, for me, the extension of what those look like is like, okay, well, I want the broader team to use them. And then the extension of what that looks like is I want people outside of our team to be able to use them or or flavors of them. Maybe they don't need every skill we have cuz some of them are more for like internal development.

But uh as a platform I want to get to the point where our partners don't have to come and ask us questions that they could just you know get very good answers from AI um you know the same way. So I am trying to make sure we have eval scenarios so that as we're updating these skills and andor trying to share them out that we have confidence like hey look there's literally an example prompt like this when you ask it here's proof that it does what we're looking for. So I was spending time on that cuz that felt uh most important to me in terms of like value ad. But um towards the end I was like wait a second like I think there's an opportunity here that I probably should be looking at and that's organizing work items. So um given that I joined this team uh it's been two and a half years ago on this particular team there's uh there's stuff that's existed in Azure DevOps.

So like where we have our our work items and stuff for a long time. Um there are things that are filed in there where you know someone will make a a request or whatever cuz it gets done a million ways right? It's either someone will reach out on a team's chat, someone will reach out over email, someone will reach out over an incident. Uh there might be a repair item that's attached to um a post incident review. like it could be a million things and one such thing is just creating an item in Azure DevOps and then and given what I was saying about how many emails I get a day if that email's filtered out I won't even see it. So there's stuff in Azure DevOps that is just it's all over the place.

And so yes, there's a subset that's like, hey, we're going to keep this focused and organized, but realistically, there's so much in there that it's like uh and I'm just talking about for one of my sub teams, like it's it's basically it's going to sound kind of funny, but like there's so much in there that it makes it useless. And it's because there's there's no consistency in how it's organized. Um, and it's just long overdue. So, I was sitting down and and there's a meta point here, but uh I was kind of sitting down with co-pilot the other day to start like I got to get this organized into our objectives and key results. I need to make sure that because we're a platform team, there's always some stuff that comes up that's not really like, hey, this was planned in the road map, but like we have to go do something operational day-to-day kind of things.

So, I need like a bucket of work for that. I need to make sure that how I want this structured and to be accessible is that anyone on my team at any point if they're like, "Hey, my work is wrapping up and before I start my next big task or my next big project. Like is there something small that I can kind of pick up in between like flighting out some changes and like checking in on some things like I'm not quite ready to pick up the next big thing. Um or I mean it could be it could be like I'm ready for the next big thing like what are the the big things on the road map that are clearly not yet assigned. Uh obviously those are conversations I'm more than willing to have with people on the team. Uh of course it's literally part of my role.

Um, but I also want to make sure that they can do it ahead of time. Like they can go look and say like, "Oh yeah, like we talked about this like or this other one or this other one over here like hm like maybe I want to go read more about that and see like maybe I should go talk to Nick about like is that the next one I can go do?" Um, I just want to make sure that people have that information at their fingertips because right now it exists, but it's like so disjoint that it's almost like it it might as well not exist. Like how are you ever going to find it? So, um, I'm trying to play around with that to get things organized and, uh, to kind of introduce this meta point. It's like I think one of the really important uh use cases is is really just pulling data together.

Um yes, I can go run ADO queries uh to go look for all these items and stuff like that. You know, there's tags, there's paths and stuff to to keep things organized. All this stuff exists, but like it's uh to me it's just it's kind of like it's a it's daunting. It's a pain in the butt. It's overwhelming. There's so much going on there that it's almost like I don't how do you even start? And so the barrier for me at least the barrier to like getting started on some of this stuff where there's so much to work with is uh is much lower when I can just talk to co-pilot about it. Um and this isn't a co-pilot plug. You pick claude or curse or whatever you want.

Um, just the fact is that like because it's able to go over so much data and then and then sort of present it in a way to me that lets me just focus on like what's the problem I'm trying to solve, not how do I go navigate the data. It just makes a tremendous difference. I can focus on probably what I consider the, you know, the the challenge at hand, not how do I go across data. And I I talked about this even for like investigating um issues and like understanding live site stuff like just being able to go over data like this. So that's like one use case that I think is going to be super helpful for me. Um I'm hoping to report back on some success with that. If it works for one of my sub teams, I'll do it for the other two as well.

Um, but yeah, I'm I'm just looking forward to having all that organized. And then on top of that, okay, so like what are some other scenarios like that? Well, I mentioned the, you know, emails like kind of nuts. The same thing actually happens with Teams to a lesser degree, but um if it's a direct message or I'm in like a group chat or something, I feel like it's uh a lot easier to keep on top of. But still kind of challenging if there's like long running group chats or things getting resurrected. Where it's really difficult for me is like these teams channels that uh I kind of get dropped into periodically because they're not like a normal team's chat and I'm never looking at them ever because I have enough other chats that I'm focused on. So, I'm never looking at these channels. And so if someone's waiting for an update from me in one of those channels, I'm like, it's never in my in my view.

So I'm thinking about doing something like a a scheduled task that's maybe a couple times a day. So say something at like 5 or 6 in the morning, like well before my day. Uh I shouldn't say well before my day because on a day like today, it starts at 6:00 a.m. when I'm back up on call, but starts at the beginning of the day. Maybe something uh at lunch where I can just go do uh a scan for some of this stuff. It's my wife and and then yeah, then can like maybe get things more focused for me. There's another one I was thinking of too. Um but it's escaping me. Oh, even for um I was going to say I think one of the things I want to try doing for performance reviews next time is not to have it write the review, but

um again to pull data because I want to make sure that I've talked about this with my employees when we have these conversations part of it is like they're presenting like they do a write up on the things that they've accomplished and like uh areas that they want to improve. So they do that, I respond with, you know, my part, but that's between two of us, we should have it pretty well covered, but I still want to make sure that I can go use AI to go across the last like 6 months um and say like what else was there? Like there there's probably other things that both of us are like, oh yeah, if we were to talk about it, oh yeah, there was this thing. So I want to see if I can use it to close some gaps. Um I still want to do the writing because I think that's important.

Um, I want it to be my thoughts, but I just want to make sure I'm not missing anything. So, um, I, you know, metapoint is like I'm trying to use AI to help in my manager role, and a lot of that for me is like, uh, going across lots of data so that I can try and focus on like the part that I think matters. So, thanks for watching. I will see you in the next one. Take care.