Measuring the Unmeasurable: Manager Effectiveness

Transcript is auto-generated and may contain errors.

Hey folks, we're going to go to a question from the comments. This is from Excel 92 and not like Excel like the office product but Excel like accelerate. So this viewer was asking about how quantifiable manager skills are and I think it's a really interesting question um because quantifying things is I mean can be quite challenging sometimes we can talk about things and have a general understanding of like you know these are good oh my god man this car uh these are good traits these are good um I don't know examples of people doing things. Uh you might have managers that you've worked with where you're like, "Oh my god, they're an asshole." Or, "Oh my god, this this person's dumb." Or whatever. Or the opposite where you're like, "Wow, this this was such a great manager to work for, but how do we how do we start quantifying what these things look like?" Right?

Quantifying is challenging. So, I think the way that I want to approach this is kind of talk about like one of the challenges. What are you beeping at? One of the challenges with uh with doing this is that we got to kind of go back to expectations I think like what is or what are the expectations of a manager? How do these uh maybe change across uh companies? How do they change across sort of the level or experience of managers? I don't want to spend too much time on that but I think it's important to kind of frame then we can break that into different uh from those expectations like different qualities. So like kind of talk about the qualitative part and then for each one of those areas can we start like seeing like how do you actually measure that? So that's my my thought process.

I will start by saying that like you know there's a lot of variance in what is expected of a manager and that can be really challenging just in the first place. So you know my my advice for level setting expectations also applies to managers right. So if you are uh thinking of moving in that direction or you're a new manager um when I try to give advice to people like you know work with your manager to understand what's expected of you in your role like that applies to also you as a manager right and um so keep that in mind the sort the biggest sets of like variance I see are really around like how much how much time you still kind of may be spending as as like a hybrid in an IC role. And sometimes this is an expectation and sometimes this is like not an expectation, but uh there's sort of a a lack of experience in a manager role to be able to understand how to do that.

This is the slowest right turn I've ever done in my life because this person in front of me has never operated a motor vehicle and they're swerving off the road. They saved it. Very good. I'm very concerned for this person because they can't drive straight or near the speed limit. Um we'll see how that goes. So, I would say especially at smaller companies or in smaller teams or like if you're kind of spinning up a team, right? If you're not managing a lot of surface area in terms of like things you're accountable for or people um one sec, I got to get around this person because I'm not dealing with that. um then I would say if you're like that the coverage is small then I would say there probably is more of an expectation that you are continuing to be more hands-on right it's

kind of like it's almost out of necessity and it almost doesn't make sense if you're like you know I have a smaller surface area in terms of people and technology and um and instead of you know like helping deliver on those things like being hands-on, I I'll just kind of sit back and wait. I would say um where that might differ is uh especially if you are put into a position where like you you're being told like we need to ramp this stuff up fast in terms of getting people on board, then you might be almost like spending more time as like recruiter and HR kind of uh focus, right? Because you're going to have to be spending time uh getting the word out, interviewing, blah blah blah. So if you go to the opposite end of the of this spectrum which is like especially as you're as you're growing in a manager role you take on more responsibility right well in theory.

So if you're taking on more responsibility that may mean a combination of more people andor uh more product or service areas because you you are responsible for managing what those are. to give you an example um in like my career has basically been as a frontline manager for over 13 years now. Uh so I moved into that early and I would have been responsible for I actually had a lot of people reporting to me like right when I started like awkwardly probably just because we were a startup and we really didn't know you know best ways to do things. So almost out of necessity, but I had a bunch of people reporting to me um really early on and the surface area was like our our product and this was carved out a little bit. One of the other uh guys I worked with kind of moved into a a PM role.

So like kind of tackled things from different angles. Uh but I had a bunch of direct reports in a single product area. And then as the company was growing, I I ended up having like I actually took on a smaller team in a more like strategic kind of product area. And then over time that grew into um you know several teams uh across several product areas and now at Microsoft I moved into a deployment team. So I actually took a more narrow focus again. Uh which is it's interesting right because that move was like from a career perspective in terms of like uh I don't know like as as a manager growing in their career uh that was like absolutely not a a step forward which might sound kind of funny because I went from like I manage multiple teams of multiple people on on different things to like a back to like fewer people in a smaller area.

But this that transition for me was absolutely going from I had always been very hands-on with my teams until the very end. I was still able to kind of code in any area that um that we had cuz I had been at the company since the beginning. Um but it like wasn't effective for me to be spending time coding. There were other things that I needed to do strategically. I guess no one's driving today. I don't know how long we've been recording, but I've already seen like too many things that make me want to not be on the road. So, going into Microsoft was uh for me something where I would say like I I don't get the opportunity to lean into some of my other skills that I would have had. And like for me that was a bit of a what the heck is going on here.

They added in this weird barrier. If you watch my other recent videos, I like was saying the highway was closed off and had to take some some different paths to work. But like this I don't know what they did here. There's like this super awkward median that I'm like, I think I could drive over it, but I don't want to with a a sports car. I don't know. Anyway, going from being able to be very technical and hands-on into an area where like uh you are now dealing with like a code base that's like I don't know in some cases older than some of the people on the team is not going to let me play into some of my my strengths, right? like as a as someone who was a net developer working in C and uh kind of working in a codebase for many years.

Yes, the language and stuff was the same so I could at least lean into that in my new role but codebase different. So I'm I'm saying this because I want you to keep in mind when we're talking about like how do we like what are the types of responsibilities? How do we start to quantify that? Keep this one in mind. So my scope was significantly smaller in terms of people in terms of product area. Uh you could argue like given that it is a you know massively distributed system across the planet like in terms of like responsibility overall arguably greater but from like a career perspective like certainly not a step forward. And ironically, like it's been 5 years at Microsoft, just over now, like 5 and a half. And so, ironically, I am still kind of front I'm a still a frontline manager, so I've not actually gone up in terms of career responsibilities.

It's been 13 years. And um what has changed though is like my amount of service area. So now it's across uh still within uh one team but like three very different areas and uh the number of reports I have is roughly still kind of uh towards the upper limit of what I've managed. Okay. So these things they change over time depending on like where you're working seniority that kind of stuff. Now what are the types of things like qualities focus areas that we need to think about when we talk about an engineering manager? One of them that I talked about was like technical ability. You could break this down even further, right? Because it's different to be you could have a like what's I'll use myself in as this example, right? So, I do not have the indepth domain knowledge of our codebase.

So, if you said, "Nick, I need you to go fix a bug in one of these products or services," I can assure you there would be potentially, you know, junior engineers that would say like, "I can go find that and get it fixed faster than I could." Right? So, like they they have the the domain knowledge of the codebase. Does that mean that I couldn't do it? No. Like I could might take me a little bit longer cuz I'm like I literally have to go get set up to do this cuz that's not in my role right now. And it's not in my role cuz it's not an expectation of my role and it's not the most effective use of my time in my current role. Now, that's really to be hands-on in the code. But if you said, Nick, like I need you to work with the engineers and, you know, help design a system here, then yes, like I could absolutely do that.

When it comes to actually getting to the nitty-gritty of like where where code is going to or where code is going to live and be structured in terms of classes and functions, like that's again not where my expertise is going to be because I'm not exposed to that. the the amount of coverage that I have that I need to be responsible for coming into this team at this part of its life cycle does in my opinion does not make sense for me to go be like I am I know all of this code base. Fortunately with AI tools like if I needed to go find things it would be much faster than it was maybe a few years ago. But so we have we have this technical area, right? We have technical abilities that are something we should talk about. There's going to be uh sort of like project management in terms of prioritization, right?

How do we make sure that things are staying on track, roadblocks, uh dependencies with other teams, whether we're the dependency or they are, how we keep things on on track and timelines, things like that. How do we um partially for this like how do we resource them? And I don't like talking about people as resources. That it always sounds kind of disgusting to me. But in terms of like do we have an our like the right set of skills from our team working on this? Do we need support from partner teams? Um like what does that look like? Right? So overall just like how is it staffed? And uh the reason like resource to me kind of fits here is I'm not just talking about like like who cuz if I'm talking about just people then yeah that feels pretty gross. I'm also talking about like skill sets though.

um skill sets and experiences more generally. And yes, there will be people that align and like slot into that. But anyway, kind of whatever you got the project management side of things. And then I would say there are uh a whole lot and this is one that I think is going to be extremely hard to talk about when we get into like how do you quantifiably measure it? But really like you have um like the the people management. Um, and I would say especially for for engineers like I think like statist I don't have stats for it but I would bet money on it that statistically most of us in in engineering roles right in software engineering we like focusing on the technical part right like we we want to solve problems we're building things like that's why we got into this for the most part right we're very technically minded.

But the the reality is like we're talking about teams, we're talking about humans, we're talking about their careers. These are people. It is people management. So there is so much to the role that's on the people side that I really don't think that some folks think about. And I would say even from my experience working with other engineering managers uh whether they are you know at my level or above I would say I have worked with many of them that really still don't realize it's people management which to me is alarming. So on the people management side of things, some things we have to consider are uh are you helping people grow in their careers. Do you have the right sort of mix of levels across a team, right? From junior up to like your staff, principal, however that looks. Um so different experience levels, different backgrounds, like so different strengths, you know, people coming in that have worked in area X versus area Y.

We want a balance of these things. Um, it's like you can't it doesn't work that you're like I don't know. I don't I never played like sports games, but I had a really one of my best friends growing up loved playing like Madden and NBA and he you know you you go build the team where everyone has all the maxed out stats. Like it's not that's not real. It doesn't work that way. And in fact, in terms of having a healthy team and longevity, you actually need a spread of these things. So, how do you do that? How are you um making your team resilient to they call it the like the the crappy way to call this is like the the bus effect. Uh the other way to call it is like the the lottery effect where if you have a single point of failure, like how are you making sure that you don't have that in terms of knowledge, experience, right?

If only one person on a team is a critical like single point of failure, you need a solution for that because if they leave because they win the lottery, you're screwed. You can see how the bus one kind of ties into that. Um, so you have that you have uh this this idea around like how engaged employees are, right? A lot of people when you're thinking about just the technical side of things, if you think about we're building software and we need to manage projects, it is much easier on the surface to go cool like who who is the best at doing X? How do we get them available to go do this work right away? Right? Always looking at what is the next priority and how do we take the best person for that to slot them in. On paper, that might seem like a a genius move because you're like, I'm hyper optimizing for delivery.

The the tricky thing, like the extra layer here is that like that's also ignoring the people side of things because again, it's not machines. You're not taking something that doesn't have its own, you know, like life, its own career, its own interests. Uh you're you're literally dealing with people. So if you keep, as an example, you have someone who's really good at all of these things and you're like, "Cool, like you just wrapped up like now you're on this one, now you're on this one, now you're on this one." What happens is that that person continues to be siloed into that thing. you're potentially totally neglecting like how burnt out they are from something. Um maybe they don't even have an interest in doing that. They just happen to have the experience doing it. Maybe there's other people on the team that want that. Maybe you need to maybe you're talking about a you know bus or lottery factor here.

Um you you cannot just keep doing that. Yes, it can be effective especially uh for short-term things. So, like, have I done that? Do I do that? Sure. Absolutely. Right. Um, but that's not the go-to method because if you're doing that and ignoring the people side of things, you're going to run into challenges over time. So, making sure that people are engaged in their work and that they feel supported in terms of their career growth, right? All of these things are critical because once you start neglecting them, you're going to have a team that's like, I don't want to be here. So, how do you think that you're going to get an effective team doing great work with high output with great results if you have a bunch of people that are like, "Screw this place. No one cares about me." Right? Like, what about me?

And I think some people have this confused that if you think about your employees and trying to do good by them, right, trying to support them, that that's detracting from being able to deliver effectively. So, it's a balance. It's not how it is. You need to be able to to do both sides of it, right? There are going to be times where you're like, "Oh things are on fire or we really need this." and you kind of have to default back to like, you know, how do we get the, you know, how do we put out the fire? How do we get the person who can do this to jump in here? But if that's your regular mode of operation, uh, good luck to you and your team. So, these are a bunch of different areas. You could break them down into more granular parts, but like I I want to get to the quantitative part because I'm almost at work and we're not even talking about numbers.

Um I think so on let's on the people side right it's not uh a perfect thing to look at but I would say some factors that you might want to measure are like are you having people going through a promotion cadence on your team that make sense I can't tell you what that number is in terms of like the rate at which they're promoted or how many people blah blah blah like are people actually performing well being rewarded for that? Like what does that look like? I think you could try looking at measuring that because and when I say measuring it, I don't mean like I I really don't want to get this conflated. Uh I'm not saying that there is a perfect target and therefore like measure it so that you can you know game the system to hit the perfect target. I'm just saying like as an example, if no one on your team ever gets promoted, probably an issue.

If every, you know, every performance review everyone's promoted, probably an issue. So the the measure is actually like, is there some good balance of people being able to make it through to next level in some like respectable amount of time? You may have individuals on the team that are fast flyers or going slower, right? And like so that's totally normal. Can you like basically when you have those situations like if you're looking at them, does that make sense that that's happening? And then I would ask the question like are you supporting those people properly for the fast flyers? like are they are they getting good challenges, good growth opportunities, right? Like are are you actually aligning with where they want to go in their career? Because they're going they're going and you want to make sure that you can align make sure that they're they're remaining interested, challenges, that kind of challenge, that kind of stuff.

And then for the people going slower, like how are you supporting them, right? Is it is it a skill issue? Is it a like something's going on in their personal life? they need different kind of support challenges within the team. How are you doing that? So you can measure to to try and get an understanding of where you need to invest time. Um we at Microsoft do what's called a signals survey and within that there's a thriving score. So is this a perfect science? Absolutely not. Um but the idea is that there's like an employee satisfaction survey, right? And there's questions that talk about you in your role in your part of the org and your team for your manager. Um, and then there are a few questions within that. Uh, so it's a signal survey, but within that there's like a thriving score. Whoa. Holy.

Come on. Now, my wife would hate to hear me say this, but that person drives like my wife, where they decide, "Hey, my signal's on. That means I'm going." That's not how driving works. Um, you can't put a signal on and just go. You actually need to have space to do it. My wife doesn't agree. Um, so with the signal stuff and the thriving score, it is a way to get like a quantitative way to ensure that your team is engaged, right? The the idea with the thriving questions are like like basically like do you enjoy coming to work and like the challenges that you get to work on like is this a is this a good deal for you? Because if it's not like some something has to change because it's a they're trying to measure like like is this a good spot for you?

Cuz if it's not then like what what do we need to do better? like how do we be better as managers? So they do try to measure like sort of the people side of things. Um it's obviously not quite that straightforward and like the set of questions it's not perfect but conceptually that's the idea. Um, I would say on projects like how do you measure effectiveness or like your you know your your manager side of things here? I would say like do you always miss deadlines for things? Some of that might be outside of your control like just as an example maybe within an organization deadlines are basically always missed. That's like just something that seems to happen categorically across everything.

But is this something where you like always grossly over or underestimate things or is it like oh we we miss the deadline whether it's you could say argue even early or late by like a day 2 days on a six-month project or a week or two weeks on a six-month project or a month on a six-month project that might not be so bad. But like if over time, every time you're doing this, it's like we're always off by like 2x or whatever, like that might be something where you're trying to measure like can I can I actually plan and coordinate this stuff effectively. It's tricky because like I said, there's going to be other factors that aren't just you as a manager that influence that, but that is something that you could look at doing, right?

I would say you know in a perfect world a manager that is you know the best would be able to say okay we have a deadline and uh we're not on track for it so like I am going to engage with all the right stakeholders I'm going to raise awareness of this perfectly at the right time with all the right people and we adjust either what we're doing uh adjust for our dependencies or we adjust the delivery time um and so So that basically things get delivered on an agreed upon time even if it's shifted or uh what's required for your deliverables changes right but again like you're probably noticing that as I'm talking through this you're like how would you ever measure that like I don't I don't know but quantitatively we could start to think about some things like this um on the

technical side like something you might be able to kind of think through is like you know if you are let's use an example of a smaller team right so I'm going to go back in time for myself when I was on smaller teams and contributing to the code base like as a manager I would be one of the in you know in that role I would be the either the most technical and senior person on the team or um on par with someone else or others right? Doesn't just have to be one person. So, when I think about um some of the teams that I manage that way, like that would hold true. I'm not saying like that means I was the best person.

I just mean in terms of like knowledge about the codebase architecture, being able to like you could you could give me tasks on that team that were like ambiguous and challenging and like the most next most senior person on the team I would be able to do it as effectively as them or more effectively. Right? So that means that I'm in a position that I can like if I need to I can focus on delivering like an IC would with a uh you know good output high quality that kind of stuff. How do you measure that? Don't know like if you're looking at the output of your like output I don't just mean like number of tickets again. How do you how do you define this stuff? Um, if you're looking at the productivity, however you choose to measure that of another senior person on the team that maybe you want to use as a baseline, are you able to do similar to that?

It's challenging because like you also have different responsibilities, right? So similar. I'm not saying you have to match it and I'm not saying it's always going to be consistent, but are we talking like a complete world of difference? And the answer might be like yes over time because you're getting more responsibility and stuff. So it's not a great measure, but the point is like can you take on work that is complex that is challenging and work on it just like the other senior folks on the team would. So if I use myself now like I certainly cannot. But if you said hey could you like could you help design a system like the most senior person on the team? My answer is still no because I have uh really really really good principal level engineers. I have really good engineers across the board. But there are more junior engineers that have overall less system design experience than me.

So could I work and design systems that would you know be similar to theirs or better? Probably because I just have more experience in software engineering. Now when it comes to the specifics and going to implement it, absolutely not. They would completely decimate me there because they work in the space hands-on. So how do you measure that? Like I don't I don't really know, but I'm trying to give you examples of things so you could see like how how would I measure that? Um I I don't know a definitive if it's not obvious, I don't know a definitive answer for like how do you actually get a numeric value on all of these things, but those are different things that I might consider if you were like trying to quantify it. So if you wanted the short answer, my opinion is is it possible? I don't think there is a tool or a system that exists categorically across all manager stuff that would give you a good numeric value.

Is it possible to come up with some type of assessment? Maybe not perfectly quantitatively. Sure, I think so. And will that look different depending on the seniority of a manager, the team composition life cycle of what that product or service is. It's going to look different like just is. So hope that helps. I think it's a really great question. I'm sorry that I don't have like a a oneliner answer for it, but I hope the the discussion, the thought around it, the perspective is helpful. And you are absolutely welcome to disagree with anything I've said. I think that's totally fine. These are just my opinions on it as an engineering manager that's been doing it for a little bit. So, if you got more questions, leave them below in the comments. Otherwise, you can go to codekinete.com. You can submit stuff anonymously that way. You can watch the stuff on Spotify.

It's a little bit more delayed, but if that's more convenient, uh, all the code stuff goes on to Spotify just with a lag. And then I got other YouTube channels, so you can check out Dev Leader. Um that is my main channel with AI tutorials, programming tutorials in C. And I would be happy to see you on social media. Take care.