Should You Be Brutally Honest on Employee Surveys?

Transcript is auto-generated and may contain errors.

Hey folks, I'm just driving to CrossFit, but we're going to go to a submitted question, and this one is about company surveys where they're gauging employee engagement and and various things like that. So, the framing for this one that I'm going to be talking about is primarily going to be at Microsoft. And so, I'm kind of acknowledging this bias up front because I think it's important. I don't want to mislead anyone. So, I do work at Microsoft. I am an engineering manager there. And so at Microsoft there are these companywide surveys. I believe it's across the entire company. It's called a signal survey. And so there's a I don't know who comes up with the questions, but there's a set of questions. I actually think that if I understand correctly, they're doing a bit more randomization in the question selection. So they're not exactly the same thing across everyone, but they uh try to survey to make sure that people uh are engaged.

there's like a a separate thriving score section. Um there uh there's there's a handful of different surveys, but in general, the signal survey focuses on how how much people feel supported, how engaged they are in their work and things that are related to that. And so the question that was submitted was really around like when it comes to submitting things like this as an employee, there's also uh usually on these surveys like a I don't know if it's for every question, but for at least a lot of them a spot where you can put in comments. And so they're fully anonymous. Um and the so if you're writing comments like these are things that would you know be visible not who you are saying it but the comment itself is the text you write so that's going to be visible. So someone was saying like how candid should I be with this you know I want to think that people read this and care.

They were saying they believe their direct manager does. um but you know like what what does this look like throughout different levels of the org. So I'm going to share my perspective on this um kind of walk through um why I think these are incredibly valuable. Uh and as I said the bias here is that like I do work at Microsoft. I am an engineering manager there. And I would say that, you know, I I hope what I'm about to say is applicable to other places, but I I literally I can't even make that claim across all of Microsoft for what it's worth, right? That's just uh what I would hope. So for us in my org, and this actually is for uh the previous team I was on too cuz it's within the the broader org for Microsoft 365. Uh internally, the platform we call it Substrate.

So, I might be saying Substrate as I talk through this just for what it's worth, but I was on a deployment team before. I'm on the routing plane team. This is still all within Substrate in our org. The um you know, from what I hear from our leadership, uh like from our CVP all the way like to my direct manager to me. Uh these signal surveys are insanely important. um they're critical and I say that not not just because like oh you got to say that Nick you're a manager like no I say that because they they truly are. Um and it's tricky because as a manager the way that it works with these surveys is like if you I don't know what the the cut off is uh to be honest there's a certain number of people so I'm just going to make this up but bear with me.

If you had 10 employees that report to you, there's a threshold that uh if you don't have a certain number of people that responded, then you can't actually see your own metrics. It has to bubble up to your manager, if that makes sense. Right? So, if I I if the threshold was five, right? If I only had uh four people that submitted their their signal survey responses, I can't even see the data. my manager could see it in their org like because it's a broader set as long as a total of more than four people submitted. Uh, but it would be completely hidden to me. And that's to help with uh keeping people anonymous because if you're like, hm, like I had one one out of 10 people submitted this signal survey and everything was scored really terribly and there's like one person on the team that I think is maybe like that.

And you know, it's like it's not really it kind of is taking away some of that anonymity, right? So the idea is that after a certain threshold, then you can see. And so it's really important that people submit because otherwise we just don't get that data coming through. Right? So if you you haven't unless you work at Microsoft, you haven't seen one of these surveys, but it's generally like a you know on a scale from whatever to whatever like how much do you agree with this? And it's like essentially a lot of it's how satisfied are you? But with the comments to me this is like the most underutilized part of the whole thing. And I truly wish that more people used it because at least in my previous org and in my current org, and this is over the course of like the almost six years that I've been at Microsoft, it's it's one thing to have a number, right?

It's one thing to say like, you know, in this area of the survey for what qual uh not qualities, for what uh you know, what things we're measuring. Cool. like I'm trending up or down or it's the same. That's helpful to get a signal. But you know what's even more valuable is to have like literally comments that say what is frustrating people, what is like good for people. And of course it would suck to get a survey where it's like every comment is just like this sucks, this sucks, whatever. But while it sucks to read that, it's actually extremely helpful, right? Um the the reality is that uh like this person was saying like does my you know I think my direct manager reads the stuff like I certainly do. I know my direct manager reads the comments. I know that my skip level would go through comments.

It's going to be obviously at his level a lot more service area but they absolutely do. I can't speak for every manager in my you know in my org. Uh, and I would I think I could probably make the the safe assumption that every manager that rolls up to my skip level manager, I I can safely assume that they're reading comments. And I say that because the the culture that we have, at least within the management group, is that we're like hyperfocused on trying to make sure that these things are getting better. And that means that if there are comments like we need to read them, we want to read them. We want to make sure that we're getting the extra detail because sometimes like with any survey, uh some of the questions can be open to interpretation, right? I'm trying to think of one of the good examples.

We had some questions before on like there was some on AI. There was I remember talking with an employee once um cuz I was again following up on you know signals feedback and trying to be like hey like you know based on based on the results you know I'm following up on these areas or like you know wanted to talk to you about these areas obviously not asking you to like be put on the spot and be like tell me what things you want to comment on this stuff. uh more like hey like if this if this is one of my lower areas I want to make sure that I'm following up with the team to figure out how I can do better and that's because if there isn't comments then I need to go find out that information and we were chatting and I

remember you know he was saying that uh like oh yeah like I think for he's trying to think back he's like I think on that question I put whatever but he acknowledged he's like I'm actually not sure like here's what I mean by that but I'm not actually sure if that's how it gets interpreted. Um so it was something along the lines of like I didn't score it a perfect he's like not because I don't or not because I have an issue with it but more like I actually don't know what it would mean to be perfect. So he's like I just kind of put it at this notch and like that's what it is right. Um, so I think people interpret what the questions mean differently. So when they try to answer it, it's not fully consistent.

And maybe on an individual basis or like at my level with a, you know, with sort of a handful of direct reports compared to a whole organization, maybe that little bit of drift doesn't matter so much. But just kind of calling it out. So the the comments though are are critical. Um and I'm going to talk about like sort of two different parts here. Um, one is back to what this person was asking about around like so we have these signals come in like what are we trying to do with the data right like how do we make forward progress and so I think what can happen and this is not unique to signals this is sort of the nature of anything where you have metrics Right. I think I said this in a recent video. It might have been the video that the audio was screwed up.

I can't remember. Um, but in a recent video I was saying like you get what you measure, right? So if the if the way that the questions are framed and sort of the metric that it gives us is such that like like we don't want to be in a spot where all we're doing is making number go up, right? Like sure, make number go up is probably a good proxy for the thing improving, but um the reality is like you we don't want to be in a spot where we're like gaming the system just to make the number go up, right? Like the whole point is that we want to make sure that we're doing the right thing for employees. And that's actually in my opinion why they have a whole separate dedicated section of this signal survey.

It's called thriving because if you were able to sort of game the other parts of the of the survey because you're doing things to make those numbers go up even though you're not delivering on like what is a good experience for uh your employees, that thriving score is not going to be high, right? It's like the way that those questions are specifically designed is like to to oversimplify it. It's like do you like waking up and coming to work because you feel like because you feel like it's an awesome experience because if it's not then like we got work to do as managers. So I do think that what can happen is that when people are trying to I say people as in managers and leadership when they're reviewing the data if there aren't comments to sort of add extra context to things here's what can happen.

We're left interpreting what the what it means. Right? So, not only did everyone who filled out the survey have their own interpretation of the question and how they're answering it, but now we're sitting there looking at it going like, "Okay, so if this number was trending down or it stayed the same despite doing efforts, like why, right? What what is that actually telling us?" And so, we kind of have to speculate. We hypothesize and then we try to come up with uh with plans that we think will help and then kind of bring it to the team to say here's what we're here's what we're hoping is going to drive this forward. But the tricky part is that I think because I think because there is speculation and uh like a hypothesis when there isn't extra context for the metrics. I I don't know how else we do it.

Um, and I I I do think that what can happen in not a malicious way is that we go cool. We think that this based on our interpretation, we think this is the path that's going to make this an improved experience. That's why we try to put it in front of the team to say like, do you guys agree with this? Um, if not, like we need that feedback. uh but ultimately we're trying to take steps to make sure that you know we can address the thing but we have to kind of speculate right if there isn't extra context. So now coming back to the comments, if there are comments, right? So if we can see like, okay, this thing, you know, didn't improve. Uh it's it's not like it's I'm saying everything's bad or anything. Like it could be something's good. It's just the lowest of the good things.

It could be something did regress. But when you have comments and there's specific extra context there, it's like, wait a second, people are talking about I'm just going to overgeneralize, right? there's a question that says uh like how are you finding AI at work or AI at work makes me more productive or I feel like my career goals are aligned with AI like something that could seem very generic if you have people that could be answering this all sorts of different ways right like I've talked about this before where are you building things leveraging AI like you're using your co-pilot CLI or clawed code uh for uh I don't know like writing out features architectural documents actually implementing the code right are you using AI tools MCP servers for debugging are you doing that uh or maybe the other flip side of it is like are you building products and services that have AI features in them are you building stuff that's advancing AI right I think those are very different questions.

And so if we don't have specific questions, like I said, we're kind of left interpreting. So if we have people that leave comments and they're like, "Yes, I use I'm just making this up. We use or I use Copilot uh for like all of my coding changes now." Um, but like I'm not I'm not building anything uh in the product or service that uses AI and that feels like it's holding me back in my career, right? Like that would be a very clear to me at least a very clear thing that's like cool. It's not that we should stop, you know, helping empower people to use their AI tools, but like hey, like PE, some people are actually interested in this other side of AI and software development. Like, okay, so what does that look like, right? We are a platform team. For us, it's a routing plane.

So, we're not going to put like AI on the hot path, right? Like, we're not going to say we need to route your HTTP request. We better go ask the LLM where this should go, right? like that would not be okay because of things like latency. So, we can't do that. But are there other things that we could do, right? Like there's tons of data. Could we use LLMs across the huge data sets to to look for optimizations offline? Like all sorts of things. But that might tell me if I saw comments like that that some people are not getting enough opportunities like that or they don't see the opportunities like that. So the comments can add a ton of context. So yes, they're absolutely read. Yes, they're absolutely valuable. Um I I think the comments I'll I'll say it this way. I would actually rather not have a numeric score and just have comments if people would provide them.

Infinitely more valuable to me, right? I I realize that the the actual survey numeric quantitative value is helpful for trending things, but like the questions change, right? So, it's hard to have an applesto apples comparison between the surveys. I would much rather have the context and go cool like let's go talk about these things. Um the last part that I'm going to talk about is is related to comments and team culture. So I I get that um you know with employee surveys and stuff like this that not everyone has an appetite for for providing comments. Um, sometimes people feel uh like uh distrust or they they feel like um maybe it's a waste of time. Maybe they feel like uh they're not going to be actually anonymous, right? They're going to provide some information and they're like, "Oh, that's going to out me and like I don't want that to jeopardize anything." I think all of these feelings are valid.

People are allowed to feel however they want. So, I can understand that, you know, like why we don't have a 100% of people providing comments on 100% of questions. Like, I don't either, right? I try to leave comments where I can cuz I understand that someone's going to read them, but I could also understand if someone's like, "That makes me uncomfortable." Got it. Okay. Like, I get that the surveys kind of make a weird dynamic that way. So, um I wish they didn't of course, but something else that we do on our team is that we actually have uh one of the one of the principal engineers actually organizes two separate conversations. And I wish they didn't have to be separate conversations, but I think it's a very helpful thing that's happening. And so, there is a separate conversation for just the principal engineers. No management.

principal engineers get together and they talk about um sort of the results of the signal survey and any like any themes and topics that they kind of want to collectively put together and it's kept totally anonymous in the group and managers get summarized notes and the same thing happens that same engineer leads a session with uh everyone that is an IC below principal level and they have a conversation together and again notes are taken and then it's anonymized and summarized for us uh as managers and for us this is you know sort of the hack that we have to be able to get the equivalent of comments and it's this alone I would rather have this than any of the survey because to me this is people talking about you know directly the areas that they're that are working well or are not working well. Again, it's it's fully anonymous and I'm glad that people are able to have those conversations.

You know, the the thing that I I dream to have for a team is that like I was saying, you don't need to have the separate conversations, right? But it's like I understand the world it's not it's not going to be perfect. I have a lot of work to do as a manager, right? to try creating a culture where people all like everyone feels comfortable to share right their feedback their ideas psychological safety I think what ends up happening is of course as a as a team gets bigger right you have more people in a in a room discussing these topics you're naturally going to have some people that are going to be more reserved right there's less comfort speaking up like that there's less opportunity because there's more people um the principles are separate from the others Just because one of the dynamics is

often once principles get talking it's like you know sometimes people will just naturally like well the you know the more senior people are talking I'll kind of I'll back out of it and that's not the goal right the goal is to have everyone discussing and brainstorming and sharing their feedback. Uh so like I I get I get that the implementation of that the way it is helps. It's just like I wish we didn't have to, right? I wish we could just I wish everyone just felt super comfortable sharing um in any forum, but like I said, it's not a that's a non-trivial thing to do. So, uh sharing all of this to highlight that the like when it comes to employee feedback surveys and stuff like this, uh it at least in our org, it is incredibly important. Um I don't know.

I think it's like this is the kind of stuff that I would love to have on a regular basis as a manager to have the feedback so that we don't wait for for surveys and more formal things. Uh I'm I am thankful that we do have a bit of a more of a formal forcing function to get the the feedback. I think that's important. It's just like I wish we didn't have to rely on it, right? I wish uh just wish everyone was super comfortable sharing stuff all the time cuz then we can action it quicker. But yeah, I hope that helps in terms of context. I'm just getting to CrossFit here. Um I I do encourage people to to leave detailed comments if they can.

Um, I do encourage you, you know, if you're anticipating like, oh, I can't wait for the the signal survey to go, you know, talk about something that's really bothering me, I would say like, why don't you do that sooner if you can, man, there's no parking spots, right? I I just want to encourage people to to have those conversations earlier. But yeah, that's my my thoughts on it. Um survey information super valuable more detail valuable. Um it's kind of like uh submitting a question on code commute, right? You could ask a general question and I can try to give you an answer that I think is helpful. But uh the more context is provided, the more the more that I have to work with, right? I can still keep you anonymous. I don't need to know who you are because that part doesn't actually make the difference.

the context in your situation to me makes all the difference. So, as a manager, same idea. The more information I have to go with, the more that I'm like, okay, I can understand where the challenges are or at least I know what things to bring up and talk about and propose ideas around. And then we try to drive progress that way. So, um, hope that answers, but do your surveys. Thank you. Take care.