Do We REALLY Need QA Teams Or Can Developers Just Do It?

Transcript is auto-generated and may contain errors.

what is up it is December 26th 2024 and it is about 20 after 7 at night so I'm not commuting right now of course um but because this week is a little bit weird with holidays Christmas all that fun stuff uh boxing day if you're in Canada which I'm not anymore kind of weird no boxing day here means I'm driving a little bit less and then I went to the gym with my wife today so uh I didn't film because I don't think she wants me making Vlogs while we're driving together so I figured I would jump in and try to get one answered from the comments before I um go into tomorrow I think tomorrow I'm going to the gym so I should be able to get two done tomorrow which is kind of cool like being able to do that but I'm not

going to the office so this week I've gone to the office just Monday um so pretty limited but we'll try to get it done still so the topic for today is from web developer Ninja 9220 says appreciate your talks what are your thoughts on dedicated QA versus Dev teams testing themselves do I need to hire a QA team for my Dev team thanks bro um and I I think this is a really cool question um the answer is it depends there we go done uh no so I'll talk through some different perspectives on this from my my own experience um that way you can see the bias that I have in either direction on this but I'll try to see if I can navigate and uh I don't know like I like being being able to share different perspectives even if they're not necessarily something

that aligns my own experience so we'll go through that together friendly reminder that um if you want stuff answered leave a comment below please or look for Dev leader on social media that's my main YouTube channel and the handle I use across all platforms so send me a message uh leave as much detail as you want and I'll keep it Anonymous and that way you can provide more detail so I can answer more effectively because uh I mean I like this question for example but it's it's generic right so if if this person uh web developer ninja 9220 had like a very specific situation that I could try to tailor my answer to it so nothing wrong when it's generic in general that's totally fine but my answers are generally going to be more generic um so uh and a quick reminder too I was letting I've been trying to do more sort of promoting of the the other channel so um on dev leader on Monday's 700 p.m.

Pacific I do live streams and it's actually filmed right here um so if you like my code commute videos but you want the chance to engage and basically ask questions in the chat um definitely try to join 7 p.m. Pacific just go to Dev leader that's my main YouTube channel and uh I generally have a topic that's from my newsletter again I I keep letting people know like if you're like oh man newsletters are dumb I don't like that don't worry I'm not trying to you know trick you into subscribing to it or whatever but like if you want to know the topic that I'll be going through on my live stream then you can go to weekly dodev leer.com with that said my thoughts on dedicated QA versus uh devs testing themselves um okay so I have lived through uh all sorts of variations

of this and I will talk through um different situations that worked well or didn't work well just so you can kind of see um like I said you can understand my bias is so that when I try to form opinions about this um you understand why and where I'm coming from so super high level um like just to kind of put it out there my my preference is that uh to not have dedicated QA teams and this is for uh for building end user software for building um platform software uh I am not a fan of having dedicated QA teams uh and now I will try to explain why and then I will also try to highlight situations where that may not be the case but from my experience I do not prefer to have dedicated QA teams separate from developers so let's get into

that so uh when I used to work at Magnet forensics before uh Microsoft in the earlier days we would hire developers and testers and what would generally happen well in the very beginning what used to happen and this isn't good so if you were listening to what I'm about to say and you're like well that's a bad way to do it you're right um at least in our circumstances it was not good um and maybe people have seen success with this but we certainly uh had to to move away from it but how this used to work is that we would have testers and the testers had a testing team and what would happen is that the developers would basically write code make features or fix bugs and then when they were done they would say great testing team here you go I'm going to

go move on to the next thing and the interaction was very much like that so the idea is that like developers get to focus on making things and testers get to focus on we would hope like not finding bugs but it's their job to find bugs and that's what they're going to do so what would happen is you would base as a as a developer you'd go get your thing done whether it was a bug fix or a a feature you'd kind of toss it over the the metaphorical wall to QA and like basically immediately they would go well this doesn't work and and or Andor some variation of that right because sometimes and many times it was like this feature is just literally broken or this bug fix is or this bug is still not fixed just like outright this would happen a lot

or you would get into circumstances where someone's like okay like it kind of works but like what about this scenario what about that scenario and then you'd have this like conversation where it was like okay like how much of that should we be fixing and like these are good conversations right like it's not just anything that they said like has to go get fixed like we'd have to go have a conversation to say like I mean maybe that was expected Behavior interesting right so maybe uh maybe the things that QA folks are finding like we have a conversation about it and it's like actually like no like this is this actually does make sense that it's this way this actually isn't broken uh maybe there's a misunderstanding maybe we should change the flow of this so that the behavior that we're having is actually uh

more intuitive like it's not just sort of like a binary black and white kind of like thing to go do so those conversations were good but the inefficiencies that it would introduce were ridiculous because you would contact switch like you you're finished as a developer you would contact switch toss it toss it over the wall go work on your next thing so you've switched over you like just start just getting into it and then like you get contacted by Q so you're like basically disrupted like a lot um and then it was like okay well we at one point we were like staggered Sprints so the developers and the testers would kind of work off Sprint from each other and offset by a week they were two we Sprints and that way how it would work is that testers would pick stuff up and you

would already as a developer you'd already be kind of like through a bunch of stuff before they start picking things up and this was even worse because what would happen is that they would pick things up at the beginning of their Sprint and you've already moved on from it because maybe you finished at the beginning of your Sprint and then they would be like hey this doesn't work so now by the time like what do you do do you go fix it right now mid Sprint and change your Sprint commitments because your shit's busted or do you go okay well let's bring that back to our like to us for as the developers and we'll address it in the next Sprint now you have this like like a two- week period so you've spent time building stuff that took x amount of time then you

have this two-e period where it's it's sitting broken it's not actually good and to make it worse it's already in the product it's like not on a feature Branch so it's in the product it's broken according to QA and now two weeks after that you're getting it back to go fix it to go repeat the cycle again so as I say this you're probably like that sounds really stupid how could you ever do that and like that doesn't mean that having a dedicated QA team is bad and I'm not I'm not saying that I'm just saying like this was like a thing that we did when we were growing and it's for like it was for good reason we were trying to optimize things and then we would try it out and we were like in practice this this is not working in our favor

so over a long period of time like there were always testers at Magnet pardon me still like got like residual cough which is pain in the ass um I feel like way better but uh just this residual cough from that cold so we always had testers at Magnet to this day I don't I'm assuming they still have testers um and I'm not I'm not here to say like oh I'm like that's that's a bad thing because if it works it works right I'm not here to tell you like don't do this or always do that that's not my intention I'm just sharing my experience um we had as as magnet was growing we had teams that would sort of gravitate towards different things but most of the time there was dedicated testers on a team and that's already a change because I said dedicated testers

on a team not a dedicated team of testers so one sort of one sort of positive sort of learning that I would take with me uh throughout all like and I will until I have disproven this to myself at some point in the future this will be sort of like a we call it like a tenant of like how I would put together software teams is that if you have roles that you think are important for a team um don't put them on Islands don't separate them you make a team and autonomous unit so that would mean if I had a team of developers and I did want other roles like I wanted testers I would put the testers into the team I would put the product owner or product manager or project manager or whatever combination into the team if we had uh so

at mag at one point we had like techn well and I'm sure they still do like technical writers we had people that would write documentation at one point it was one person and they operated on an island and to no fault of their own like because that's just how it was structured they were a huge bottleneck not because they sucked or they were bad or something like that but they were a huge bottleneck because if they're an important part of what has to get done and there's X number of teams waiting for them they have like they're set up with a different set of priorities now we have all this over head of coordinating priorities it just doesn't make sense um in practice from what I've seen so I would very much um recommend putting roles within teams so that the team can operate autonomously

as soon as you're creating more and more dependencies between teams which you're already going to have when you're building software um and you're contributing to the same like Suite of products and services you're already going to have this like this overhead of working together but um I would try try to minimize that by putting roles together in teams so what I would not do personally if I had my own way I would not put product managers on the product manager team I would not put project managers on the project manager team and testers on the testing team developers on the development team I would make teams and if I had those roles I would embed them as necessary within now there may be situations where depending on the size of the of the team or like the life cycle of where a product or service

is at um or I don't know there's a lot of other factors but something like perhaps I might have a product manager that's able to to work across two teams but again they're not on an island and everyone else is sort of using them but I might span them across a couple teams the same thing could be said about like the the documentation person for example or this other role which I want to talk to you about uh regarding testing so before I get there though I wanted to say that in this Evolution we started to have started to take on some of this structure and we were putting testers on teams now we still had documentation entirely outside of it um product managers I mean project managers came later but product managers were like at some point they were like kind of in the

teams and other points they like became an island so kind of went different directions but testas were on the teams now many teams stuck with this um and I think like they found that it worked well for them um and I I want to as I move forward I'll kind of talk about where I saw challenges or I'll call them inefficiencies and I want to be careful how I say this because as I say it my intention is not to say because the testers were bad or because testing is useless I don't want people to misinterpret what I say so I just want to be careful about how I say it so it's it's none of what I just said it's like I think testing is valuable I think that there is a a sort of a role or a time in place that makes

sense for this stuff so um I think there were some teams that had embedded testers on the team and that worked well for them from their perspective and uh on the teams that I manage we we moved away um mostly from that uh in the LA and I'll it's a bit of okay I'll I'll wait till that one till the very end um but what we started to do was uh a lot more of empowering developers to test and again that's not because we were saying oh testing is useless or the testers are dumb or they don't have it like they don't have the skills or we're better nothing like that we were trying to look at the effectiveness of what the testing was that was getting done and the type of work we were doing and we said look like we have all these

regressions and it doesn't make sense for a human to go click through and try to verify everything so my teams tried to structure code in a way that was more isolated so that we could make changes in one spot not affect everywhere else that was one thing from like an architecture and software perspect like software design perspective but the other thing is and this is almost a forcing function we didn't have like when we were spinning up some of the teams IE we didn't have like people to go staff them very easily so we brought in a tester um and he's actually uh I I have a we haven't done it in a while but I have another podcast called dude wears my code and uh my co-host Graham Harvey is uh you know great friend of mine friend for life uh and we've worked

together on teams and stuff he's now uh I don't know if he's got his VP title but he's a director at least a director of engineering at a startup now so uh he's doing great he's managing teams and stuff but he was our tester and we consciously said and like not like we told Graham we had to do it this way I feel like this was a lot of a lot of Graham coming through and it wasn't like hey Graham we're just going to do what all the other teams are doing or what we've always done we're going to make the stuff we're going to toss it over the wall now it's your problem instead we worked as a unit so we would as developers work with Graham and he was our test strategist now we made sure across everyone in the team and the

things we were building um there's another Nuance to this that I need to introduce when we talk about product owners product owners are like our user proxy okay so I think that testers need to take on this user proxy a lot so they need to act on behalf of the user because I think that it's not sufficient to just say like I'm a tester I'm going to click buttons until things break like yes that is helpful for catching things but like say that's all you did and you weren't thinking about the usability of things and I understand that I'm oversimplifying the role of a tester I'm just to make a point if they're not taking on the user like as a by proxy then they're missing out on tons of it so like there's a lot to be said about about QA right it's like

literally the quality of what we're producing not just is there a bug but is this thing usable is it valuable and we were confident that we could try to optimize this process so not only did we have like our product manager that was our user foxy but at Magnet they did I think this is a really brilliant idea they actually hired on Forensic examiners and investigators into different roles than the company like I said I think it's a genius move so they would have some salespeople that worked in digital forensics right they had trainers that worked in digital forensics they had people that worked sort of like in the product space and with the engineering teams that worked in digital forensics and it was super cool because it meant that we had literal users that were working with us so yes they were a proxy

for the user but like they actually were users so um we would have our product owner so our product manager who acted as the product owner we would also bring in sort of like one of the investigators or examiners that worked for us and they weren't doing investigations they had different roles and they worked for us but they could act as the user proxy we could say hey if you were doing an examination doing an investigation like how would like walk us through like is this usable like kind of take us through this stuff so we had access to these people it was tremendous so this really helped as well with the test strategy which is what um so when I mentioned Graham he was our test strategist so we could get on the same page as a team right and you might not need

every developer on the team I'm talking about like as we're taking on a feature or something like that we'd all get on the same page for it and then it was like cool like now we know from a user's perspective what we want to go build I will go work with the test strategist to talk about as I want to go build this stuff out what what things am I building like where am I touching and what tests am I going to write that are automated so what was really cool about this and like because I love writing coded tests and making sure that I have not just like test coverage but like confidence built up in what I'm in what I'm coding I could go have a conversation with someone like Graham and say hey look here's the stuff we talked about in terms

of usability and I'm going to walk you through the different tests that I wrote that try to cover some of this stuff and how this would work is like Graham was technical enough he wasn't writing code like at work with us but he was technical enough that I could walk him through like a test setup and say Here's Where I'm setting up these things and I could walk him through and he's not going to know every single class and everything else in the code but he would say like have you thought about a scenario like when that service is working or um when we were doing mobile forensics like what happens if that's not supported or or the phone disconnects like these these things were he's thinking through and it's like oh you're right like I don't have an edge case like that so Not

only was he really helpful for going through stuff like that with us but also kind of looking at the tools that we had for testing and being able to say like we don't have automation that like or like a framework that would support this kind of thing so we're starting to notice like a gap like maybe we should pause and like go invest into some of this stuff so we could go have better test coverage then we're not like avoiding these types of tests because we can't build them we actually have something for it so the role of the test strategist became much less about actually testing things and helping us as a team make sure that we could go forward and have a strategy around testing um one of the things that like that was really cool in terms of how we evolved was

like we would continue down this path right and it's it's always a continuous Improvement so we get to the point where like we' you know we're we we show Graham the the new tests and stuff we added um Graham would go try it out so there was some exploratory testing so usually what would happen is like we'd get it kind of working and let him try it right so that he could say like this feels like what we talked about it wasn't so much like oh let me try every Edge case but like let me just get my hands into it and see if this if this is actually kind of flowing like what we described or did we did we maybe have a misalignment so we would get him something early that he could at least try to poke around and and that would

give him some ideas too about edge cases for writing more tests like coded tests so we'd work in this pattern and something that was really cool was that by the time like we just became very efficient at doing this and a lot of the times we'd hand it over to him and it kind of felt like he was doing like a sanity check in the UI and clicking through stuff he would still catch some bugs that way and we got to the point where we're like hey Graham like we think that we can actually write coded tests for some of the UI things that you're seeing and what was awesome about that was that we started to change how we were design it was a desktop based application so we changed some of the way that we were writing code and that way we could

show him a test that would say hey look this test actually says when we go do X like this is how the UI is going to change we can actually check the sizes of things so you don't have to go resizing things and look for weird edge cases or visibility or things being disabled like we can show you this stuff actually working um so again it like changed how we started to to write code so that it we could prove in test um and I think that worked really well now that team like we worked together for a while and I don't want to say that like we didn't need Graham like I think that it helped tremendously but what we did was we essentially were able to instead of having multiple testers on a team we had one person that was a test strategist

that we became so effective testing our own stuff writing the test by like talking with them communicating with them we didn't need like traditional testers on the team going through stuff so again it's not to say that the activity of testing or QA is not valuable it's just that we accomplish it in different ways very connected with our product owner very connected with examiners or investigators because they were literally in with us and then we worked on strategy and it forced us as developers writing the code to make sure that we were always writing code to test for things and I've kind of stuck with that being my mental model is something that worked extremely well I understand that not everyone's going to have the luxury of having like you know your your end user in house so to speak but um that model of

like let's get the developers to actually think about what needs to be tested and make that part of the culture of development that was transformative for us um now there were there was a time after that later on where I took on sort of stood up another team and there was someone that we added to the team that was not quite like a test strategist role and um their role on the team again it was like out ratioed compared to other teams we had one tester and we had multiple developers again and it was kind of this forcing function of like Hey look it's not going to be sustainable for you as the single tester to be like sort of the bottleneck to go find and be responsible for all these things and that's on purpose like we want to make sure like in the

beginning it's probably going to be a little bit rough right things are probably going to feel a little bit slow because you are going to be the bottleneck but that's we're not here to like stress you out or to like make this terrible for you what we want to do is make it a forcing function so the in this team it was unique because a lot of the work we were doing was very I've talked about this before um very repeatable in terms of the the structure of work so there were going to be edge cases all the time but like the bulk of what we were doing was extremely repeatable so we said like because it's repeatable we should be able to codify a lot of what's being tackled so in the beginning it's going to be rough we want you to teach us

teach us as the developers what types of things are you looking for we will start adding those things in we'll make them part of what we're building we will codify what you are trying to do and it worked we would start to like in the beginning yes it was slow and then we'd say okay for the next one I'm going to add this type of test oh we don't have this type of infrastructure let's set it up and then we would do it and then we would find like some Edge case and we're like okay like this is proving why having someone else like second set of eyes looking as valuable great and then oh like there's this other emerging pattern of things that we're looking for and it's catching bugs like great okay developers like let's make this part of like how we're how

we're testing our code and writing writing tests and that worked really well too um so again it was about changing that role of the QA to not be you know uh you're not operating on an island so to this person's original question like dedicated QA or like QA teams like I don't like the idea of it I just don't um I don't like having islands of people it feels for my experience it feels extremely inefficient um to touch on the other side of it I think some of the benefits you get are that you have like-minded people that are thinking about these types of things that can work together but I would accomplish that in a different way and there's different and I'm going to forget the companies and stuff for the different names for this type of stuff but I've heard it called like

guilds at Magnet we had I think we borrowed the term from somewhere else but we had like a guild concept so there was like a testing Guild so we had different teams that had testers on them and there was a testing Guild that would meet periodically and they would talk about testing topics so hey like what framework should we start looking at what styles of testing should we be trying out more so they would meet and it was like cross fun fun and cross team guess it wasn't cross functional it was like pure a pure function meeting cross team uh we had like an architecture Guild where people talk about software architecture so um we would we would set these things up so that people could meet with people that were like-minded focused in these areas to talk about ways forward but the whole point

is that I have found a lot of success in embedding um if you're going to take testers or QA I would embed them in the team person that's my go-to um and ultimately um I think the most effective thing is that you don't you don't separate that as a different role in terms of like someone like a QA or a tester being responsible for the quality or the gatekeeper of it I think that needs to come from the team I think it needs to be built into the culture you need to be having conversations with your product owner um you know if you can be if you don't have access to like a user proxy inhouse right if you're even if you're product owners like they're not a a great representation of the user unfortunately because they're not experienced or something being able to do

things like um beta testing surveys with users um even like outside of work now when I'm building brand ghost like we have Telemetry and analytics built into everything so we do try to talk with our users but we can also measure how users are using the product and if we're like Ah that's weird like we designed it so that users would kind of flow through this way and we're like they're not doing that either they're not doing it and getting a result or they're they're getting the result but they're going around something like okay we're using data so that's more about user experience right but we have Telemetry for errors and exceptions and stuff too so we're able to leverage that on the one team where we had I mentioned Graham already um when we had him as a test strategist there we built in

a ton of analytics for error reporting and I've talked about this team before but um I was so proud of the team because um we had awesome we had an awesome tech support guy um like yeah he's just such such a good hire early on he's just like an absolute Legend of a tech support person I could never never do that role um so kudos to him he was always so calm and I don't understand how like impossibly calm and uh and he like he knew what I was like too cuz I'm like I'm a little bit High Strung there's stuff that I like I absolutely would not say to clients like I would get frustrated and it's like safe place right so um but then you know we'd be talking I'm like okay I know he's got to go back and he's got to

walk through this you know walk through the user walk the user through something sorry and uh so I'm like okay let me get my my thinking cap on like let's talk through this um but yeah like on that team we were able to get error information faster than tech support would get the calls in for it so he would come talk to us about stuff and we'd say we already have a fix for it which is super cool um so again it's not like you know if we had better testing yeah maybe we wouldn't have the escapes but we were we were changing how we were developing software and it allowed us to move incredibly fast we had a hand full of people that were building a product that was competing with a company that had had a monopoly over the entire mobile forensic space

and we were doing a good job so um it worked and it worked very well so um for us we moved and just kind of to summarize we moved away from having dedicated testing teams early on in magnet and then the teams that I managed moved mostly away from having dedicated tester roles in the team um to move more towards how do we address testing strategy just like how you might with software architecture it's like testing architecture and then analytics Telemetry and then uh really engage with your user proxy so either users or product owners so that's my recommendations for this kind of stuff so uh web developer ninja 9220 I hope that helps um I think there's many ways to do this but as I said that's what I found to be most effective in my journey and um in the future if I

am putting teams together that's exactly how I will approach it until I see other data or try something else out if that happens to work better I am happy to adjust so everything I've just said is sort of the culmination of experiments trial and error and that's where I left off with those things so hope that helps thanks so much for watching Remember Dev leader YouTube Channel Mondays at 7.m Pacific for the live stream I'll see you there take care