GitHub Copilot Agent Mode Will Do My Pull Requests?!

Transcript is auto-generated and may contain errors.

Hey folks, I'm just headed to the office. Uh, I just got gas. It's actually like it's hot. Um, like I have to turn on AC here. Maybe not that aggressively. There we go. Um, cool. Let's get it done. Uh, I'm going to go to experience devs for a topic today. Um, it's not so much of a let me share my my wisdom kind of thing because I I don't actually have experience with this yet, but I'm I'm personally going to be trying it out and I I literally started trying it out this morning. So, the topic is going to be about um GitHub co-pilot um agent mode. And I don't mean like in your uh VS Code or in Visual Studio itself. I mean like literally they have an agent mode now where you can create uh issues and then assign them to uh like an agent.

And the interesting part about this is like like if I weren't driving a car, I could take my phone right now. I could go make an issue on my phone for a repository I have, assign it to co-pilot, and it will just go try and do it on a pull request for me. Uh, and I think that this is super interesting. Uh, so that's what I'm going to be talking about today. I saw it on Experience Devs, I guess. Um the context and I haven't looked at this but the context is that uh for some of the net stuff from Microsoft I guess they tried turning it on and like assigning issues to it and apparently like I said I haven't seen it but apparently the the code that it's been like putting up in in poll requests is like absolutely terrible.

So, um I haven't gone through the Reddit thread, but the OP updated his post and said like, "I can't believe this blew up." And uh and sort of summarized that everyone was kind of like, you know, making fun of uh the results and stuff. So, uh that's what I'm going to talk about. Friendly reminder, this channel is all about the questions that you submit. If I don't have questions, I go to Reddit generally. So, leave them below in the comments if you're interested or you can send a message to Dev Leader on social media or Nick Causantino on LinkedIn. If you message me, I'll keep you anonymous. You can write as much as you want. Happy to do that. Um, Dev Leader is my main YouTube channel. So, there's Coding tutorials. There is a podcast. There's free resume reviews. I'm trying to like call this out to people that are interested.

I'm a little bit behind cuz I was on vacation and I have uh a couple of things that I'm doing in the evenings this week. I will try to get to them on the weekend to catch up, but I have to spread them out on my main channel because they tank. They're like some of the worst performing content ever. Um, but I don't want to give up on this because I think that it's really valuable. So, I do want to remind you if you want your resume reviewed, go to Dev Leader, watch one of the ré review videos. Um, they don't cost you anything if you want your resume reviewed. Uh, they do cost me time and money because I have to get a video editor to do it. Um, so if you find that it's helpful, all that I ask in return is please try to share um some of the content cuz it helps me tremendously.

Like I I cannot tell you enough how much it helps to have content shared because I'm at the mercy of of YouTube in this case doing whatever it wants, right? If uh if the first batch, the first audience that gets sent my content, if they don't click it, it basically nukes it won't ever get watched. So um makes a big difference when people share stuff. So thank you. Okay, so co-pilot agent mode GitHub. I was talking about this from the IDE perspective recently where I was trying to work with agents and it was a pretty bad experience, right? I still owe everyone a a follow-up video about doing the code commute landing page. Maybe I'll do that this weekend, too. Um, that would be fun to take a break from some of the other coding stuff I have, but um the uh the agent mode stuff hasn't been going so well for me.

It's not that I don't think the concept is cool or that I don't think it'll get there, but um I think from what I've been hearing from people and uh I have another thought just a second, but what I've been hearing from people is probably around either models or uh my take so far is like the context window. Uh it seems like it's getting too much context and it's just like it's all over the place. uh but I changed my subscription on on copilot to get uh pro plus so that I can try this agent mode directly in GitHub and it unlocked other models for me to use. So I'm I tried it in cursor when I tried some of these other like models and didn't seem to make a difference but in in Visual Studio I'll try like clawed and other things as well. So we'll see if that makes a difference.

I'm not totally convinced on it because I tried some things in cursor, didn't seem to help, but we'll see. Point is, so far agent mode stuff has been like, sure, it could probably do some stuff, but the amount of handholding that goes into it is so outrageous that I might as well just do it myself or just talk with chat GPT or talk with co-pilot and then like take the code, copy paste it, update it a little bit, and then just like reask it again because when it's off doing its own thing, goes off the rails. I find like it it's not even I've asked it to do simple stuff and it's just like nope. So, we'll see. I expect that to improve but going into um into the GitHub side of things. I think this is super interesting because this does unlock a bit more um sort of flexibility and usability.

So, uh, I just want to start by saying that I'm not totally convinced the results are going to be good to start. And I think based on this Reddit post, right, I think that's what people are saying is like it's hot trash. And I kind of expect that right now. I have been, like I said, been using agent mode locally and it seems like it's pretty but I I think it's promising. I totally expect that using agents in GitHub where they spin up the environment and stuff to run the agents. I totally expect that's going to be garbage to start with. Um I don't know when I'm looking at this stuff I get kind of pissed off cuz I see so many people talking about like oh like we don't need developers. It's so trivial. I spun up like a multi-million dollar company with like a two-line prompt and like now I live on the beach.

I'm like that's just not real. It's just not. And if you did, if you did, it has literally nothing to do with the software you built and all about how you must have marketed this thing. Like you must be very good at convincing people to give you money, which realistically has almost nothing to do with the software creation part. I I'm just saying, right? Like I've built pretty complex and never got a dime for it. More complex than I could see an LLM spit out, at least at this point. So it's not about like, oh, the LLM just does a better job. No, like you can build stuff, but if people aren't if you're not selling it and marketing it properly, it's not going to matter. So, I just, you know, I I kind of trying things out. I'm not super impressed, but I'm not letting that like deter me, I guess.

This person's taking a picture. Hi, It's so weird. Oh, I guess they're they're also in a white sports car as well, so maybe they're like we're white sports car friends. Anyway, um I don't look at this stuff like, oh, it sucks, so like let's let's on it. I'm like, this stuff sucks, but like I'm going to keep using it. I'm going to keep building up my skill sets with it because I think that it's not going away and it's only getting better. So, I want to be positioned to be able to say, "Hey, as this stuff gets better, I'll be tuning how I'm prompting. I'll be tuning how I'm using the tools." And hopefully, you know, if everything if the investments keep going into this stuff, like it will improve. Will it improve to the point that everyone's claiming or some people are claiming? I don't know.

I mean, realistically, I don't really care. I know it's going to improve. All right. I I totally think it's within grasp that, you know, within a year, within two years, that I'll be able to like work very effectively with agents inside of Visual Studio. And I'm excited for that. Right. Like one of the things that I would love to be able to do is, you know, be designing some type of functionality in chat mode cuz I want to be part of that the creative process. I think it's super fun. I love building Like I want to be doing that and then on the side have a conversation with an agent that's like, "Hi, by the way, we have like three different architectural patterns going on in the codebase. Go refactor this stuff to make it the new pattern and make sure that it's tested." The person that was taking a picture of my car is driving erratic behind me.

It's a I think it's an older woman, so it's kind of confusing, but she's swerving and we're like at a light. Oh, I think she's dancing to music. That's awkward. Please don't do anything stupid as we get on the highway. I'm behind a truck that is going 20 miles an hour getting onto the highway. So, I might have to do something stupid to get around this thing. Come on, buddy. This person's still taking pictures. Like, we're on the on-ramp. She's trying to race me. See her go by. Very weird. I've never never seen like an older older woman try to do that. I think she's an older woman. Maybe she's not. I don't know. Um anyway, now that I don't have to worry about this person driving like an idiot behind me, they're driving like an idiot in front of me. Uh not my problem now.

So with agents, I want to be able to delegate all of the work that I don't want to do. That's important. When I've talked about code bases before, by the way, the primogen has made a like he's made a ton of videos obviously, but he's made a great video where he talked about like realistic expectations of code bases and he's like they all go to It's all just a matter of like how long you can sustain them. And the timelines that he was putting around them like I think would shock some people. But it's like, you know, they're as much as people want to like have perfect designs and stuff like that, it's just not it's not going to last for forever. And the more stuff that you're adding and building, it's going to degrade. even if you're investing into keeping it, you know, in tip-top shape, over time it will degrade.

And one of the I think one of the main reasons this happens, aside from just like the complexity growing over time, is that we're constantly, especially in a business setting, we're constantly faced with with tech debt, right? And at some point we're making decisions where it's like look like yeah we probably need to go rearchitect large parts of this if we want to go get even more velocity or agility out of how we're building things but from a business perspective it truly might not be worth it. Right? It might be we might be at a state where it's okay like we're progressing at like development speed is at an okay pace. We're shipping features and it's gonna frustrate developers. I say this as someone who writes code all the time, but I did did live through a startup that grew to a very large scale.

And sometimes you have to make decisions where it's like, look, it's going to be a little bit slower to develop in, but we're developing fast enough to get the value out and people are paying for it and the business is growing. So like sucks, but that's that's life. if we were positioned to be able to have agents go off and do a lot of this work for us. I think that this could be pretty fascinating. Now, there might be people Okay, let me I want to back up a little bit. I want to I want to touch on this a little bit more. Some people might have heard what I just said and they're like, "Well, the rearchitecting part, like that's the part that I want to do. Like that's the fun part. That's the you were saying, Nick, like you want to go build features and let it go do all this other I want to do the rearchitecture stuff." And like my thought here is you can you do participate in that.

You do lead the design, the rearchitecture, all of this stuff. But when it comes to carrying out the work to go port different parts of the code base over, do all this stuff instead of you being the one who has to go across the entire code base updating things or or whatever else. Like the part that honestly in my opinion probably sucks for big architectural changes. You don't have to go do that. you get to be in the driver's seat to like orchestrate how you want it to happen, which I think is the really cool part of rearchitecting things. So regardless of like what parts you like to focus on, I think that like there's this opportunity that we can be leveraging agents to go do the we don't want to do. So, we can spend more time, you know, I'm not saying you're not going to interact with AI outside of that.

Like, go talk with an a um uh like an LLM and ask like design questions or ask for feedback and like, you know, bouncing ideas back and forth, but we get to participate more in the creative process. That's like for me, that's what I want, whether it's architecture or features. So, I think we're a ways off from that still. That's where I'm really hoping we get to. That's how I would like to be interacting with agents personally. And I say that we're a ways off, but I am I feel very confident that we'll get there. Right. Right now, I don't like comparing like agents and stuff to junior developers because I think it's like a false dichotomy. Like it's not it's not like, oh, agent bad, so therefore it's like junior developer who is also bad and stupid. It's like that's not it's just it's weird.

I don't like doing that. But um the the part that is similar is that when you have junior developers or even new team members when they come into a codebase, they might be building things in a different way, right? Um they might be missing some things and you have to kind of teach them and coach them to go understand these things. Oh, I'm getting very n like I just had a huge wave of uh nausea from from vitamins. Oh, that's very delayed. That's the worst feeling. I don't really uh I don't really drink alcohol anymore, but I've been If you've been drunk before and thrown up from drinking, like it sucks, but at least like you're drunk, so you're like whatever. like nothing really matters right now. But um when you're very sober and feel nauseated, like it's such a disgusting feeling. That'll pass in a couple seconds.

Um sorry, I kind of lost my train of thought there, but the uh the idea being that I think we'll get to where we can be having those interactions with agents. Uh, I think that's probably within the next year or two from from my explanation. Oh, I was talking about juniors and and LLMs, right? Sorry about that. The the idea being that in each of those scenarios, it takes time to correct the behavior or teach or coach. Um, so it's not about it being stupid or it's not like whether it's an LLM or a person, but I think it's like, okay, if the LLM is not doing what you want, how do you prompt it more effectively? How do you choose the right model? How do you give it the right context? Right?

If a junior developer or a new team member is not doing what you expect or living up to um you know, not delivering things the right way, whatever it happens to be, you don't just say, "Well, you must be dumb and like we'll just get rid of you." Like, no, you coach people through it. So it's a you know different tools very similar tools but different to accomplish tasks like we're trying to get features built bugs fixed whatever it happens to be you know um if there if the person doing the work or the thing doing the work is not doing it to your expectations you make adjustments so I feel like we're kind of in that phase where again I don't want to equate it to a junior developer because it's wrong in my eyes but there's uh a lot of uh tuning and refinement that has to go in and at some point it starts having diminishing returns.

Uh, a difference here I feel like is if you have, you know, eager junior developers or eager new team members, when you're like, "Oh, this is going slower. I could have done it myself." The um the the hopefully the the the positive side is that as you're coaching people through it, they get better, right? You're investing less and less time each iteration because they're getting the hang of it. I don't know. I don't I don't feel like with LLMs like that's necessarily the case. I feel like right now it's more about training myself like am I prompting more effectively? uh waiting for, you know, models to become more performant, waiting for the tooling to get better, but I don't feel like if I sit there and keep reprompting like I'm making it smarter or necessarily coaching it, but I feel like I have to do a better job on the prompting.

So okay the the thing that I wanted to focus more on and I spent a lot of time on not it is the sort of the the co-pilot GitHub agent integration right so was talking about agents a lot in general but and how I would like to use them but I think that this setup where you can assign tasks that sounds very much like just sitting down at your desk telling an agent to build stuff. I mean, it's conceptually very much the same idea, but um the usability or the interface of this is very uh interesting to me. So, to give you an example, before I left, I have two repositories um that I asked GitHub Copilot, the agent. So, I I basically made two issues, one in each repository. Um, my prompts probably weren't effective. I've never done it this way, so I don't know a good way to do this yet.

So, made two issues and in each one in each repository and assigned them to Copilot and it started working on them and it's, you know, you can see the like the the session where it's thinking through things. So, I let that start running and I left. I'm driving in a car now. It's running. I think one of them's completed. Maybe both have. Um, it's pretty slow. That's my first piece of feedback is when I say like I told it to go run and left. Like in one case, uh, one I was running one for like over 10 minutes on a a very simple task and it was still going, but I think it was getting caught up on some of the tools, like being kind of stupid. So, who knows? Maybe like there's some weird quirks to go iron out. But um the the the super interesting part is like I'm just detached from it.

It's off doing work. It's not using my computer. It's using some cloud comput somewhere. And at lunch when I'm at work at lunch, I can check my phone. I maybe I can review the the pull request. I think for these ones, I want to pull the code down on my computer, though. So, I can at least review it and give it feedback to go make adjustments. But, I'll probably It's funny. I said that I made videos on this the other day about pulling down code from pull requests. I'm going to pull this code down. I don't trust it at all. Um, so I'm going to do that and but at lunch I can review it. I can give it feedback. if there's something else I can think of, I can go ask it to do it. And this is exciting for me for so many different reasons.

One of them is like I don't know if anyone else has experienced this, but because I build stuff outside of work, okay, there's sometimes during the day or like on the the drive into work, less now when I'm vlogging, but if I'm driving and just kind of thinking sometimes I'm like, "Oh man, like that would be a really cool thing to go try or like uh oh, I just figured out how I should go refactor, redesign this thing or something." And then I'm like, "Okay, well, you know, if I have time after work, I'll do it." Or like, I know I'm busy for the next couple days or something. So, like, don't forget that one. We'll come back to it. But what's really cool is now I might be positioned to be like, I'm going to open my phone. I'm going to go to GitHub. I'm going to open up an issue and just ask co-pilot to go do the first implementation of it, the first draft.

It might be totally wrong, but I almost don't really give a I think it's more interesting that I have someone or something in this case that's done a prototype of something I was thinking of. Like for me, that is that's super cool. That's so valuable. And that's just for like my hobby stuff. I shouldn't say hobby, like I'm I'm specifically thinking about Brand Ghost right now. Brand Ghost is a little bit more than a hobby at this point because we have users, but I mean like it's side projects, right? It's not my my 9 to5. So I am very interested in that. um because the usability of it is so accessible, right? The the alternative to that scenario I just said is like I have to go back home. I have to go or I could file an issue in, you know, Azure DevOps or or GitHub.

I could file an issue and then like go back and read it later or something, but if the code is already like there's a draft of it, man, like that is that's a whole other level. it's just so accessible to be able to do it. So I really like that. Um, another interesting part like a work specific thing is like as an engineering manager at Microsoft, I I I'm not writing code with the team, right? I code every day outside of work. I've worked before Microsoft for eight years at a startup where I wrote code every day with my teams. I don't do that at Microsoft. Uh I suspect there is more of a push for manager like upcoming for managers to be coding. That's not been no one has explic explicitly sent this to me but I suspect it's coming and I'm not afraid of that.

I don't think personally I think it's the absolute wrong um it's the wrong investment in my time to be doing that in my role doesn't make sense to me not at the scale we're at I think in a small team or a startup like hell yeah everyone build uh but at this point the scale we're at to me it's a complete step backwards uh I don't agree with it at personal opinion. You can disagree with it, that's fine. Um, but I suspect this kind of push is coming. The part that I really don't like about that is like in terms of internal tooling setup and that kind of stuff. Like I don't want to have to deal with like developer build issues, like that. It's just another level of headache.

And I already have so many things that I am trying to juggle that the last thing I want to think about is like, oh, there's merge conflicts or oh, um, for some reason I pull down the repo and like now things don't build or like, you know, Visual Studio is not working today for some reason. Like there's a million things that go wrong and then we waste time on. I gotta merge over here. You got to let me in, buddy. Thank you very much. Hey, little car. Speed up. Um, so with that said, like I I feel like it's a distraction from my ability to lead teams, like a huge distraction. It is not in terms of career progression for me 100% backwards. This is all my opinion by the way. Like I said, you're welcome to disagree with it. But uh in terms of scaling my ability to manage and lead people, the number one thing that helped with that was taking my hands off the keyboard.

Um I'm I still remain very technical and this is where I'm going with this is like the nice thing is that if we have this ability to leverage this type of technology at work then if there's simple things then I can go ask cuz I do code reviews right for the team. I'm not going to I'm not going to be or generally I'm not in a position where I'm like telling people like absolutely don't submit this like I know the architecture better than you. Like no like I have senior and principal engineers that can help with that. Um I can understand what's going on. I can ask questions about it but I'm often not like hard gating people on their poll requests. But if there's more trivial things where I'm like, I understand what's going on or I understand what has to happen, but I don't have the repository cloned down.

I don't have the tooling set up. Um, whatever it happens to be, it would be extremely convenient to be able to open an issue, assign it, and then sort of walk the agent through like on the poll request like, "No, go change it this way, go change it that way." And then I don't have to distract my team from the important Like there are I was talking about this in the video I recorded earlier today about um reducing chaos for people, right? This is the kind of thing that I think it's going to be so helpful to have agents to do. Everyone's so worried about their job being replaced by AI, but like I'm looking forward to being able to like accelerate uh no to to being able to give like agents like totally mundane that's like good to have would be helpful in the codebase, but I'm like I can't prioritize that with the team.

There's no way. it absolutely would not make sense to do. But okay, like you know, I can go through the backlog and be like, "Oh, I would cut this." But I'm like, "Maybe I could just assign it like I could assign it to an agent after I go polish up how to go make it usable for an agent. This guy's his turn signal on and just isn't doing it." Um, so I think that would be super cool. Um, I don't I'm trying to think if I have an example off the top of my head that would make sense to even talk about on a video, but I don't I don't think so. Um, yeah, not without kind of getting into too many details, but anyway, I think like I'm excited for this type of thing. Uh, no, I don't look at that just to if people are like, but isn't that isn't the next step just replacing all the developers with that?

Like, no. This is why I kind of talked about the stuff I said earlier is like if if we can give mundane stuff to agents to go do and it can do it like at least reliably or semi-reliably to get it to a point where like it's most of the way there, then we have to just tweak it a little bit and we're good. In my opinion, that frees up so much time for other people, other developers to go do the more important work personally. And you might say, well, if the AI gets better, then aren't we just, you know, isn't it inevitable? Like, I don't know. I I think that we're so far away from being able to just sit back and let AI do the the stuff that, you know, we have developers doing on the trivial stuff. Like it's the kind of work that I might even be using to onboard a developer to a team, right?

Is it going to replace the person who's permanently stuck in an onboarding loop? Like, yeah, I'll say that, you know, the the the thing that you give to a new team member or like a an intern that's just starting where it's like, "Hi, you've never seen the codebase. You're not familiar with our tooling. let's assign you this bug that's like change a typo just so you can walk through like how to clone code, how to build stuff, how to push it up, what the review process looks like, those types of things. Yeah, it's going to replace people that can only do that. Sorry. Um someday will it replace all of us? Who knows? I just like that I that is not even on my radar. But I do think that it's going to be super helpful and convenient to be able to get some things that I would consider like lowhanging fruit that truly just aren't a priority for the team.

That way going back to the video that I recorded before this in terms of uh I was talking about information clarity and sort of workload clarity. I can keep people focused on what's important and then when I have stuff that comes up where it's like I can't I can't justify prioritizing that like I hear what you're saying. Yes, it's valuable and maybe the argument Yeah, but it's it's it's just a quick fix. It's just a quick change. Like like I get it, but when you keep stacking those up, it derails people like crazy. The context switching, right? It's like it's nuts. Come on. What are we doing here? So, yeah, TLDDR. I think that's going to be pretty cool stuff. I think it's probably just pretty pretty garbage right now, but that's okay. I think it'll get better. I think that's it. Otherwise, I'm going to start repeating myself uh because I already do that a lot.

So, I'm going to sign it off here. I'm just stuck in some traffic right by the office. So, I will see you folks next time. Take care. Thanks for watching.