Claude Code or Copilot CLI? Which Harness Is Superior?!

Transcript is auto-generated and may contain errors.

All right, folks. So, I'm already thinking that the title I use for this video is going to feel a little bit like clickbait, but it's not my intention. My goal here is to talk about the differences in my experience with Cloud Code and Copilot CLI, and I know I know what's going to happen. Yes, I work at Microsoft. You're going to say obviously they're telling you to say this, but they're not. No one's telling me to say anything. So I just want to talk about uh my usage of co-pilot CLI versus clawed code um kind of how things came up and then just yeah my experiences and comparing them in my my development. So uh for a little bit of context yes I do work at Microsoft. No I don't work in any part of Microsoft that has to do with co-pilot CLI or anything. Um, I have been using Cop-lot CLI like, you know, for an overwhelming majority of this year.

Not because I, uh, am sitting here trying to make a claim like, oh, it is, you know, the the best thing that's ever been made and and Claude Code sucks and down with anthropic. It's literally because of credits, right? So, like if I have the ability to use, uh, more AI tokens and things like that, then I'm going to be sticking with that and trying to get what I can out of that. And so, uh, I've been using Copilot CLI, you know, uh, quite heavily. Uh, I I do think it's great. I don't have I don't know. I have obviously like any software. I have things to complain about, but um, it's been a tool I use heavily.

And then the side effect of that is of course I'm going to develop bias for how I work is you know over a longer period of time um definitely like um you know aligning to there's going to be quirks with co-pilot CLI that like I just end up kind of you know they end up becoming habit in my development process. So recently, especially with AI limits and stuff changing, um I found myself last month in a position where I'm like, "Okay, like I need I need more AI tokens." So, um I I'm saying tokens like tokens, credits, like I I need to expand my budget for AI usage. Let's say that. And so when I'm looking at the options, I'm like, I actually don't think that uh I want personally want to pay out of pocket uh for for more co-pilot, but and I was

talking about this in I think a semi-reent video, but with clawed code, I hadn't used like a paid uh paid plan in I don't know like six plus months or something. And so I think things have changed since then. When I'm looking at the limits from like the the Pro Max plan, I'm like, "Okay, I see that it says like 20x, but like what? But what? 20x what? If the limits zero and you 20x it, like it's still zero." So, I'm like, I just don't understand what I'm actually getting out of this. But I said, screw it. Um, I know, you know, people use Claw Code. I've used it. I've enjoyed it in the in the past. And so like there's no way that it's worse at this point, right? So let me pay for the Pro Max plan. Let me do this for a little bit.

Um see how things go and then what I can do from there is try to get an understanding of like what is the 200 bucks a month get me in terms of a weekly budget and things like that. So uh that's what I've done. And so in terms of like usage limits and stuff, yeah, the I think the Pro plan uh the Pro Max plan, sorry, was uh was a good fit. Um I dialed back a lot of my usage and you know started with uh one project and then kind of started seeing like okay I'm not hitting like limits within the day kind of moving through the week and sort of adding in more things throughout the week. Um, and I think it was like the last day of the week or something or like the end like two days before kind of thing where it's like, hey, like you got 10% left your weekly budget.

So, I'm like, okay, in terms of getting my what I'm like what I'm calling my money's worth in terms of token usage in in a week's time frame, like it's pretty good. Like, I'm I'm quite happy with that. I think the alternative compared to to co-pilot would have ended up being you know significantly more. Um so from a you know value perspective I'm like yeah this this makes a lot of sense. Um and uh especially if I compare that to myself trying to set up what was it like free LLM API. I was like trying to see if I could use one of these uh open- source repos that you know you set up a lot of uh uh accounts for the like the trials or the providers that offer like some type of quot like a you know free monthly quota but like the that was just horrific in terms of like ability to develop software like just I can't I can't explain how horrible that was.

Um, I felt like I was going back like 5 years in terms of AI software development and I it's a it's a joke because I wasn't doing AI software development 5 years ago. So it was it was awful and then so you know with claw code not taking a step back like that by any means. So, um, the $200 a month kind of thing, I'm like obviously obviously if I don't have to pay any money, that's ideal, but for 200 bucks a month, I'm getting my money's worth. So, um, that's kind of like on the budget side of things, uh, that seem like actually a good fit. And so, that's why like this month I'm continuing to keep it for sure. Now when I think about my development workflows and how I've been approaching things, um I wanted to talk through like where I was having challenges and and some friction with uh with cloud code and the sort of things to add in here.

Um there are some projects I have where I'm either doing uh you know small features and bug fixes all the way up through uh large features. Um certainly like say on that same type of project, not doing like fully autonomous just like here's a backlog of like ideas, go have fun. Um a lot more targeted. And then with that said too, like it could be uh larger factors where I'm like it's probably a lot more mechanical, but it's not just a search and replace. So like I need I need something that's a little bit more intelligent than search and replace to go through and make things work. So, um I have, you know, Claude doing some things like that and then I have some other repos where uh it definitely is able to go take a backlog of things and work through it. At least this is what I was doing with co-pilot in these situations.

And then the other thing to add in is that uh for what I'm discussing now, these aren't brand new repos. These are all established with with code in them uh shippable things, products or services. and uh and instruction files. So, I want to be clear, I'm not I'm not going from Copilot CLI or I've been tuning instruction files um you know, for months and months and months and then jumping over to claw code and then starting basically with an agent that's not using instruction files. Um, I think I said this in a recent video, but one of the things I got lucky with was that I built some of my repository tooling to basically automatically keep in sync some clawed instructions uh and co-pilot instructions.

So I can keep the co-pilot once as a source of truth and then it can run a script and basically get a conversion to to clawed ones because the the file based like filebased ones that uh that look at the path and try doing matching those are slightly different between cloud code and co-pilot CLI. So anyway, um these repos have that for what it's worth. And so I found that the sort of uh autopilot mode and co-pilot CLI, that's that's different than auto mode in clawed code. So if I were to draw a comparison, auto mode in claw code is very comparable to uh like / yolo in copilot CLI. Uh, even auto mode though has like a bunch of restrictions that were I can understand why they have them in place because there's still some things where you're like yeah like you know you probably realistically should you be giving your agent this kind of thing and the answer is like probably not.

So, I think they've probably done overall a good job in in some limiting things, but I found that if I'm like basically if I'm saying yolo or I'm saying auto mode, what I'm genuinely expecting is that I don't have to approve anything. I don't want to have to do that. And that's because what I'm setting the agent up to do is within the realm of like what I'm expecting it to do, right? And that means, for example, just to give you something kind of uh made up on the spot, right? If if a a PR is going through CI and it's stuck for some reason, like the triggering is not working, whatever, and for some reason it needs admitt approval to like adjust it just so it can unstick it. I just wanted to do it. Like, please just do it.

If we're if I'm going to sleep and it's going to be 8 hours until I'm back, I don't want to come back after 8 hours and see after 30 minutes you were like, oops. like couldn't do anything. Just just do it, please. Um, and I know that for other people that might be something where they're like wildly uncomfortable with. And I get it. I'm not saying that Clawude Code is doing the wrong thing here. I'm simply saying that like that's not what I was hoping for. That's all. And like for anyone who's watching this and you are a very, you know, avid Claude code user, I'm sure you could tell me in, you know, in one sentence in a comment like how to go fix this and not worry about it. And that's not my point. I'm not saying like this is something I'll never recover from.

I'm just saying it's not what I expected. and like uh that kind of uh discovery along the way like it was not obvious what was happening until um it became pretty apparent there were some things like it was just not going to do in auto mode. So then I was like, okay, well, how do I get the same effect as uh autopilot in Copilot CLI's cloud code? And I guess, and again, if you're watching this and you're an avid Cloud Code user, please feel free to make fun of me, but I'm assuming in Cloud Code, this is the slashgoal um feature where you can set a goal and just kind of like let it work. So after discovering this, it seemed to work better in terms of some of the things that I'm building where I have a backlog and you know, I'm I'm letting the agent kind of work through.

Um the prompt limit for slash goal seems annoying and stupid uh to me personally. It's limited to 4,000 characters. And I'm like, I So, you're telling me if I have a prompt that's 4,01 characters, we just can't use the autopilot feature? Like, the I don't know. Something seems really bizarre to me about this, but I'm sure uh lots of smart people building clawed code have a good reason for this. Uh I just I don't know what it is. And that's not facitious, by the way. like I genuinely believe there's a reason and I I genuinely don't know what it is. Um so the slash goal thing helped a lot but I also couldn't figure out when it would get interrupted. So let me explain. Uh slash goal was helping to keep it more autonomous. Good cuz that's what I was wanting. But um say across two different sessions, one of them while it has a /go goal active, I could ask it questions.

And so like while it's working, it would kind of stop the specific thing it's doing, answer some questions for me, and then just get back to work because it has a goal and it keeps running. And in other cases, even if it was purelyformational to to ask it a question, it would it would just stop, but the goal would still be set. It's just that it wasn't triggering the schedule to keep up with the goal anymore. So, there were times where I had a slash goal running, would ask a question, like I'm popping into the session to be like, "Hey, what's up with this?" and then and then go back to check the session later and it hadn't done anything for hours because I had asked it something with slash goal still set. So, um just some weird kind of behavior like that that I wasn't expecting.

And from again just my experience with co-pilot with the autopilot mode. So/ yolo plus autopilot, I can ask it stuff while it's running and then like I I know it's just going to keep going on autopilot from my experience. Not have that weird stopping behavior. So didn't didn't like that. Um I mean still don't like that. It's kind of weird. Uh, but the I think that I want to talk about like the number one thing that's kind of bothering me about cloud code and this could be a model thing. I'm just I've let uh cloud code kind of just do its own automatic model selection. Um, and maybe that's why, right? maybe when I'm talking about uh co-pilot CLI if I'm actively picking you know say GPT56 or uh 61 now um if I'm doing that and cloud code is on auto maybe what's happening

is that there's too many examples where it's just using worse models right like it's actively choosing to use worse ones for for budgeting purposes and then the side effect is maybe there's more cases where like hey a a better model would have been more useful. I don't know, maybe that's what's happening. But there's a behavior that I see Claude Code doing, at least in my examples, right? I'm not my goal here is not to attack Claude Code. I'm just talking about my experience with it. Um, and the best way I can explain this is like like the the chat GPT factor. And I don't know when this started happening with chat GPT, but um if you were to go to chat GPT now, if you haven't in a while, if you ask it a question, you say like, "Hey, I want to know about X," it will go do whatever it needs to do to tell you about X.

And then what it does is basically has a like a call to action or a followup at the end that is always structured to basically get you to do more with it. Always. So it might say like, "Oh, here's the answer to to X that you wanted to know, but like there's also this other really interesting thing and this other really interesting thing over here that you probably want to know about. would you like me to tell you about these super important things? And then you see that and you're like, well, well, yeah. Like, I mean, you said it's super important and it also answers this question like, of course, I want to see that. So, then you say like, yes, chat GBT, tell me. And then it's like, no problem. Here's all this really good information. And then blah blah blah.

And then at the end, it's like, but also, did you know that there's this really secret thing that you probably want to know about or I could point you to this example that I know is going to be your next follow-up question. this is super critical or this would be very helpful and then once again you're like yeah like why the hell did you already tell me this like of course tell me and it keeps you kind of stuck going like well give me one more give me one more cuz I'm I'm trying to get something understood right um I'm noticing Claude doing a little bit of this and so what I mean by that is it would like even if it's building out a feature It would be like, I built out this feature and everything worked, but like um and we're ready to merge, but like just just a heads up, like I also cheated on this test.

And it's like, well, wait, what do you mean like you also cheated on this test and like things are good to merge, but what it like? It's clearly not like and why are you telling me now? or it would build out something and then say like at the end like oh yeah but like there's a gap in the design after it's been like 2 hours and it's doing it in a way that like the chat GPT conversations were doing it where it's like obviously I need to ask you a follow-up question here because this is the most ridiculous situation you've put me in like I need to know or have something fixed like obviously ly and I think that's been one of my biggest frustrations with it because it is across everything. It's like also found this glaring hole. There's a huge bug here and like like what why are we talking about this at this point?

Um and also like if it was directly related to the work you're doing like you introduced it then you should have been like you should have fixed it. You can't just say like, I built this feature and introduce this terrible bug, but it's good to merge now. Like, no, it's not, Mr. Claude Code. It's It's not. And I shouldn't have to like sign off on you fixing a bug that you just made. Um, so anyway, I think that's been like the most annoying thing if I have to be honest with you. Now, with all that said, will I continue to use Cloud Code this month? 100%. Yes. So, I'm going to be balancing my budget between C-Pilot CLI, Claude Code. I think there's a set of projects that I am more willing to use Cloud Code on. And after going through it for a couple of weeks, there's just a set of projects where I'm getting too frustrated using it.

And I won't. And I think that's totally fine because I don't think that Cloud Code is a bad product by any means. I don't not sitting here saying Copilot CLI is universally better. I think my workflow has adapted to C-Pilot CLI. It works really well for me. I'm happy with that and I think cloud code is absolutely great and I think that there's different use cases in terms of my workflow that are going to fit better. So curious to hear from you. I'm assuming a lot more of you are cloud code users because um I know Microsoft makes co-pilot CLI and Microsoft is you know we're evil I guess according to the whole world. So um would love to hear from you on what on what you use um and why. I think that's always interesting and this will be interesting to look back on years from now to see what uh what people were using today versus what we're using in the future.

So with that said uh other than that if you're interested in having questions answered related to software engineering career development leave those below in the comments as well or you can go to codecute.com submit questions anonymously and then I make video responses for you and other people on this channel so you get to watch a video like this about your topic. So thanks for watching see you in the next one. Oh, it didn't turn off. This is awkward.