We Switched To Vercel's Eve AI Agent Framework

Transcript is auto-generated and may contain errors.

Hey folks, we're going to do a little AI discussion. I've been talking about some stuff like at work and things I'm trying. I'm going to go back to a little bit of uh coding projects like at home and what that looks like and how that's evolving. So, um where to even start bunch of things. So I think one of the things I want to touch on is I'll talk about brand ghost and we have uh we have some like agentic flows that we have for working with content and things like that. And so a lot of this was things that I like in terms of building them and prototyping them and all that really just like working with co-pilot over time, right? Like uh having the conversations, doing the things, being like, "Okay, this is the repeatable thing I want to do." As Yeah.

I mean, it's it's kind of funny to think like how fast this stuff changes, but like as things like skills and everything else are evolving and uh just becoming a lot more common. Okay. So, like getting all this stuff wrapped up so it's not it's not just a handful of prompts I run now. It's skills that are saved and the skills link to each other and and the models are better and the harness is better. you get all this cool stuff that keeps evolving. And so a lot of what I was putting together um really was like first class in something like I mean I don't uh be first class in cloud code but first class in copilot CLI which is what I primarily use. So really just to say like something like the harness itself is the powerhouse and um makes it super simple for me to kind of take the logical flow of things that I want and uh make it repeatable.

So over time as I'm doing that the the challenge for me has been okay well I can't I can't ship my computer right like if I need to have it in the product I need to have uh something that can effectively run the same set of steps. So what I've been doing for months is is trying to recreate that in the same way. And originally some of this was like bits and pieces with semantic kernel and then we were getting uh Microsoft agent framework and again so I'm a primarily a C developer. of this stuff is like like I'm going to latch on to that that's in the Microsoft ecosystem like makes sense for me to to try and start with.

And so the even as I'm saying this I I should have a disclaimer like at this point in time maybe I haven't updated Microsoft agent framework SDK stuff to like a a later version but uh I am certainly unable to drop in to Microsoft agent framework just like my set of skills directly and have it work in a really like in a harness that I would expect like co-pilot pilot CLI is like I simply just don't have that. And I know that there is now the co-pilot SDK. There's a whole bunch of stuff that continues to evolve, but I don't have at my fingertips unless I'm missing some very recent update or something. Um something that just lets me take what I'm doing in Code Pilot CLI and and make it reality from my C code. it feels extremely um divergent. And so again, I know like Microsoft Agent Framework even weeks andor months ago could load skills, but I I'm talking about like truly having like this harness equivalent.

So, I've been trying to recreate these skills that I've essentially had, you know, working and refining for for months now into into C. And I'm just like I'm so frustrated and disappointed with what that process has been like. Um, I mean, we're talking about having sets sets of skills that uh are across a couple of files, right? They're literally like lang English explaining the flow of things. Um, you know, a couple of uh scripts and tools and integrations and stuff, but primarily just driven by these uh these markdown files and so a human could read them pretty straightforward. They're only more complicated because I was uh trying to repurpose them across a couple of personal projects. But like I introduce that complication otherwise they're they're quite straightforward and the the C code to try and facilitate that is like just an absolute dumpster fire. Um I can't I can't explain how complex and convoluted the code has continued to grow because of it.

And so, um, I mean, having something like eval frameworks and things like that to be able to go run, um, the flow has been, uh, truly like a necessary thing because what would happen is I would change parts of the C code that run some of these stages and the whole thing would crumble. So, okay, now there's an eval um set of scenarios that can run on one of the stages in this flow and then I have some scenarios that go over multiple like say an end to end or across um you know a few stages connected together where I'm like I think there could be some some focused interactions I want to look at here. Uh so tr truly treating it like you know unit and functional test kind of thing and that's been helpful but it's just like the the amount of time and effort and complexity in terms of the code that it's creating just to be able to to do that is is just so unsustainable.

I can't I can't continue to go down this path. So um we were having a bit of a a discussion on this the other day. I was like, "Hey, like I'm I'm making progress, but like it feels to me it feels so outrageous that I've had something that's been working um outside of this 4 months. Um I see people spinning up all kinds of agentic things and I'm like, I'm clearly doing something wrong. Like I am the problem here." So, um it's this balance between like how much of the agentic flow is allowed to just be totally flexible doing agentic things and how much um structure is needed. And so I think one of the biggest issues is that when it's being ported over to C the way I was doing it was that if each stage becomes more more structured at its boundaries um like in theory that to me sounds like a good thing.

This person's really got to let me in here. This is nuts. Um in in theory that sounds like a good thing because you're like hey between these two things make sure the shape of data is right and no one's screwing up. Um, but I think in practice it's making it more brittle than than it needs to be. Like the the harnesses seem to be so good at just like correcting these screw-ups when they happen that like you don't need to like have convoluted checks and and stupid like edge case scenarios covered in all these different ways like you would if you had like structured code. So, uh, it's really just like having all of that structure ends up adding so much complexity with edge cases to be checked for um, in a really deterministic way, which again we're I I'm very used to doing and I think many of us are used to doing that, like you know, sanitize your data and stuff like that in and out.

Uh, but, you know, it be it just makes it so brittle. So anyway, um we played around with a couple of things historically like we had um some stuff we were trying in temporal. Um I think Strands was another one we were going to check out, but we were poking around a little bit the other day and we came to the conclusion that the um the Verscel offering called Eve um is uh Sorry, I had to stop. Yeah, not a chance, buddy. Um, I had to stop at the the lights cuz the the ramp is metered. And so in the carpool lane, which I'm not in, you're allowed to go past. And so I had to stop. I stopped. And then the person who was in the carpool lane was uh was trying to fly past me to kind of they would have ended up like cutting me off at the speed they were going.

So um we avoided that. Uh what was I saying? Oh, so we're looking at Eve. Eve seems to be a pretty good fit for us. um still exploring it, but um I'll I'll tell you why it seems like it's such a promising thing. So, I was just giving you a high level of like why it's been so frustrating trying to port some of the C stuff over or stuff over to C and having this structure and like basically I'm recreating parts of skills with I don't know like Microsoft agent framework chunks and it's like just terrible. We we literally dropped into Eve um by a set of skills. tiny little bit of cleanup just because of uh disorganization on my part. Uh so drop in the skills, deploy it, hit a hit an endpoint and we had uh basically my skills running end to end for for one one of the examples of them.

And uh we're talking about like taking something that we've had zero experience doing like never used Eve like we just kind of read about it said hey this looks like it's a pattern that like seems to fit our needs. We use Versell a lot like why why not right and uh I would say yeah within an hour just had something working. Uh so uh does that mean it solves all of the problems? I don't know. But I think that's uh going to be something I'm trying out more because I I I'll kind of say it this way. I I love the idea of taking some of these like what seem to me like very open-ended like agentic flows which are really just like whether it's co-pilot or claude running a bunch of skills in a terminal like it's powerful. Um I use it all the time.

I have some stuff on schedules now that runs and it's literally just, you know, some skills that are used and it it works. But part of me is like I haven't yet accepted that in some type of larger scale system that I can rely on that. I'm for some of the stuff I'm running on my machine or whatever, I'm like, it's my problem. If it doesn't work, I can go address it. I can fix it. I can debug it. That's fine. Um, but part of me is just so nervous to have like super open-ended flows. Like I'm I'm have to have a lot of trust that the LLM's just going to do what I what I hoped. Um, and so yeah, I I think it's this like lack of trust and that that's going to be I think closed with more uh more experience in the space for me.

uh more exposure to different things, the models getting better, the tech around it getting better, the tooling getting better. Like I think it's it's just heading in that direction. Uh but with that said, I I love the idea of having like I don't know like a a structured boundary that I could easily integrate with. Like my hope with Microsoft agent framework was um was really what I was just telling you with this uh versel offering called Eve where I could just like say literally here's what I'm using in co-pilot take these and like I want a car interface to that. Now, as I say that, um I do know they have a co-pilot SDK. Um and so you can literally run like a headless co-pilot CLI. Um I I just don't I don't know if that's intended for for production use.

Um, like who are is micro maybe they are uh I work at Microsoft and I don't know this but like is there a like a containerized offering because I'm not just going to run a co-pilot CLI instance like on my server in production and like how does that scale like there's so many questions that I don't want to have to go solve that way but I want the behavior of that uh very easily accessible. So again, I'm saying all this right now and by the time you know if I were to pull over, open up my browser and go search, maybe I've just had my head down for long enough and I kind of I kind of missed when it was released or something. That's how fast this stuff goes. But anyway, we're trying out the Verscell thing.

It's looking very promising and sort of uh my next steps with that are just making sure I can wire up uh an evaluation framework to it so that uh that from day one I have confidence as we're changing things like is it going in the right direction? It's one of the the big scary risky things that I I kind of arrived at too late with the C equivalent I had which was like things seem to be kind of mostly working. So like let's put an eval framework on them now and then I don't know like is the eval scenario like actually good? Is it passing because it's not grading things properly? Like I have so many questions about like the quality of what I what I introduced on on the evaluations themselves. So um yeah, I don't know. I think having it from from day one will be helpful.

And I have the none of this stuff is like a waste, right? I have the experience of having gone through it and I can say like these are the parts of this workflow that make me nervous. These are the parts uh that are edge cases that I think we need to cover. So I feel I feel like it's not a waste. Yeah, it's like more more work. Sure. Um but I'm going to have AI help me build it anyway. But I think the knowledge that comes along with that's going to be super helpful. So I will keep people posted on what our experience is like using Eve, but it looks pretty cool. Um, otherwise I think uh another sort of uh it's I mean directly related but for a while I've been talking about this dependency injection uh library I have called Neler that just helps me organize registering things, scanning assemblies for dependency injection into uh ways that I like building plugins.

uh a lot of that repository was getting uh a lot of like AI integration so that I could basically have um this framework set up to work really well with Microsoft agent framework. Uh I was doing some reporting out to Langfuse, so putting some some things in for that. And then I'm just kind of realizing like I have basically hijacked this one entire repository for uh for like some of these AI integrations. So um pretty cool. Like in the past this would have been like a really frustrating realization to be like oh my god like who the who the hell is going to clean up all this code now? Like it would almost be an event for me personally that would feel like so overwhelming that I'm like just frustrated and don't want to deal with any of it anymore. Like oh I I must have screwed up.

I can't believe I screwed up this bad. Like screw it. Like I just not interested. Too much work. Like so disengaged, demotivated from it. So when I had this realization I was kind of getting to a checkpoint working with co-pilot. I just stopped and said like if we look at the repository now, how much of this is still like set up to be dependency injection focused and how much did we like deviate into AI integrations? And then my next question was like is there like a like a basically like a clear split between these things? And uh good news it said like almost a perfect clean split. It's like there's like one spot in the code that like um there's probably a bit of like overlap in terms of uh who owns what and it was like basically everything else could immediately move out. Um and then like if we can just prove that this other piece can be like basically uh inverted and pulled out then then you're in the clear.

So I was like okay that's good news. And again, this is the kind of thing where previously I would have had to go and do that investigation across like all of this code. I'd have to go do that. And then after investigating, if I realized it was doable, I'd still have to go do it. And I don't want to do it. That's just like the most for me the most boring kind of work. It's necessary, but I'm like, I don't want to do that. Um, so was able to get the clarification from co-pilot, was able to um have it come up with a plan to do it and it's uh today it's splitting it out. So like as I'm driving right now, it's uh it's already split it out to a new repo and then it's just doing some of the final like CI/CD integration pieces that I had for like releasing uh like a documentation site and all that.

So, I mean, I'm just sharing this because this is one of the things that in the past I would have had projects that probably reach a point like this, like, oh no, I I now that I'm taking a step back, I realize maybe some of this went too far in one direction and I just give up, right? Just going to give up. it's like it's too much to salvage or it feels like I don't know maybe maybe it's just like how my brain works but I'm like it's so overwhelming to think about addressing it that I'm like just go on to something else and so now um just really cool to have this ability to say like nope like it's fine like AI will go do the the boring work um so that's that's pretty cool so needler is now split uh into what needler originally was which is just dependency injection stuff.

And um the other part that's being pulled out is just called Foundry and that's going to be all of my C like uh integration points. So for example, one of the most recent things was like uh like an XUnit evaluation harness and then being able to publish the results into Langfuse into the the Microsoft what is it called? Microsoft AI extensions or something. Um, and then it's you can basically build your own um sort of like integration onto that using uh this like sort of needler harness for evaluations. So like that's that's something that came out of all of that and it's reusable. So will anyone use it? Does anyone want it? I don't know. But like it's been a fun building experiment and it's helped me along the way, right? It has been used as I've been doing things. Um, and maybe Maybe with some of the work now going to EVE, maybe that will be less important, less of a focus, but again, it's not wasteful.

Um, and it's free and it's anyone can use it. So, um, I also talked the other day about just like me trying to build more things and have them available. You can either sell things or have them open source. So I just did another little quick repository for um for running like local workers. Uh so I can set up a docker handful of Docker containers um like with a script, right? It it points at a repository. Um you can set up these local runners to there's no parking spots. The hell and then you can scale them up and down and stuff. So, I was noticing that I was just chewing through a lot of uh GitHub actions just from building stuff with AI. Um, it's funny. My my limitation now was not tokens because I get lots of tokens, but it's uh action minutes and so because those aren't paid for.

I have to pay for them. So, I realized I have like machines at home or whatever that were kind of sitting idle. Like I have an older laptop. It's like a It's a nice Alienware laptop, but like I'm just not using it. So, my Alienware laptop is now my my server farm. So, yeah, I I released something called Pit Crew, which is just supposed to be like a really lightweight way to stand up uh like local worker nodes. Again, this isn't breakthrough anything. I'm just sharing this because uh number one, if you're like, "Hey, that sounds useful." like now you know, great. Um, I'm not claiming I'm the first one to do it or anything like that. Not by any means, but the point of sharing this is more that these are just things that I'm encountering while I'm de developing stuff. And instead of being like stuck or having to go spend a ton of time doing it, I have the idea.

I talk with AI about it and like it's either there's some obvious thing I can go take that's open source and just go use it and lean into it or I'm like sometimes sometimes that research doesn't even happen because I'm just kind of in the zone doing something and maybe that's kind of how Pit Crew came about. Um I think I I think on Pit Crew I I researched a little bit around like local runners. Um, but I don't think I found anything that that lined up for how I wanted to do it. So, it kind of just evolved into some little standalone thing. So, Pit Crew is available. I've talked about Narnia before. I've been putting more work into Narnia, um, which is my my web UI for your co-pilot sessions, uh, just managing being able to restore them, like that kind of thing. Uh, what else?

Oh, and then I guess uh I have a couple of like side little private projects or whatever that maybe won't go anywhere. I talked about one that's like a personal uh assistant for um for basic but my my point of using it is going to be like interacting with GitHub through voice. So I haven't tried it uh on a phone yet because I haven't put it through anything that would build for a phone. Uh, but it's built to be multiplatform and uh I've used it on my computer just to to test it. Um, looks like trash, but I don't think it's really going to have a much of a UI. The whole point is that um I have a would have a voice assistant that's basically uh driving co-pilot and then can uh like basically go navigate uh GitHub for me and so create issues like check status of things, assign them.

Um, so what I'm really hoping to do is when that is all working that one of these code commute videos I'll do when I've proven it works is like I will have this app running and I'll make a video where when I'm driving to CrossFit, which I just arrived at uh CrossFit or work or something like that, I'll make a video where I'm just working with uh the app is called Laya. My my first dog's name is Laya and she's getting old. So, I wanted something to uh to build and kind of name after her. So, uh this app Laya will um will be what I use. I'll show it when I'm driving because there's no UI, right? It's just a conversation. So, I can say like, "Hey, can you check the status of this repository for me? Any open issues?

Are these pull requests like are they ready to go in?" Um, and because it's GitHub Copilot, I can ask it things like, "Hey, I want you to go review this and then answer back to me like when you find things, right?" I want to have it in a spot where it's very easy to use that way. Um, and if that is something that works, then um, I'll probably figure out like a release model for that. I'll probably I mean, there's a Everyone's got voice assistance. I'm not I said this before, my goal is not to to tell you I'm coming up with something that's never been done. And by all means, if you're hearing this and you're like, I'm going to build that, please do. uh I'll probably sell it um so that it'll be either free to do some version of it or you know some cheap subscription or for anything that's hosted that I need to run because right now it's all kind of standalone.

But if I need to run something then um I'll probably just you know get the service and mark it up a little bit. So, like, hey, get the app for free, but if you want the premium features, I'm just giving you, you know, a bit of a premium charge on top uh so that I can kind of scrape a little bit off of the top of whatever the providers are charging. Um, so like for example, if I need to use uh like a hosted model for text to speech and speech to text, if that's whatever cost, I'll just bump it up a little bit and then people have to pay that. Uh, I have Yeah, I think the other stuff I'm doing is kind of kind of private right now. Um, but I'll talk more about them as I as I get built out. Oh, um, I talked about agent orchestration and stuff like that.

So, I I took a break from that for a little bit, but just kind of spinning it back up cuz I realized it was pretty close to being usable, but like I'm not dog fooding it yet. So, it feels kind of stupid to have like an agent orchestration system that I'm not using on itself. So, I just had a quick conversation with co-pilot yesterday and said, "What do we need to do to get in shape for that?" and then tried dog fooding a little bit and realized like it's not working. And it's not working because the agents can't do the work. It's not working because it's just getting like it's getting stuck on some stuff or when it does get stuck there's no visibility. Um so I'm going to try to close out some of those and then my goal because this will be the proof that it's working the way I want.

And the goal is like if I'm thinking it's called Maestro. So if I'm like hey the the dashboard in Maestro like I want it to look some other way I want to see something else or I want to have this feature I can make it in GitHub and then it just gets picked up by Maestro to go do or I can say I want to have like a maybe like a a skill is the best way to do this or it's maybe more first class in Maestro to be able to say like I want I have an idea around something like basically Uh, I don't know. Like I want to give it sort of the the minimum amount of my own creative input and then have it go off and make the work item that might be hey we have to go research all this stuff first and then it will assign it and we'll get done and it will go create more work items based on the research.

So some of that might be hey humans got to review this. Uh, in other cases, I might say, just to give you an example, I want to have some type of functionality in Maestro, right? Let's talk about the UI. Maybe that's the easiest thing to to like not care about. Um, I want something that I can do in the UI or I want to see stats in the UI about all my agents and stuff. And like in my head, if you were to ask me right now what I want to see, I don't know, surprise me basically. So, I would want to be in a position where I could tell co-pilot like uh either I go make the issue with as little information as possible or I say to co-pilot go make me an issue for maestro that uh adds uh agent stats to a

dashboard and then give it some indication about the level of complexity and breadth that I'm working or looking for and then rely on it because it has the capability to do so like use spec kit go build something out that's like really robust for this. But if I tell it a level of complexity, it will either take what I say and just do something minimum like, hey, here's the number of running agents on a dashboard, right? I give it very little information and little uh level of complexity and I don't know what the right bounds are for this or I give it the opposite with a little bit of a still a little bit of detail. So, hey, I want, you know, agent uh stats on a dashboard. And again, level of complexity set to high as an example. I would expect that maestro goes and does the research.

It goes and uses spec kit. It goes and splits out the work items and then it actually automatically orchestrates. Like that is the whole point of where this is supposed to get to. So, um, yeah, I think it's cool because if it's pointed at itself, then it should be able to iterate on itself. Um, and right now it's not able to do that. Um, it picks up work, the workers are getting stuck. Um, they're getting stuck and then I can't tell what they're stuck on kind of thing. Um, so I think there's a few kinks to work out that way and then it'll be uh, okay. And then I've designed it as well that uh like I have it hooked up to GitHub right now, but the entire intention was that it's not reliant on GitHub. It's not reliant on any actual third party system. So you can use it entirely self-contained.

Um you could use it with Trello technically. You could use it with Azure DevOps or whatever. That's just like an integration piece that you slot in. So again, this would be something I'm just playing with it. It's for me. I've said before other people have way more better stuff like this. Um, but this is mine and it's kind of fun. So maybe this will go somewhere, maybe it won't. That's another thing I'm working on. So thanks for watching. I'll see you in the next one. Take care.