Skip to content
TrackPodcasts
businessOct 2, 20268:07

You�re Using Jev the Hard Way

Get every episode summarized

Each time Leveling Up with Eric Siu publishes, we email you a written briefing from the transcript — the topics, who appeared, and any specific claims, with the ad reads skipped.

Email me new episodes

Free for 3 shows. No card needed.

About this episode

“Jev is a big deal, but most people are using it the wrong way. I know for a fact, I used it the wrong way in the beginning, because my Cloud Code actually called me out on this.”From the transcript
👉 Single Brain - Revenue agents to scale faster using our unified intelligence layer. https://www.singlebrain.com I was using Jev after the expensive model had already done the work. In this video, I show the mistake, the filtering workflow I changed, and how I define criteria before trusting the output. We walk through real content examples, Grill Me versus Grill Me Overview, pre- and post-work checks, and why a human still needs to inspect the evidence. In one batch, Jev passed 189 of 517 chunks to the costly model. That is a filtering result, not a measured percentage of money saved. Clarifications: The 198/225 and 189/517 figures come from different datasets, not a controlled before-and-after test. Jev’s Noul returns the probability of yes; Choice selects an option; Score rates against ordered criteria. Score is not a confidence score. Chapters: 00:00 The biggest Jev mistake 00:14 Filter before the expensive model 01:07 Define the right criteria 02:23 Grill Me: questions one at a time 02:43 Work with Single Grain 03:05 Grill Me Overview: batch the questions 04:36 Add pre- and post-work gates 05:15 The real article and evaluation path 06:08 Keep a human in the loop 06:52 Teach your team the workflow Footage and sources: Eric Siu’s original camera and screen recording. Single Grain website footage is retained from the original edit. Grill Me is credited in the narration to Matt Pocock; the article and workflow excerpts appear within Eric’s recorded screen demonstration. How to Connect IG: https://www.instagram.com/ericosiu/ X: https://x.com/ericosiu 👉 Single Grain - AI marketing firm https://www.singlegrain.com 👉 Get 25k Words of AI-Powered SEO Content: https://clickflow.com/ 👉 Growth Newsletter for top marketers: https://levelingup.beehiiv.com/subscribe 👉 Connect your tools with Single Brain Gateway: https://www.singlebrain.com/gateway

Hosts & guests

Transcript ready

200 searchable segments. Every word is indexed and playable.

You�re Using Jev the Hard Way

Leveling Up with Eric Siu

0:00
8:07

Full transcript

Leveling Up with Eric Siu — You�re Using Jev the Hard Way. Machine-transcribed; use the interactive transcript above to jump the player to any line.

Jev is a big deal, but most people are using it the wrong way. I know for a fact, I used it the wrong way in the beginning, because my Cloud Code actually called me out on this. It was like, hey, you're using Jev after you use the expensive tokens, the expensive models. Instead of Jev coming in after the work was done by the model already, Jev would do the work first. Then I actually put in, let's say, a pre-publish gate and a post-publish gate for the content that we're releasing. In this case, after we added it, it said, hey, Jev tagged 517 chunks of calls and posts and passed only 1809 to the costly model. If you do the numbers on that, that's 36%, only 36% of it made it through to the costly model. It actually ends up saving you money at the end of the day. That's a very good thing. It shows the hit rate that you have. When I used it on, I think I used it to classify stuff from an AESU standpoint. Maybe only 7% of content was a fit to pass through.

It just shows you this classifier is extremely powerful, but you've got to use it in the right way. Here's where I'm going with this. If you think about using a classifier, you have to have the right classifications in place where you have to have the right scoring criteria. Remember, keep in mind that the way Jev works is that you need to make sure that you're defining whether something is something a true or false is in a confidence score thing or is a multiple choice thing. For example, if I'm publishing a piece of AESU content, I want it to be non-AI Slop rate. Maybe that's a gate that's in there. I can have it tied to a skill for that. Non-AI Slop could be one thing. Are we making sure that this is a piece of content that was never produced before? We don't want to produce more similar content. Duplicate content, we definitely don't want that either. Are we faking stats? If we want to have legit stats in there, or if it's something that I've published content around, can we actually cite that content?

Can we actually cite content from the last 30 days or so? And this is this meaningfully better. We can probably have gates around that. In order for you to think through all this, it takes time to do it. You definitely should spend a time on it. But let's say you're very busy as an executive or as a leader. You don't have all the time to do it. Because there is a skill out there. So Jev is very powerful on its own. But I was thinking about a skill called Gromy, which is from Matt Pochock. And Matt Pochock has a skill called Gromy, which will ask you a bunch of questions one by one, which is very powerful. Because it's asking, hey, do you mean that we should do this? Should we do this instead? And then based on that answer, should we do this? Which I think is great. One by one is ideal if you have the time for it. By the way, if you like what you're hearing right now, and if you want help from my AI marketing team, that helps create these managed marketing agents that live inside of Slack and Microsoft Teams, helps with AEO, SEO, helps with outbound infrastructure, helps with scaling ad creative. You can work with my AI native team. We'll build these things for you, and we'll manage them for you as well. Just go to single grain with the gsinglegrain.com.

And we'll see you on the other side. The way I look at this is if I'm just looking to just Jev max the maximum number of things that I can Jev, there's going to be trade-offs here, or it's not going to be as detailed. So for me, I actually created a skill called Gromy overview. What the Gromy overview just does is it just asks all the questions at once. The downside, the trade-off to that is again, you can't ask the one by one, which will form the next question, the next question after that, the next question. It's basically all at once, right? But the cool thing is, I'll just say this, Gromy is so good that sometimes it'll ask me 50 questions at once. It's just over and over and over, right? But what if you get distracted? What if you have other things to do? What do you have to go into a meeting? What if you have to go talk to an important customer? Life gets in the way, right? And so I would prefer all the questions that hit me at once, and I just hit dictate on my whisper flow, and I just boom, I just hit them all at once. And for the most part, if I see a list of like 20 questions or 50 questions usually from Gromy, I would say that maybe I might want to make like three changes. So the percentage is actually very small,

but these are high-level changes that I want to make, right? And so I found that to be useful for me, and it kind of gets just gets the job done. And then after that, you're plugging into your harness, right? So I literally just wrote, you know, how to build fast AIEVALs with Jeff, GROKBOTS, and Gromy. And so if you're using Jeff, okay, great, you got to make sure that you are setting in the right criteria, the right class fires, okay? So use the Gromy overview skill, and you could just modify what Matt Pokec has. I'll probably put this Gromy overview skill onto my AI marketing skills GitHub, and you could search that. And then you combine it with your harness, right? And so for me, I love using GROKBOTS right now as of this recording. So with GROKBOTS, I mean, I just say, hey, use it. I want to make sure that there's a Jeff gate pre and post, right? So pre, the work getting done, so I save money on the expense of tokens, right? And then post. So like if I'm publishing a business content, there's a publish gate, right? Does it fit all these things? Does it fit our guidelines ultimately? Or maybe there's a gate before you reach out to someone that you're trying to recruit. And then after they answer questions,

you're pre-qualifying questions, then you can gate them that way too, right? So I think having these really inexpensive gates changes the game completely, but again, you have to know how to classify these things, right? And so I'm going to show you a couple of things over here. I mean, on this X article, you can see, Matt Poecock, right? It's literally right here. So he created GROKBOTS, and you can fork it and make a version of yourself just using any bot that you have. Literally, that's just what I did. I just said, hey, I just said, hey, I want like a list of these questions instead. All at once. And so, as I go down here, it says, okay, let's just look at this piece over here. So if we see fast feedback, so this is the e-vow path now, right? So fast feedback validation still ahead. So source packets plus locks back. Okay, so Jeff, a three runs, probably on some medium plus rules and code, right? And then you have observed, which is shadow evidence, check the run stability, compare routes with production, and then pending owner-answer key, label-free items, people validation runs, then a human launch decision,

right? And the Evalid accuracy, and plus route decisions. I would just say, when I look at this now, I do feel it's important to be in there as a human in the loop. You can't just let this yellow and run on its own. But even if you just take this article on its own, and I just cut and paste it, you should be able to get what you need, right? And so that's what I would say. And I learned the hard way here. I did continue to burn on tokens until I ask Claud code, literally, hey, what are some outside the box ways we can work? How have we been working inefficiently? And oftentimes, your harness will know how you've been working inefficiently. So ultimately, that's what it is. Hope you enjoyed this video on how you can maximize the time you're working with Jeff. I mean, this is one of the biggest mistakes. And if you unlock this, I think you're going to be able to do a lot more. Because then you can truly just Jeff Max across all the different things that you're doing. And then you can teach your team how to do this as well. That's where I think that the leverage comes from, right? Because the team, your team sees a lot of things that you don't see. So this is one of the keys to Jeff Maxing. So hope you enjoyed this video, and we'll catch you in the next one.

More episodes

More from Leveling Up with Eric Siu

View all episodes →