#642 Neil: GPT-6.1 Sol In Codex Surprised Me Across Three Real Tests

0:00Capital One's tech team isn't just talking about multi-agentic AI. They already deployed one. It's called chat-concierge and it's simplifying car shopping using self-reflection and layered reasoning with live API checks. It doesn't just help buyers find a car they love. It helps schedule a test drive, get pre-approved for financing, and estimate trading value. Advanced, intuitive, and deployed. That's how they stack. That's technology at Capital One. It's not just electric. It's Toyota Electric and during Toyota's easy-to-toys sales event, going electric is easier than ever. Right now, save instantly on a new Toyota EV with special incentive programs for first-time EV buyers and prior EV owners. Choose the sleek busy, the adventurous, busy, windland, or the sporty CHR. But don't wait, these special savings won't last. We make it easy. Toyota, let's go places. We hear big promises and technology constantly.

1:02Oh, absolutely. Every single week. Right. But GPT 6.1 sole just launched with a massive claim. It promises near GPT 6 astra performance overall, but it costs drastically less to run daily. Words are incredibly easy until you test them. Welcome to the deep dive. We are glad you are here. Exactly. I mean, new models always sound amazing on a spec sheet. Always. But then you try to make them do actual work. That is when the wheels usually fall off. Yeah. So that is exactly what we are testing today. We pushed GPT 6.1 sole to the absolute technical limit. We ran it inside Codex for this experiment. Codex is essentially our secure internal testing sandbox environment. We are going to look at the raw numbers first. Then we will analyze three intense stress tests. Okay, let's unpack this. We will cover a full browser operating system build first. Then we will recreate Jerry Seinfeld's apartment in 3D. Right. And finally, we will run a

2:02dynamic robot arm simulation. And to wrap it up, we compare it directly. We stacked it up against Claude Saunet 5.5. We need to understand the baseline numbers first. This release is incredibly interesting on paper today. It really is. Open AI is essentially commoditizing raw processing power now. We need to look at the standard pricing structure. Yeah. So the pricing structure is the biggest selling point here. You pay two dollars per million input tokens, output costs 10 dollars per million generated tokens. Wow. I know. That is already roughly a fifth of the cost of its predecessor. That aggressive baseline price completely changes the math. But the caching features make it significantly better. Cash to input drops down to just 10 cents. Yeah, that is a massive operational cost reduction. Cash writes cost two dollars and 50 cents. Right. This makes repetitive daily tasks vastly cheaper over time. You are basically saving compute on things it already processed. We also have long context pricing to consider now. It jumps slightly for those massive prompt

3:02payloads. It costs four dollars per million for standard long context. Right. But cash to input sits at exactly 20 cents. Cash writes go up to a five dollars total. Output costs 15 dollars per million tokens for heavy context. This pricing structure feels very intentional to me. It is like stacking Lego blocks of data. Yeah, exactly. You only pay a premium for complex builds. The long term memory builds costs slightly more upfront. It's still easily beat standard GPT-6 astro pricing. Oh, without a doubt. Plus, you get a massive overall context window. We get 1.05 million tokens. That is a completely staggering amount of data. I mean, you can feed it entire code basis all simultaneously. Yeah. You also get up to 128K max output tokens. And the knowledge cut-off date is April 30, 2026. Which is an incredibly recent date for these models. Let us define the context window very clearly. It is how much text the AI can remember at one time that perfectly summarizes the technical concept for everyone. So also introduces several

4:08different internal reasoning levels. You could choose settings ranging from low to max. It offers low, medium, high, X, high, and max. That gives you way more granular operational control. Right. You can perfectly balance speed, cost, and task difficulty. You do not always need maximum reasoning power. Exactly. Simple data extraction runs incredibly fast and cheap. Complex architectural coding gets the heavy processing it requires. Like why burn expensive compute if you are just summarizing emails? So why do adjustable reasoning levels actually matter here? Well, they let developers surgically manage their API budgets. You prevent wasting expensive compute on completely trivial tasks. Matching computational effort to the problem saves real money. We know the context window is absolutely massive now. So we wanted to max out that output capacity. We tested a continuous massive coding task next. Yep, we gave it a really heavy project prompt. We asked sold to build a full browser OS. We forced it to run locally inside Chrome.

5:08And the entire project required one single HTML file. No external frameworks or additional build tools were allowed. No react, no node, no helpful libraries. Just vanilla web code packed into one document. That forces the model to be incredibly efficient internally. It had to build a fully functional desktop environment. We required a polished UI with a taskbar. It also needed dragable and resizable app windows. We demanded an app launcher and full screen support. Right. We also asked for procedurally generated desktop wallpapers. Those wallpapers needed subtle animated movement built in. They also had to use user-adjustable numerical seeds. Yeah. Generating moving visual patterns through peer math is tough. The test required five working applications to run smoothly. We demanded an email client and a text app. It also needed a sound or music making app. We required a dedicated system settings application to. Plus one additional useful app of its own choice. It actually chose to build a

6:10working journal application. We also demanded two fully playable 3D games. The first was a GTA style city sandbox game. It needed vehicles, police chases, and open environments. The second game was a 3D flight game. That one was called SkyRunner in our prompt. It needed responsive controls, level progression, and sound effects. Both games had to play directly inside the OS. They could not just open as separate browser pages. Here is where we really pushed the cognitive load. We added one special requirement to the prompt. The system needed a connected feature across different apps. We wanted it to feel like one unified system. So activity from SkyRunner had to show up in the journal. That kind of cross-app communication is incredibly hard. It requires managing global state within a single file. Sol finished everything in exactly 33 minutes and 42 seconds. Whoa. Imagine generating an entire operating system in just half an hour. That is staggering.

7:11Yeah. Human developers take that long just setting up their environments. Running NPM install alone takes half that time. Right. It is honestly mind blowing to watch happen live. Everything actually worked perfectly right out of the gate. It successfully avoided fake interactions and placeholder UI buttons. It kept performance smooth enough for standard Chrome playback. It even tested major interactions before finalizing the code. Though it was slightly less visually polished overall. Yes, on at 5.5 definitely looks a bit cleaner visually. But Saul reached the second playable game way faster. It's core strength is clearly raw speed and task completion. What does the SkyRunner to journal connection actually prove here? It proves the model understands complex global state management. It comprehends how independent programs share underlying data architecture. The AI truly grasps how separate programs share data. Yeah. Saul handled massive logic code incredibly fast today. But raw code is just pure logical

8:13instructions. We wanted to test its spatial awareness next. Exact visual replication is a totally different skill entirely. We gave it a very specific cultural touchstone. We asked it to recreate Jerry Seinfeld's apartment. It had to be a highly detailed 3D scene and had to match the real TV set perfectly. This forces the model to understand physical 3D space. It cannot just generate a random generic sitcom room. The layout and furniture placement had a maturity. We demanded specific details across the whole living space. We asked for the main living room area specifically. We needed the kitchen, front door and the bedroom. We specifically requested Jerry's iconic green client bike too. We wanted the TV and surrounding wall decorations included. The prompt included the desk and computer area too. It even needed the window view of adjacent buildings. We also asked for highly useful viewing controls. We wanted daylight and warm interior lighting toggle options. We required multiple camera presets for the entire scene. We even

9:17asked for a full overhead dollhouse view. The build process took almost an hour entirely. This was much more ambitious than the browser OS. The initial results were honestly highly impressive to see. It successfully created multiple angles and interactive lighting controls. I could spot the green client bike very easily. The kitchen setup and the front door looked great. The dollhouse view made it exploring the space really fun. The small props and recognizable cultural references were there. But the model did reveal a very clear flaw. The room positions did not match perfectly overall. The layout proportions felt a bit off geometrically. It looked completely convincing at a quick first glance. But checking the full apartment exposed spatial mapping errors. A true recreation requires exact geometric layout math. I still wrestle with getting AI to understand basic spatial layouts myself. What's fascinating here is the distinct difference in skills. Generating specific recognizable objects like a bike is easy. It knows the semantic concept of a bicycle. But understanding true

10:22spatial geometry and room flow is much harder. It built the pieces but struggled with physical placement. A language model maps token proximity, not actual mathematical space. It knows the word couch is near the word TV. Exactly. It does not know the exact physical coordinate distance required. So why does AI struggle with exact room proportions? It relies heavily on predicting sequential text patterns. It simply lacks a true internal physical physics engine. Text prediction simply fails at creating precise mathematical layouts. Mid-roll sponsor read. It struggled slightly with static 3D layout rules. We decided to push it even further next. We wanted to test a dynamic 3D environment though. We needed to test dynamic planning and failure recovery. We asked it to control a virtual robot arm. And the prompt required inspecting the codex environment first. It searched for access to real hardware components initially. It checked for a robot joint or standard Cartesian controls.

11:23You know Cartesian controls just mean moving in x, y and z space. It also looked for real camera feeds or poses. It checked for local control frameworks or setups. When it found none, it intelligently built a simulation. A local interactive browser simulation to be exact. The constraints were incredibly strict for this specific test. It needed a multi-joint articulated arm with limits. The gripper had to visibly show its reach capability. We also required a virtual toy car on the table. The car needed realistic collision physics to slide correctly. It also required visual feedback from a virtual camera. It needed a main view and a secondary angle. This created a very complex internal logic loop. The AI agent had to estimate the car position. Then it had to plan a safe physical approach. It moved the arm in small, highly controlled steps. You check the virtual camera after every important movement. If the car moved accidentally, it had to adjust. It had to align and close the gripper perfectly. It had to

12:26visually verify the grasp before lifting. We specifically forbade it from repeating movements blindly. It had to actively learn from its immediate mistakes. It had to detect exactly why grasping failed initially. Then it had to rewrite its code and change its approach. Like watching a toddler learn to pick up a block, but the brain is a massive neural net. Right, and it actually stopped when things got unsafe. It evaluated its own failures by reading the camera logs. It respected the mechanical joint limits perfectly during operation. It acted as a true agent in a continuous reasoning loop. It wrote code, observed the output, and adjusted dynamically. What is this complex failure recovery truly tell us? It proves the model can dynamically alter its own plans. It processes real-time feedback to overcome unexpected physical obstacles. The AI adepts instantly instead of blindly repeating failures. Yeah. We have seen souls, raw speed, and coding capability. We also saw its slight spatial geometry

13:27limitations. We need to compare it to the direct competition. We stacked it up against Claude Sonnet 5.5. Both of these models sit at the exact same price, two dollars for input, ten dollars for output. Prices completely removed from the decision process here. Which makes us a purely capability-driven comparison test. GPD 6.1 sole is definitely faster in most coding scenarios. It is highly efficient for long coding tasks. It remains incredibly stable, handling massive, complex, prompt instructions. You can use it reliably and longer developer workflows. It handles this massive context windows incredibly cleanly without forgetting. Sonnet 5.5 is slightly slower on those large builds, but it delivers far more polished visual results consistently. Yeah. Sonnet creates richer visual environments and better creative detail. Rendered characters and environments just look much more natural overall. Souls sometimes feels a bit rough around the visual edges. It acts more like a pragmatic engineer than an artist. But Saul is just so practically efficient for logic. Sonnet feels more visual and

14:32heavily creative in execution. So what does this all mean? We have a clear user choice matrix developing here. You choose Saul for a serious logical building and raw speed. It is perfect when you need complex systems working fast. You choose Sonnet for visual polish and artistic creativity. It wins when the task depends heavily on exact 3D detail. Saul handles massive prompt workflows without structurally falling apart. It rarely needs much manual cleanup after finishing a project. But Sonnet easily wins if you need exact visual replication. How should everyday users choose between these two powerful models? Evaluate whether your project requires logical speed or visual beauty. Then simply pick the tool that matches that requirement. Match the AI model strictly to your specific workflow needs. This brings us to our core takeaway today. GPT 6.1 Saul left us with a very clear impression. It is not just a cheap budget AI model. It is a true powerhouse for

15:34long complex workflows. It turns massive prompts into highly usable digital tools and it does it with minimal human intervention required. It rarely falls apart under heavy cognitive logic load. The reliability factor alone makes it incredibly valuable today. If we connect this to the bigger picture, wait, I mean if we connect this to the bigger picture, AI efficiency is taking a truly massively forward. We are getting top-tier capability for absolute pennies now. We want you to really think about this idea. GPT 6.1 Saul can reliably engineer a working operating system. It builds interconnected applications from scratch in exactly 33 minutes. What happens to the definition of a software developer when iteration speed is effectively zero? Thank you for taking this deep dive with us. Outro music. Capital One's tech team isn't just talking about multi-agentic AI. They already deployed one. It's called chat-concierge and it's simplifying car shopping using self-reflection

16:38and layered reasoning with live API checks. It doesn't just help buyers find a car they love. It helps schedule a test drive, get pre-approved for financing, and estimate trading value. Advanced, intuitive, and deployed. That's how they stack. That's technology at Capital One.