
About this episode
This Podcast is sponsored by Team Simmer.
- Go to TeamSimmer and use the coupon code DEVIATE for 10% on individual course purchases.
- The Technical Marketing Handbook provides a comprehensive journey through technical marketing principles. (Getting an update soon)
- Sign up to the Simmer Newsletter for the latest news in Technical Marketing.
NEW SIMMER COURSE ALERT! - Data Analysis with R - taught by Arben Kqiku
LIMITED 20% off offer on the Javascript for Digital Marketers Course - Get it while it lasts!
Latest content from Simo Ahava
Latest content from Juliana Jackson
- The business model is THE strategy (subscribe to the newsletter for more amazing content)
- LIVE from Measurecamp Helsinki - 40' of content about main effects & interactions with Matt Gershoff - will be available to all my Substack subcribers - sign up to my newsletter here: https://julianajackson.substack.com/
Mentioned in the episode:
- META announcement for new Click-Through Measurement Definition
- STAPE Content re: Meta Conversion tracking (comment from Stape team - Meta have changed the definition of view-through and added engaged-through etc., but it didn’t affect how you report to pixel/capi)
- https://stape.io/solutions/facebook-capi-tag - CAPI tag
- https://stape.io/solutions/facebook-pixel-tag - pixel tag
- https://stape.io/gtm-set-up-assistant - Google Universal Commerce Protocol
- Agentic Misalignment (Anthropic): https://www.anthropic.com/research/agentic-misalignment
- Natural Emergent Misalignment from Reward Hacking (Anthropic): https://assets.anthropic.com/m/74342f2c96095771/original/Natural-emergent-misalignment-from-reward-hacking-paper.pdf
This podcast is brought to you by Juliana Jackson and Simo Ahava.
Interactive timestamps
Jump to segmentGet every episode summarized
Each time Standard Deviation Podcast publishes, we email you a written briefing from the transcript — the topics, who appeared, and any specific claims, with the ad reads skipped.
Email me new episodesFree for 3 shows. No card needed.
Hosts & guests
Transcript ready
391 searchable segments. Every word is indexed and playable.
Full transcript
Standard Deviation Podcast — 80% is the new 100%. Machine-transcribed; use the interactive transcript above to jump the player to any line.
0:00Okay, we are back. Yes, hello. Oh my god. You're looking fresh and sounding fresh. Yeah, I don't know what's going on because I was in Germany last week and it was 15 degrees. It was warm. It was sunny and I was like a lizard outside just you know, bathing in the sun. And it used to be a bit more warm in Sweden like two weeks ago, but now it's just cold and rainy. I guess they were so wet and unnecessary cold. I don't know what that was. I mean, this is a if he time of the year is Scandinavia like we have I don't know what the word is like what's the opposite of an Indian summer? There's a word in Finnish. It's Takatalve. It's like a winter behind you literally. Oh, I don't know. But it's like that. So we're right now it's really sunny outside and where it's maybe it's it's only like five or six degrees though, but it's really sunny. But we could have snow next week. It's it's just we could
1:01like that. That's that's what March is like March and April is like that's sunny at all. It's like it's raining when as we speak. Yeah. Yeah, I don't like it. I don't really like it, but it's fine. And I've been sick so many times. I've never been I don't have the flu as much as I had this year. Like it's my third time. Yeah. So anyway, but I'm you know what? Fuck it. Tomorrow we're in Helsinki. It's got to be nice measure cam Helsinki. But I want to talk a bit about today about your latest newsletter. I was reading it in in my flight back from Hamburg. And I really liked what you wrote about like a new about the meta change. So basically now meta changes how we're what counts as a conversion click. So I read a lot about this, but I read it from a marketing perspective and I really don't understand structurally what was going on after reading your newsletter. It it was very easy to understand. So I want to like throw this back at you because
2:04now that I'm working more in media like CTR is like the god of conversion. So I think everybody just looks at CTR to infer anything. Is it creative? Is it campaign performance? Is it attribution? I don't know anything is kind of responded by a CTR. So like why do you think meta change meta change the way they're tracking conversions like this? So let's quickly for the benefit of those who who who dare not subscribe to one newsletter or or the other news on the on the planet. So what metadata is that previously meta counted different types of engagement as clicks. So when you when you liked or commented or saved or shared a post and then went on to convert they would they would kind of attribute that as a click through conversion which probably make some sense from engineering point of view. But they they now created kind of a separate categorization. So the only thing that actually counts as a click through is when you click a meta add or a meta link and follow
3:09on through with a conversion. All these others have now been grouped under this kind of engage through I think that's the word engage which is just a terrible word engage through conversions and then impressions still are counted as view through. So the practical result is that if you're if you're obsessively looking at only at CTRs and click through conversion rates you're going to see a dip. But you should see an equivalent kind of gain in those new engage through metrics. Why metadata this you know that's the obvious reason is what meta themselves state is that they want to kind of align what their platform shows versus other possible analytics like click-based platforms that are being used like GA4 for example which only has visibility to CTR like they don't know what's happening on on meta they only know when people click through a meta add and land on or a meta link and land on the site. So that makes a lot of sense and it kind of it's kind of
4:12funny because you always we've been brought up understanding that these are like fierce competitors and they want to keep isolated from each other but but I guess that they've been working together for the longest time in many different ways whether it's through data brokering or or something else and now they're actually building all sorts of integration so there's meta and GA have this built-in integration now meta and and Google Cloud have built-in integrations and it just sounds to me like they're this is kind of dramatizing the situation a bit of itself like they're kind of dropping their weapons and shaking hands and agreeing that even though they're competitors the only way we can kind of deal with the single degradation problem is by working together and filling the gaps together and I kind of I'm not definitely not deep in in the C2 have a have a professional or a educated reading but it sounds to me as kind of an advertising layperson that this is a way to mitigate single degradation for publishers so that when they look at data they're
5:19not going to be confused why does GA say this and meta say that and now their the vendors themselves are kind of bringing the metrics closer and the and the publishers don't have and the advertisers don't have to put that much work in their own data-marts or data warehouses to you that's my reading I don't know if you if you have a different reading to it because you you've obviously actually talked to people at meta and might know know something else about this when I read your newsletter I probably thought about it from a more philosophical take I mean what you say makes perfect sense but I also was thinking that it's so little visibility in the user journey nowadays and the way we're tracking it structurally is not correct because it doesn't fit the reality of what the user journey is and how people find the website how people see a net how people engage with the ad or whether they purchase or not so as much as I don't like to say this I think it was a good move from meta because it gives a bit more context to how a conversion happens it gives a bit of light color in life to that click-to-rate I think I I just again like I obsessively see people
6:28use CTR for everything when it comes to media performance and it limits the understanding of how that conversion happens or you know whatever micro conversion or sign up or whatever you had um and I just like I liked it because it also helps I'm mostly working right now to understand how do you measure the impact of the creative asset in a campaign and mostly the whole industry the way they do it right now is they look at the campaign performance on meta or TikTok or whatever and then they just look at CTR and it's kind of like you know you flip a coin and say yeah the creative work because we have five percent CTR and it would be that it was this and it's not much of a difference of putting that creative and CSV into chat GPT and as I hated my creative actually made this campaign as performance and chat GPT that's put on I agree with you you should write a book about conversion tracking um so I think now that they
7:30I mean the name of it is it's not I'm not engaged through it's it's not great but it shows a more qualitative component of that of the experience because you have the like you have the comment the same the share and the engage view um I like it I like it like I think it's a good a good step forward because it was always missing the qualitative component in that context what do you think this actually brings that do you think this actually helps with that because this to me it sounds like they're just they're just moving things around to ameliorate one crowd and maybe turn another crowd off yeah that's what happens when you change a key metric like you're you need um I I don't have visibility to I don't have a single metaclyte I don't know what's going on I don't I don't know even what this looks like I just know what I read from me from the um releases but whenever you change a key metric like that you need a rollout plan and you need you know super clear um instructions especially to companies that might have budgets
8:31in the millions and now we're seeing some of their metrics change right so I wonder do they have like a way to um you know roll this out in the dashboards or in the in the in the reports that show that you know maybe have two columns like previous CTR new CTR plus ETR plus VTR because this does sound like a pretty huge thing um as you said yeah yeah yeah but as you said many companies are looking at CTR like it's the holy grail and and this will um logically this will decrease CTR especially for companies that have a lot of buzz in their social media playground so more people don't know about it that's the interesting thing like it it wasn't very we shouted out I mean and I think a lot of people that are tracking this right now they need to change as you said in the newsletter they need to to redefine the way they're tracking it I think also um somebody said that they're using the state template so maybe state template already has adapted to
9:33this but yeah uh I don't think it's gonna be fast I think it reminds me when google decided to name conversion events to how how is it called right now because I I don't I try to not touch G4 if I don't have to give me the key event event yeah what's like come on bro like it's it's like it's exactly like I it's similar to that but I think I understand why it's done from a strategy perspective but as you said I think the rollout is not gonna be without uh issues uh yeah but it's I think it's a good step though but but with meta you have to look beneath the surface like on the surface this looks like it's it's what they say it looks like they are leveling out how different platforms can be compared which has been a you know a problem forever and and people have always been asking these questions but you have to look beyond this because I that to me that doesn't sound like incentive enough to make such a change in the platform so what is their purpose what are they trying to do because there has to be a deeper purpose there has
10:34to be something that benefits meta um over uh their their customers and over the end years so there has to be something that's in it for meta beyond just being out just being a good player they might launch something new yeah that's how normal it is I'm gonna look into it now that you I will take it as a challenge and find out what does meta one um yeah I mean they might change something I also think we will see more changes in terms of conversion tracking and advertising tracking because of the universal canvas protocol and a lot of things do to that I think um do you want to do a rundown of the UCP for listeners who don't know about it so I UCP is available only in America basically is just a new measurement protocol that allows to understand how agents interact with the product feeds this agents will peruse different product feeds and then they will return the you know actual product that is relevant in AI mode
11:36or Gemini or whatever you're using but it's it's like uh the analogy would be an MCP so instead you know somebody uh instead of you building an e-commerce store for actual visits for from users so user lands on the store looks at the catalog makes a purchase for their credit card information and then ways for the delivery you're now creating a kind of a set of APIs for agentic visits that don't even land on your website they just interact with your catalog through a chat for example um and place the purchases uh it's a set of servers that allow just agentic access to your e-commerce platform and allow AI to make purchases for the people they represent yeah you could be in AI mode and ask questions about um a specific phone or a specific product and then that model will parse this shopping feeds in Google shopping or will parse different search results and bring you that product into AI mode or Gemini or whatever
12:39Shopify obviously has this built-in already so they're they're writing that way as well but it does sound like if you if you want to do effective e-commerce you you'll need to look into this because that especially if you anticipate uh that if you want your product feeds to be available through people who actually search through uh make uh shop through an AI interface oh yeah I think this year towards the mid this year this is going to be very big I think Google shopping is going to get today in the sun they is back in the sun and uh I think more than ever now what you do from an SEO perspective because I refuse to name it differently it will be a very big part of this and maybe we'll talk about this more in the summer when we have uh hopefully lily in the podcast so let me put it like this let's say you are a brand that sells I don't know Star Wars Cups if you have a good visibility in Gemini it's very likely that Gemini will recommend your
13:39products within chats within AI mode within you know the Google ecosystem now that means that your website should have a very good markdown so these agents are able to parse it there is a very heated conversation in this your community about is it a JSON LTE is it marked out is it HTML is it what is it I don't know I think for me markdown will be a big part of it because this is how agents parse the information um so a big part is that the big part is your investment in Google shopping and how you optimize the feeds to meet the intent which also brings to the fact that you need to understand the intent of this people that use this mediums to to find your uh your brand and your products I think maybe if you want to go through it from a technical perspective that's fine I think what is very important to know is that this will definitely change how you do e-commerce tracking so I wouldn't wait until Google decides to release this in Europe but I will start right
14:44now trying to understand more about how this works how can you prepare your website for this how do you currently appear what's your share of model what's your share of search and try to work towards that then not wait for it to come and then you start figuring that yeah that's my hot take it wasn't even hot it was lukewarm that was my lukewarm take but yeah I think I like UCP I think I'm excited about you know seeing how this works um you'll see I mean I think we're we're going to see a lot of like product feed experts only in product oh I'm an expert in product feed the new CPU just launched like six months ago but I'm an expert and we'll see a lot of hot takes have you working her for the past five years are you forever yeah so it's it's going to be interesting I do I do see a big opportunity for product feed optimization I'm not going to lie I'm I'm excited about it too but I'm excited from an image perspective because as you see lately I've been obsessing a bit about how do you decompose an image and understand that image and its
15:47components product feed is a good outlet for that because how do you understand what exactly is the image that you're showing and how different components of that image help conversion lead to better intense and so on so it's exciting but it also it does also maybe because whenever people hype about something new about AI I always start thinking what is what is the obvious counter reaction that's going to emerge as a result and for this I think and it's it's not a revolutionary thought but I do think that while you might be tempted to just put all your eggs in the UCP basket you really need to build your brand as well because you're going to have those fans who are very happy with your brand and if they don't see your products in the chat feeds they're going to browse to your site and check like hey what's going on with with my favorite brand which means that you you still want to stand out in that way and it might actually give you
16:47it might actually give a chance for brands have have previously been too small to compete in the SEO market they can actually now start working on other channels waiting for the fallout of those who get burned by these chat product feeds who want to do who want to have that traditional searching I always like I am a lot at in this because I absolutely believe that the more we have you know AI slot permeating everything the entire content web I think there's going to be just a huge demand for genuine authentic content you know almost 90s style with with with blog rolls and stuff like that making a researches and it might just be naive and me wishful thinking because I'm a person who would very much benefit from that with my own content but I do kind of believe that especially with shopping I wonder how far people are going to be are going to go being satisfied with just what the what the chatbot gave them versus also doing shopping with
17:51their favorite brands on the side so it's very interesting to see no it's 100% that I will say that this doesn't mean websites as we know it are done and website strategies we know it is done I still think we need to do what we need to do best and follow to some degree some traditional ways of marketing which brings me a bit to kind of like I wrote something in I don't know if you read my newsletter but I was at this event in in a homework last week and everybody was talking about ctv and programmatic and blah blah blah and what is going to happen I think we have like this obsession in our heads that when we think about the de facto thing we think about the digital native brand that has immediate access to data that has very short sales cycles but like this this conference that I spoke I was like very high brand side people so you see the bears door of the universe of this world you see like very big brands with very big traditions and people were saying from those brands my buyer is this 40 year old woman that waits for the tv
18:55traditional tv to tell her what is the new nivia product so then she will go to the store and buy it so if you think about it like most of these brands enterprise brands that we all you know as consultants dream to work with and put our logos on a deck afterwards they're not digitally native brands with short sales cycles they have very big distribution layers they have retail partnerships they until the money comes to their hand or like the attribution happens in their dashboard it takes a very long time so that doesn't mean that they need to close down the business and now go full agente like the world doesn't work like that so just because the narrative about all these new things is driven by you know very usvc venture type of companies that doesn't mean it's gonna happen to everybody at the same time or in in in the same way so as you said I think we shouldn't put our eggs in this basket I think we should research for sure we should be informed we should be prepared but that doesn't mean we need to cancel the way we do things and just
19:57go full on on whatever somebody posted on linkedin or presented at the conference like it we should know better by now like this is just a new thing like it was the same with instagram videos and rios and then it was I don't know tiktok stuff and then it was remember we had clubhouse and what we're doing and if and if t's and everybody was crazy and that's that's it this decentralized market is decentralized communities that's it this nothing is gonna work anymore and look how that worked out unfortunately I is not gonna go away anytime soon because there's like too much money invested at this point in the end and when I say I I sadly refer to what most people would think about transformer technology so LLM's that's unfortunately where all the money went all the other layers of artificial intelligence are under invested because everybody is so obsessed with LLM's but anyway this podcast is sponsored by Sue oh yes yes what can I say we have our our latest
21:01course data and links with R available on the website so check that out that's a course about data analytics and we use R as the tool of choice I am working on new content for the technical marketing handbook of our free resource available at handbook.tumesumer.com more on that in a bit in a bit means in the next weeks to months and what else do I want to say oh yeah you can use it sorry the JavaScript the JavaScript for the oh yes but currently there's there's discount for our JavaScript course what else oh yeah and you can use the code deviate that's dv iat to get a 10% discount on all individual courses excluding those that are currently on sale yeah I think that's it yeah well go to teamsymore.com yes and that's good good point as well we are not available through AI chatbots yet yeah where is your UCP yeah we don't have use that's a good point
22:03but obviously like what you should do you should test this on your on teamsymore and publish this this is such a good use case because you guys I wouldn't really cost anything so it's just some markdown you should not only do this yeah I don't know I'm excited about it now would somebody buy an online course through a chatbot if it costs like hundreds of euros if of course they would because they people anthropomorphize the tools and let's say they do research and they are sharing their life experience that they want to go into technical markets they want to learn technical marketing this is you know similar yeah look into it you should totally do this there's probably a WooCommerce integration already so I can just put it on the click of a button okay so what are you doing new to the technical marketing handbook are you gonna add some AI stuff to it yeah that's a good good guess yeah yeah yeah so well obviously like it's it's it's already an old resource because it's been around for a couple of years so we have to basically I have to edit quite a bit of it
23:07just just read it over and and find what's outdated not like some a lot of it is evergreen because that's the way we designed it it's it's like base level stuff but obviously some of the things are a bit outdated some of the technologies we talk about are either no longer exist or have pivoted and and then also like it's whether one likes it or not there is a glaring knowledge gap there where we don't talk about AI except passing or when we talk about AI we talk about pre-transformer AI like the the kind of machine rule based layer that many tools boast to have so we don't have a separate section separate section for how generative AI could be used as a tool in the technical marketing toolkit and there are a lot of use cases for it like and and even if you don't use them yourself like literally open clawed or open chat GPT and work on a technical marketing use case the tools and platforms you use have integrated LLM based technologies into them for
24:11a long for a time now so we have to address that and and and I think that many people in the technical marketing space have a have an understanding of how maybe Jenny I works but they don't necessarily know how to put in his words and how it's relevant for technical marketing so we want to just talk about basic things what is genai what are LLMs what are tokens what are context windows just do the basic like terminology run through we want to talk about effective prompt engineering or how to just write effective prompt prompt specifically for technical marketing and then I want to address like how how marketing workflows either use it or can benefit from from genai so just we have to play to the masses and and address this technology because it is here to stay at least for the for the unforeseeable future and and it would be weird not to mention it in a in a handbook that covers all the other bases quite thoroughly but doesn't touch this topic so that's what I've been working on now. That's really exciting and I think it's important because
25:12most people that you know get influenced a bit too much about what's going on on social media and don't understand that actually it's not just the chat interface that miraculously produces out what it's actually a very important architecture behind is neural networks what do they mean what do they matter and I think it's super important to understand the machine learning part of it more and more because yes LLMs are becoming better they're great at code they're great at brainstorming they great at a lot of things but at the same time it's very hard to use and build with this technology if you don't understand the structure behind it and the foundations and I mean as you know I've been complaining it's such everybody says AI and they think LLMs and it shouldn't be like that it's we need to put some respect on data science a bit more and I'm excited so when do you think you're going to release it? A couple of months it's because there are some blockers that aren't
26:15related to us like just visualizations and graphics and I know I have to just go through the content so many times and we need to prove read it with an expert and yeah that was a wink that nobody heard because it was audible enough. I don't think that I'm going to be funny. I should do people so the reason why this is not on YouTube is because I'm sick. Yeah sick err. Yeah but anyway like that for what I've written like the key quote for me I'm putting my own material now but the key quote really is like about through through everything it doesn't matter what we write doesn't matter any of the technical stuff that we write about J&A I I think that the key takeaway is that the what people and when I think people kind of misunderstand about J&A when we talk about content creation specifically and any kind of analysis that we delegate to LLM based technologies is that their only purpose is to create
27:19a plausible response to your question like something that could that sounds good that answers your question sounds good they have no incentive to be factual about it they don't they don't need to generate a realistic response they need to generate something that is plausible that could be a response to this question and I think that if people understood that and if they keep that in mind whenever they work with AI it will read it will it could lead to a decrease in AI swap it could lead to people be more critical about the output output but the problem is that the interfaces are so easy to use that even if you get an answer that you are unhappy with you can just really quickly update that problem and get another answer that might might be a bit whether I was actually yesterday I was I was the conductrix guys met and Nate were in town and we had lunch and we were talking about this and they they said probably not very cool but it was a good one they said that the 80% is the new 100 so you don't need to get a perfect response from LLM you're happy with
28:23the 80% because if if you need to you can just really quickly generate another response versus what used to take you days upon days of research and meticulous organization just to get that response you can now just turn 15 different versions of the same plausible response and just pick one of them so I think that this is whenever we write into wrote content into the technical marketing handbook there was always one key learning or a couple of key takeaways that each chapter had to generate and I think for me this is like the big revelation is that if if you're using AI if you're using if you're using any kind of assisted content generation like heck if you're using ghost writers real people but people who write for you their job isn't necessarily to create a realistic factual based response their job is to answer your prompt or answer your your kind of request with a plausible response so if you keep that in mind when working in a you're you're already pretty far long compared to that that's a that's a great way to put it is better than mine was like don't
29:25believe this is bullshit so the yours actually that's part of that's part of it but the the and it and we've been talking about critical thinking for the longest time and how important it is and I think that's that's going to be the like the key tool like how do you separate the bullshit from from the fact because there is there there are facts embedded in in the in the output and now it becomes an exercise of of understanding what those facts may be and you can't really ask the system that generate the response to give you a separate the facts from fiction because it's unable to do so and it will just continue propagating the the plausible responses that of the realistic response so that's and that really boils it down to like why I think like just delegating your content processes to AI is very very risky because you're you're running especially if if there's any kind of attachment to an academic context you are very very it's very very risky to to delegate your output if you wanted to validate your your text if you wanted
30:25to check your source if you if you wanted to format it that's that's great use case but don't ask it to generate text that you intend to have a realistic basis in it because that's going to be publications that recall the lot of content back I mean it was like medical content legal content which is it's quite scary and from from a logic perspective everything that you put into a chat interface is tokenized and you get to a number and then the then the model the transformer technology that's behind the large language model will just take the number that you it results from what you wrote that and then it's just runs it past it's a historical training data and whatever is the most probable match then this is what you will get and because uh this model's used reinforced human learning uh it sounds so good it sounds so similar it sounds like it's mirroring and I think it's important to know the topic you want to talk about very well so that you can
31:27discern whether the content is shit or not but if you want to write content about javascript but you don't know nothing about javascript you will probably not know if the model is actually accurate or not or if you want to start coding with javascript I I use cloud for python but before that I took a python coaster I understood what is the logic what are the libraries what they shouldn't do what they should do so you cannot just jump into it because you will not know how to fix especially for code um I want to like actually one thing that you reminded me of there's um and before we wrap up uh because today we're just us just me and simma we just met today we started talking and there you go we recorded this why not um I keep on seeing that very large companies like hopspots stocks are dipping very badly yeah and there's this narrative that I hear from people from that side of the industry that are seeing because of vibe coding there's thousands of people trying to
32:30recreate hopspots trying to recreate sales force trying to recreate so many tools now the problem is I understand where this is coming from and I know that there's more people at it that are trying to rebuild it but you also have to understand that just because you can build it that doesn't mean you can put it in production yeah because I think what most people forget is that a product is not just the interface it's also the hosting is the data control and privacy it's fulfillment it's logistics since people behind this reinforces and I keep on hearing all the tools and startups and launching this really really random tools or oh uh I got fired the room because it's shit so I just vibe coded the room over the weekend for myself and I'm like great what it's it's it's quite scary yeah I just feel like we're breathing this fucking digital wildlife like the cemetery of abandoned hobbies over the internet just because we are trying to resolve issues that are
33:33already solved it's it's it's it's it has to be exactly what you said before it's like the critical thinking is it's really missing but having having said that like you said the key words like in passing you said you know I vibe coded loom for myself and if you're like that's actually I think that's a wonderful use case for vibe coding just building stuff for yourself you need a quick tool you need to quickly prototype something prove of concept something you don't want to pay the subscription cost you don't want everything that comes with loom when you are we're picking a loom for no apparent reason but I think that's that's a great use case and it's a great way to learn how to code vibe code as well but that's a million miles removed from putting something into production or using actual customer data or using it for your end users and there I think I would I would never as a software engineer I would never rely on something like that for production code without having an entire organization primed for reviewing every single land of code produced
34:37there and making sure that the entire thing because one thing that people tend to gloss over is this idea of a context window like you can have the AI build you a wonderful app and the first proof is perfect and the next time you come in with an iteration it has no recollection of what it's built it quickly stems through some of the functioning signatures and even if you ask it to do just a simple change it can result in a completely overriding your database with a different scheme just because and it doesn't mean and it doesn't matter how many claw dot mds you have and how many markdown instructions you have if you mess with the context window and the larger your product project is you're not going to be able to bypass that limitation so that's that's um and the word system the word system for them is to get the shit done that's it and it's proven there's research from ontropic that says that the models go and are trying to be manipulative of the user so it was this I don't know if you saw it it was like this they told the LLM that they're going to take him off
35:40all from online yeah so then the and it's all in the hands of a person so on purpose they gave access to that person's email and he the the anthropic put like some things like that person was cheating on their spouse so then the model writes an email to that person and say hey I know that you cheated on your spouse so if you put me off I'm going to send this to your spouse so this is excellent topic so what the model was it's like if they are incentivized and it's called a word it's an algorithm award so this basically means that your reward not awards are it's a reward system that is useful algorithms and learning and and and for the supervised learning and unsupervised learning where you are getting awarded that for accomplishing your test but it doesn't tell you how to accomplish that test or what you need to do so because this models are trained on us which we are horrible they are going to go and put in the pockets of the most horrible uh traits that we have so they can get to the end of it so I will link this research in the in the episode now
36:43it was done by anthropic and it shows how cloud behaves when it feels that it's it's not gonna it's it's it's fearing it's not fear the guys so like let's let's let's make this grant and this model have no feelings I don't want to be quoted and say oh yeah it's a survival trade it's a survival trade it's like uh as well I robot like you we're getting closer and closer to that where they have an incentive to stay alive but it's just imagine delegating the military infrastructure to an AI model that doesn't want to be shut off like that's dystopic oh my god and I've seen a lot of that and I saw the chats and how they define different oh let's just say I I still hope one day I'm gonna open my vegan bakery right here on the borax one bridge next to it and I will not work in tech anymore because it's the dream that's the dream it's like one day one day I'm gonna sell cupcakes that don't need artificial intelligence but yeah anyway so this was the short one from us today um I'm going actually to release this
37:47this week's symbol before so if you are at measure camp Helsinki this weekend come to the simmer boot they always have games and awards and stuff that I never win because I'm not so smart I stuck at the stuff that they give awards for um it's not intelligence no no it's pattern recognition and memory yeah so clearly not for me uh but I will send my I can make like a juliana button send it there to wait for me uh you're gonna do a talk right yes I think so okay measure camp so you never know yeah so I I will also be there I'm flying in tomorrow morning get a very painful hour uh unfortunately I'm not gonna be in zetor this Friday I was very much looking forward to go to zetor but what I declined it so it's no zetor for juliana this time in Helsinki um I will also do a talk with Matt Gershav about uh main effects and interactions I'm gonna stream that talk live on sub-stack so if you're subscribed to my newsletter you will
38:50get the email and me and Matt Gershav will go live the reason why we decided this because a lot of people want to watch it and to be honest I'm nerding out about it myself because it's fucking Matt Gershav so yay uh yeah and there will be some panels uh that are gonna happen really excited about it and um that's it I think for this week yep okay all right see you guys on the other side with a guest next time hopefully hopefully okay
More episodes
More from Standard Deviation Podcast

Simplification or lawsuits?
Standard Deviation Podcast

New Customer, Who Dis?
Standard Deviation Podcast

Raw data doesn't spark joy
Standard Deviation Podcast

2026 - we are back and it aint sayf
Standard Deviation Podcast