Developer Chats - Oleksandr Piekhota

0:00Today's episode is brought to you by Dot Tech Domains, and this one hits close to home. Back in 2016 I was building my startup and went hunting for that perfect.com and found next to nothing. So I did what every founder does, settled. Here's what I wish someone had told me. You're building a tech startup, just get a Dot Tech Domain. It instantly tells investors and customers what you're about. Don't overthink it. Secure your Dot Tech Domain today from any registrar of your choice. This episode is sponsored by Unblocked. Unblocked is the context layer your agents are missing. It synthesizes your PRs, docs, slack, and tickets into organizational context that agents actually understand. So they make better plans. Write higher quality code, use fewer tokens, and require fewer correction loops. If you're running Clawed Code cursor or any agentic workflow, unblocked is worth a look. Learn more at getunblocked.com slash code story.

1:00This episode is sponsored by Mesmo. If your team is collecting large volumes of logs, metrics, and traces, but still struggling to get timely answers, Mesmo can help. Mesmo is an active telemetry platform that processes and enriches observability data in real time, before it's stored or analyzed. That means lower data volume, lower cost, and faster root cause analysis across your existing observability tools. To see how it works, get a demo at mesmo.com slash code story. This episode is sponsored by BrainGrid. If you are building with AI coding tools, but your features keep breaking, you need to check out BrainGrid. It is the product management agent for AI builders. BrainGrid turns messy ideas into clear specs, tasks, and prompts that coding agents like cursor and Clawed can actually build the right way. Real software, not fragile prototypes. Start free at braingrid.ai

2:01Hello listeners. Today we are dropping another episode in our series, Developer Chats, hearing from the large-scale system builders themselves. In this episode, we are talking with Alexander Begota, principal software engineer at Teaching Strategies LLC. Alexander helps to show us, at what point of scale, platform approaches are required, when to run experiments, when to stop, and perhaps more importantly, engineering ownership beyond the code. Alexander, thank you for being on the show today. Thank you for being on code story. Thanks for having me. Absolutely. Really excited to dive into all of your experience today as moving into more technical leadership as a principal engineer and platform versus feature thinking and all the things we're going to dive into. Before we do, tell me in my audience a little bit about you. So basically, I am currently working as a principal software engineer at Teaching Strategies.

3:03I can count like 14 years in production engineering, so I think I have at least of some understanding like what I'm talking about. My background is mostly focused about PHP and GoLanc. I do have some extra experience in different languages, but that's not that reach to say that I'm strong in it. Have two masters degrees, one computer science, and the second one in high performance computing. The last one I've got a few years ago in University of Edinburgh, really happy about it. Last year, I also found it a community called TechDocs Edinburgh, and we're just trying to make some meetups and events for people to share their technical experience and thoughts and give some richness to the audience. Very cool. Congrats on all your success in your recent degree acquisition and that's a lot of work put into to get those advanced degrees. I'd love to dive into my first question, which you've moved from hands-on engineering

4:06into principle and technical leadership roles. Kind of more working on architecture and platforms. At what point did you realize your work was no longer about individual features, but about the system as a whole? Oh, good start. I think it's responsibility shift. So basically, when you realize that your decisions influence performance, cost, reliability, maybe like team velocity at the same time, and when you understand that you're responsible for the system at the hall, like not individual features, then when you understand that like your manager can call you during the night because of the client issue or something happened to the production instance, and they need your help, no matter how long it takes and like how late it is, then this shift starts. I remember like a case from like my producer's public API experience when a manager called me about client's issue and I tried to help him and debug it even though it was late,

5:08but the problem was that we haven't store all the necessary traces to figure out like what exact problem was. It was a flaky issue when one time it repeats and the second one, it doesn't. And because we didn't have all of those necessary traces to reproduce exact particular problem, at that moment I understood that I need to think a little bit out of the box and like you need to design those kind of like things because the next time like when the manager calls you, you need to be prepared like to answer those questions and be ready to handle like what are your situations. Yeah, like that's kind of like shift I think is important and when you get all of those responsibilities like this kind of understanding comes to you like naturally and it's very hard like even though it sounds simple, but many developers don't get it unless they get in do like similar scenario. I appreciate you walking through that.

6:09Okay, across several projects growth doesn't break functionality right across projects growth doesn't break functionality. It can expose architectural limits and I'm sure you've experienced this many times. Can you recall a moment when it became clear that shipping more features wouldn't solve a problem and a platform approach was required? I have one example in my mind. So when we were working like in my previous company on a B2B product, we faced a situation where we got like rapid team growth like we tried to hire like many people from within the company like we tried to hire like some people from like outside the company and I guess we managed to get from maybe 10, 15 people as a core to about 80 like 90 people and it's a huge growth like we got that like from like for like during the period of half a year and lack of standards like cross team development, microservice approach like pushed

7:10development like in such a fast pace that when we eventually get the point that we have some features, we faced a situation that local development is terrible. It's nearly impossible to run like the whole environment. Integration between services like is just a complete mess. We even tried to like assign delivery managers from different teams just to make a release possible. So the delivery manager like goes to each team and ask them basically, have you guys deployed everything? Did you do this? Did you did that? And yeah like it made those features very complicated. And finally when we were ready to release like production flow and our first production flow basically was our internal test and when we ask our internal people to go and test like some flow, we figure out that the product can't handle like the load we created like within

8:13the company. I'm not even saying like we published it to some other external customers. And yeah, so this gave me like the understanding that like with such a pace like with such an approach, going further makes no any good progress at all. Like things has to be with a with thought, to rethink to in some way that the approach going to be different. And yeah, we started thinking about it and like we tried to make some microservices, try to put them together. We tried to introduce tracing. We tried to invest in like more mature local development flow, hard people that were responsible for like automation tests and the creation tests. And yeah, all of these together gave a lot of experience to understand that we need a platform approach. Okay, cool. Let's move into my next question here and with PDF filler started as a BDC product, but entering into BDB required public APIs and much higher reliability standards for business

9:16to business side of things. What was the hardest part of that transition from an engineering perspective, not necessarily a business one? Yeah. It's actually funny because the hardest part wasn't even like scaling infrastructure, but it was more about contracts and like formalizing interface, which you develop. So basically when we developed like some kind of like core functionality for the BTC part of the product. So we knew that we can develop these, we can develop that. You can go and talk to front-end engineers or you can talk with mobile devs. You can always introduce something else for them. If like previous part of the code like doesn't work, they can just use another point. They can work in any way like you agree with them during the call or like during the lunch or something like that. And yeah, this in turn, the here like it makes things so much easier and simpler to work with. So basically like you're focusing on completely different things.

10:20When you go like into B2B and you're trying to develop a public EPI, there's going to be used to by other companies, other teams, some sort of party engineers. Then like it's becomes more like about a public contract itself. Things like backward compatibility, like version in strategy, like rate limiting, so things like where like much more important now. And the design that you put into every end point, it's actually interesting because I was working with my manager a lot, spending like a huge amount of time just discussing a single EPI end point and how we should name like end point itself, how we should name like the request programs, response programs, IDs, like errors, like names of those errors, maybe like some kind of messages and stuff like that. This would make public EPI much more valuable for other people.

11:22And at some point, how easy it is to read it, how easy it is to work with this, how easy like it would be to develop SDK for such an EPI. This kind of like predictability and trust that this is what makes like just an EPI like a great EPI, I would say. This episode is sponsored by BrainGrid. Building with AI coding tools is exciting, until the moment things start breaking. You ask for a small change and suddenly three other features stop working. AI gets confused, misses edge cases, and loses track of your intent. The problem is not code generation, the problem is planning. That is why BrainGrid exists. BrainGrid acts as your product management agent. It writes clear specification, maps UX flows, asks the clarifying questions you forgot to ask, and breaks big ideas into engineering grade tasks that AI coding tools can build reliably. It guides cursor, clog code, replet, windsurf, and others so they deliver features that work and keep working.

12:23Founders use BrainGrid to build real AI native SaaS products, without a technical background. If you want reliable features, instead of fragile prototypes, try BrainGrid for free at braingrid.ai. Today's episode is brought to you by DotTech Domains, and this one hits close to home. Back in 2016 when I was building my own tech startup, I went on the hunt for that elusive.com. Looked high, looked low, and guess what I found? Nothing. What I did find cost me an arm and a leg. So I did what every founder does under pressure. Through an extra letters settled for the less than optimal name. And here's what I wish someone had said to me back then. Noah, you're building a tech startup. Just get a DotTech domain. Techstartup.techdomain. It could not be more obvious. It tells investors, customers, and anyone who looks at your website really, that tech is at the core of your build. And I've kicked myself plenty since. Especially when I see the clean and sharp names tech companies have landed on DotTech.

13:26So take it from someone who learned it the hard way. If you're building a tech startup, don't overthink it. Secure your DotTech domain today from any registrar of your choice. Very cool. So moving into around internal systems, examples like invoicing and HR workflows, they begin as automation, but eventually they evolve into real products. What tells you that an internal tool is worth developing seriously, rather than treating it as a temporary workaround? When I think about that, I think about winner use gaping processes together for an internal system versus really productizing something. I think that it really depends and it's hard like to reflect on this question from my experience. I can say that we try to develop internal tools that we've seen like a problem with because

14:28the company I was working in was about like document and workflow automation. Basically, like we've seen that, for example, invoice flow wasn't optimized properly like inside our company. And on the other hand, like when we started developing public EPI, we've noticed that before we go into real production with real clients, we still need to have some feedback about our API usage. And the best thing is to develop like a client for ourselves and see how it goes on a scale of some workflow like this. And what you figure out is like when developing such a tool, like such like internal tool, we were able to see all the kind of like design issues we made during prototype in this type of API and this helped us a lot to get all of those feedback. And on the other hand, we managed to create like an app that's really simplified and optimized internal flow. So all of those internal clients that are going to use the tool that you've built, they

15:32are going to give you like a very valuable and important feedback that you probably won't be able to get from real customers because they see it outside of your office and maybe like you need to analyze some kind of like logs and traces, which is completely different story. That's like solution that an approach that worked for us. I'm not sure that it's probably going to work for everybody and like everywhere, but again, if you're really looking into a good client for your system, maybe it makes sense to try to use your system itself by your particular needs and this will help you maybe finding some issues and maybe solving your own problems in one or another way. Very good. I appreciate you walking through that. Okay. So in R&D, you explored CICD automation, serverless infrastructure experiments, the whole nine yards, not all reached production. How do you decide when an experiment should continue? I don't even add in there when it should start and when it's no longer worth the engineering

16:35cost. Basically, like from what I've noticed, like from my personal experience, is when you trying to do some R&D work, it becomes very emotional. When you spend big or two or like even more like trying to understand some technology, then you spend some time like trying to develop a proof of concept and then like maybe prepare some presentation about it. But trying to be like not exactly trying, you are going to be looking like from the perspective when you want to sell this tool because you already invested like so many your personal time in it and this is the problem because sometimes it really makes no real sense or sometimes like perspective, maybe like managers perspective or like teams perspective could bring you a very valuable information like feedback. So answering the question, I would say that it's very nice and good to have team working with you together.

17:37So you can share some thoughts and hear some feedback and it's really cool if you're going to have a skilled manager that can lead you towards the right direction or maybe slow you down if you're trying to go too deep or like to or trying to become like too emotionally attached to some particular like current reward. Very good. Okay. So you've hired team set standards and shape long term technical direction and at what point does an engineer stop being a contributor and start owning business level outcomes and I'm really excited to hear what you have to say here because I think it applies across the board. Hard to say when exactly this happens and I assume it also depends on the environment you are in and the type of company you're working with. So for example, many outsourcing companies, people they are working like just from project to project and like when you talk to such people that their focus mostly is shifted towards

18:39delivering the project, like doing what requirements are and just working towards this kind of like cadence like where everything is repeated. And even there are some people that have some kind of like opinion about their architecture about like the things how it should be developed. They try to like get knowledge about what's products going to do. The problem is that the opinion usually doesn't matter in terms of like business and I can say even more that people who make decisions won't even able to hear them or have a conversation with them because they maybe sit like in different country, in different office and there are a whole bunch of operation managers that just proxy the information and because of that such a structure, this information would never like reach proper years. And because of that, people who choose to stay in such environments probably won't become owners of business level outcomes and on the other hand, there is small product oriented

19:42team or maybe like a small startup and there are like very few people there like maybe some found an engineer like few folks that works with him and they see what they are doing and the impact they do like the code they deliver like reaches the clients instantly just after the deploy they see like how it changes like on the real product and maybe like when they see that their work is appreciated when they try to have some conversation and they hear like real like discussion and basically like when the moment of appreciations happens, I think this is where like people starting owning like this business level outcomes. Very cool. Yeah, that makes sense. So you've contributed to open source tools that later became part of your company's infrastructure. Why do you see open source contributions as part of serious engineering work, rather than aside activities, I think some people can see it as side projects and scratching itches

20:47that maybe you don't do during the day to day job, but really can contribute to the whole thing, the whole picture of what you're trying to build. So tell me about that. Basically, like when you start a new project, you're basically trying to include some framework or library like in PHP world, for example, you can use like composer right to install something that are like many frameworks and majority like of those tools like frameworks are open source and many people don't think about it, but the fact is people invest a huge amount of effort to make those tools which later become part of your code. That's funny, but we trust many of those libraries blindly just because we try the community. If you take a look and add some popular libraries, for example, in GoLanc, there is PGX library for PostgreSQL. It has a huge amount of likes and community in PHP world, there is like a lot of framework

21:52like everyone knows about it. In JavaScript, there is like React and this is also like open source solution. And I think like we need to pay our debt back in one form or another. And I can say that even like in the company, like I'm working at the moment and like in all of those like R&D work in the past and so on, companies do a lot of research like proof of concepts, experiments, but you often you already have the code that is possible can be open source. You just need to maybe add a readme and publish it. And it's very simple to do, but like the all of those small steps, all of those small publications will create a whole bunch of like very interesting open source tools. And from a practical perspective, I can say that if you can publish your PSC code and in the future it becomes very useful for somebody, they can make a production ready solution, which means that you can get benefits of such a solution for free if your original problem

22:57is still relevant. What I've noticed recently is like developers, for example, in my team, understanding that we are going to open source like some component, they just starting thinking differently. They trying to put much more effort into developing the library because they know that this work will become their like public activity and maybe like part of their resume. And just saying let's open source it, it helps people to produce much better, much more code quality, stronger solution. And maybe like last but not least, like sometimes you need some experience working with something, but you don't have any project that you can work with or maybe like you work in the company and company doesn't have the stack you're interested to working with to work into. And I guess like the easiest way here is simply to come an open source contributor and you will get all of that knowledge and just doing so, you probably can simply jump from

23:59one position to another in terms of seniority by simply expanding like maybe an hour of time after work or something like on the weekend and this is so simple to do. Absolutely super simple and it doesn't take that long and that I really access to these types of things is just astronomically different than it used to be. This episode is sponsored by Mesmo. If you're responsible for liability, performance or platform architecture, you already know the problem. Telemetry volume is growing faster than teams can manage it. Mesmo addresses this by moving observability upstream. Instead of storing everything and asking questions later, Mesmo processes telemetry in motion, filtering, transforming and enriching logs, metrics and traces before they reach your observability back end. The results is cleaner data, reduced ingestion costs and faster root cause analysis using the tools you already rely on. Mesmo integrates with platforms like DataDog, Dinotrace and open source stacks, giving teams more control without adding operational overhead.

25:01This is especially useful for platform engineers and SREs supporting complex, distributed systems where context and speed matter. To see how active telemetry works in practice, get a demo at mesmo.com slash code story. That's m-e-z-m-o.com slash code story. This episode is sponsored by Unblocked. Your coding agents have access to your code base. Maybe you even connected other tools via MCPs, but access doesn't mean context. Agents can't reason across MCPs. They don't know your architectural decisions, your team's patterns, or why the API was shaped the way it is. So agents look in the wrong place and deliver bad outputs. Then you spend time correcting. More loops, more tokens. Unblocked is the context layer your agents are missing. It synthesizes your PRs, docs, slack and tickets into organizational context that agents actually understand. So they make better plans, write higher quality code, use fewer tokens and

26:03require fewer correction loops. If you're running cloud code, cursor or any agent work flow, Unblocked is worth a look. Learn more at getunblocked.com slash code story. If you look across your projects, I'm curious how you recognize a truly mature engineering system. Is it based on code quality? Is it based on process? Or how teams respond when things go wrong? How do you measure that? To be honest, my thinking about it's going to be maybe funny in some way, and maybe some people won't agree with my like vision, but I think a mature system is like the system that makes basically money, and where customers have their use cases covered. This is like the base, this is like the foundation. If you have these, everything else is secondary. So for example, it can be like something with a terrible code base, very bad design practices, no architecture at all, but unless like it makes money

27:10and solve the problem, who cares? So on the other hand, we can have a strong team. They can do everything solid, dry, Kubernetes, microservices, cloud, and their code is written like in Kotlin, Golan, Scala. It's great. From the engineering perspective, it's just some kind of like a ruby. And at some point that the project has no funding or like it's difficult to change product perspective, make a quick shift, like simple cost to pay for the infrastructure and the team like actually cost a lot of money itself. And funniest part, like when somebody is leaving the product, you can't find replacement due to the fact how complex or like how mature your particular architecture is. So basically summarizing all of this together, I can say that like mature system is like a balance of everything. And the highest priority I would still say that

28:11is going to be like making money. If you make money and like you have nice, friendly, responsible team that can do like trade-offs and keep it working and still keep it yet simple. And probably like architecture that can be like not that strong, maybe like somewhere like simple in some terms, but yet flexible for like business changes. This is where like it became like really mature. So yeah, long story short, wise trade-offs and maybe some money income. That's pretty much it. Very good. Okay, Alexander, last question. So if we look five to seven years into the future, which architectural assumptions we treat as standard today are most likely to turn out to be naive or limiting. This is, this will be interesting because this is a little bit of forecasting here. What do you think? Yeah, I'm not very good in terms of forecasting things. Let me try to think about it. Basically, everything like this change now like with AI. And I can say that AI will quietly or maybe like loudly affect many of our

29:18current architectures or maybe like architecture habits. I think this happens like in one or like another way, not because those are architectures or like habits are wrong, but because they were built like for the world where engineers are like a primary operators or like integrators, right? And very soon like places where like those engineers would become bottlenecks, like the first places which is going to be affected by this AI boom. I have a good example. So there is a book called Clean Code by Robert Simartin. And in this book and overall, this is what we should think about is like the way how we see how we treat a clean code like as a primary quality metric, right? And in they are like forward, they're like phrases, something like honesty is like in small things is not like a small thing or like maybe God is in the details or maybe something like because small things matters. And the fun part is if like now most of our

30:25code bases are AI generated, then all of those kind of like small things like about code elegance, it makes no sense anymore. And now what shift is going to be is like to focus more on like tests and contracts. If such an adoption happens, we probably would require like to make our CI CD pipelines much more mature and maybe put some much more advanced like gateways that will check our code base, that will validate that everything works. That for example, like test and test coverage would be like much more important here. Or maybe like deployment pipeline supposed to be like exactly something like kind of a deploy where like changes goes to 1% of traffic than 5, 10 or something else. And maybe like we need to like add some kind of like security checks that will validate all the compliances, vulnerabilities, secrets, etc, etc. Maybe like human focus

31:26in this case has to be shifted into defining some kind of like good policies and rollback strategies. So the production won't fail at some point. I can say also like that. And I really like the way how it goes. So about model context protocol MCP, the way how popular it begins now and how it grows. I can say that maybe like in some close future, it can replace all of the usual APIs we used to have and things like open claw or like none about the proof that like you can get 25,000 likes in two weeks, which is just crazy. Maybe like monolithic architecture would be much more popular against microservices because again, like all of those agents are much more productive when they work with a single code base rather than just when you try to do things across 20 like microservices and like then trying to glue them together like with

32:30some kind of like a contract. Long story short, right? Yeah, I will affect a lot of practices. I tried to maybe bring some examples, but who knows, I guess we will see soon. Yeah, for sure, we will see it and no one can predict the future. But I appreciate you sharing your forecast there and I really appreciate you being on the show Alexander. It's clear you've built and led teams to build platform architecture. You clearly indicate that a mature system is one that strikes a balance between all the right things and I really connected with your point on experiments and when to call them done. So I really appreciate you being on the show today and sharing your industry experience. Thanks for having me and really happy to give such an answers. Alexander highlights some really interesting points around moving from hands-on engineering into principle and technical leadership. You got to think a little bit differently into end architecture, infrastructure, platform thinking and the like and how to hire teams and explore experiments that support bringing features to market. If you'd like to connect with Alexander,

33:34check the show notes for all the appropriate links and thanks again for listening.