Jeffrey Palermo: AI-Driven Software Projects - Episode 418

0:00AI-driven software projects. The AI DevOps podcast is a show for those shipping software using AI, .NET, Azure, and DevOps. Each show brings you hard-hitting interviews with industry experts, innovating better methods and sharing success stories. Sponsoring the podcast is clear measure, a software architecture and engineering firm that implements AI to empower software teams to establish quality, achieve stability, and increase speed. And now your host, Jeffrey Palermo. Welcome to the show. I'm Jeffrey Palermo. Your hosts for helping you and your teams move fast and deliver quality and to run your software with confidence in Azure, all while using everything that AI and Azure and the .NET ecosystem has to offer. Today, I wanted to take a step back from the interviews and just describe what a modern AI-driven software project actually looks like. And just talk about that a little bit. I'm getting quite a bit of questions, just emailed in and linked in.

1:02And by the way, I'm easy to find there. And I love answering questions and talking to you guys. And a lot of you have been listening since 2018 when I started this podcast. But we're in a new world. And just want to talk through what we're learning so far and I welcome the feedback and the questions. And so this is what I think a modern AI-driven software project might actually look like. Or what it can look like today. And we don't know what it's going to look like years into the future or what it might look like someday. But what it can look like now on a team that has adopted these AI tools and worked them into a real software development lifecycle. And it really starts with a good foundation. All of the engineering practices and the software development processes that are industry as built up over time are still the foundation. None of it goes away. And we have to have

2:04that foundation. We have to have a complete and clear development lifecycle. We have to have a work tracking system so we can track the work. We have to have a DevOps environment. We have to have the full feedback loop open to limit. We have to go from the product management vision all the way through production. We have to connect that to limit. We have to go back with all kinds of early information around the product vision so that we can have that full cycle from having a market idea to put it in place and figuring out how it went. That's our foundation. All of all of those big topics that our industry has wrestled with and matured on over the decades. Those are still where everything begins. So how does AI make that different? It makes a lot of things different. But gener of AI, it takes the amount of the schedule, the amount

3:06of time spent per day that goes towards the typing of characters and the typing of lines of code. And it shifts that time from being a big significant portion of the effort to just a very small portion of the daily effort. And to understand what that means, we have to break down how much time is actually spent on various types of activities when we're making a change to the software. And so let's maybe look into one of those samples. All right, let's say that I'm in the development portion of the work and I'm ready to make some code changes. My hands are on the keyboard. Ready to go. Here's what actually happens. Now, first I'll load into my head what I'm actually doing and why that's important. I need to remember it because I've got all kinds of things that I'm working on. But when I start to work on one task, I need to hone in and remember, okay, which one is this one? I need to understand the concept. Remember the details around that. And that means I'm either reading

4:06some stuff or remembering some conversations. So I need to remember what I'm doing and understand that concept. Next, I need to review the user experience information. That's what it's going to look like, how someone's going to use it so that I make sure that I'm not that I'm not missing anything about how someone is going to expect this change to manifest and how they're going to use that software once once I'm done with it. And that'll be some conversations. Maybe I've had or I'm reading something that we wrote down. I'm reading the body of a work item ticket in my work tracking system. After that, then I remember or review the technical decisions that we've made around how we've decided that we're going to implement the change. Does it require any new architectural elements? Does it require any new design patterns, libraries, or is everything that we have in the software perfectly sufficient for this change because it's small? After that, then I need to wrap my head around the definition of done. What is in scope,

5:10what's out of scope of all the things on the board? Exactly what is this particular item? What does it mean to be done with it so that I don't accidentally increase the scope? And I'm also thinking about when it's done, what am I going to do to test it so that I know that it's done? After that, I start writing the code. I go through my test room development, loop, red, green refactor, I write a test, see it fail, make it pass, drive it to green, and then do some refactoring, make sure I'm happy with it and rinse and repeat. Then I run the software locally as a sanity check. When I think that I'm done, I'm going to back merge from the master branch because my teammates are also working and there's some changes coming into the master branch and I pull those into my branch so that stays really, really close to master because of merge conflicts. And then when I think I'm done, I run the software again, everything's good, I run my private build again locally on my workstation. Then I do my check-in dance, I create my pull request,

6:15it push the leaves to the branch, create the pull request, and then I'm watching that pull request for anything that the integration bill is going to catch or the static analysis checks. And I fix any of those and I draft a good explanation about what the pull request is and any decisions that I made during the implementation. I probably respond any comments depending on what kind of interaction we have on pull requests and I make those adjustments. And then if I'm good with that process, then I'll merge the pull request and then monitor the master branch for the integration bill to run again, make sure that I'm green, and then I move on to something else. And that particular work item is done from a development standpoint. And then of course we need to do some functional validation and user experience validation. But as far as the development portion where code is going to change, that's the process. Now that's a lot of steps. And all of that flow, that entire workflow for just what happens when the card is in the development column on our work tracking board.

7:20All those steps, even for something small, something that might only take a couple hours to actually put in place, that's kind of a minimum of half a day. And let me if it's a typo, you know, sometimes you get really quick, but a lot of times even small things, they take half a day. And if it takes me a little while to figure something out, if it's not quite clear, that's a full day. And so with AI, the mechanics are changing this half a day this, full day that, well, the development mechanics have now shrunk in duration to a couple of hours, because what they're run by an AI tool. But more importantly, I can now have half a dozen of these independent changes, automating those mechanics. Same mechanics, but I can have them in parallel, doing it at the same time, during those same calendar hours. And that's the multiplicative effect

8:23of automating the programming mechanics. So I'm personally spending less time on the mechanics of software development, the typing part. And the running it locally and the testing every detail and red ring refactor and running all the tests over and over again in private builds. These are the actual mechanical pieces of programming and typing. And that is changing. Now, where does the time go instead? It's not like we just work less, but the time is shifting. So, so here's here's what's interesting. And I think I think that a lot of people are observing this. As a percentage of my day, I'm spending the same number of minutes thinking and advising product managers and people I work with on what might be possible with software, helping them hash through the concepts, asking good questions. And the same amount of time assisting in, you know, proposing the appropriate user experience. How someone might might use it. We're constantly having conversations.

9:27I'm probably spending the same amount of time doing technical design to figure out how we might make something technically happen. I'm probably spending the same amount of time scoping things and doing test design. Probably the same amount of time doing functional validation, pushing things to the next environment, helping out with user experience validation and user level testing, UAT, things like that. I'm of course answering questions, providing feedback to the product team is they're using it in the test environment. I'm probably spending the same amount of time supporting production, second level customer service, the same amount of time with all kinds of these other aspects outside when the card is just in that development column. So, the time that I'm saving on the mechanics of the programming itself, that gets more development done. And that savings that time gets reallocated to do incrementally more of the other activities. So, what happens is as a percentage of my day, I go from maybe three quarter time

10:33typing, like acting programming with my hands on keyboard typing characters and lines of code and authoring and changing and reviewing lines of code. It's been maybe three quarters of my day doing that to doing more design work, more validation work, more architecture work, and more software strategy work. And so, I think about what's actually happening that makes people able to deliver more software. So, the mechanics of programming for a subset of changes that are compatible right now, for a subset of changes, the mechanics of programming, it doesn't require me personally to type those lines of code. And more and more people on the team are empowered to deliver more. For example, all right, let's take, I love this example. If I'm the only one on the team who understands how to use a particular library to do something and a simple change comes through, simple change.

11:36Well, then conceivably, anybody on the team should be able to do something with it. And now with AI, literally anybody on the team, as long as they understand the architecture and the dependencies, they can use an AI tool to get simple changes done and validate it. There's the learning is accelerated. Also, for very simple UI things, like maybe aligning something to the left or to the right, let's change a screen in some way, maybe putting something in all caps, maybe fixing some spelling errors in a properly empowered AI-driven project. Does it actually need a fully trained software engineer to get those changes into the software or could an analyst or a product manager, you're properly trained and equipped with the AI tools and this particular software environment, could those types of people shepherd that kind of change through? They do have to have the

12:41expertise to validate the change since it's in the user interface, but spelling errors, adjusting screens, they certainly know that. So a subset of the software can be changed by a larger pool of people. Now, we're not talking about every change. We're talking about the easy stuff. It's the easy stuff. We're learning different ways to use the tools that we have. There's several ways to use the programming specific AI tools that are currently around us. I think everybody agrees on the broad categories. The first is the auto-complete category while you're writing the code directly and tap, tap, tap, tap. Then you move to prompting an embedded AI tool in the IDE and you observe every line of code that's changing. You curate it all. You're monitoring it and you use it effectively. You really save a bunch of typing and a bunch of searching. The next method is to give the AI

13:43agent a task and let it work on that task and then you come back and decide if you like it. After that, the next category is to give it a task and actually give it multiple sessions, multiple tasks in parallel and then you have your builds automatically check 80% of your quality standard. You really rely on your private builds and your automated test suites and your static code analysis and other checks. You really rely on that to make sure that what came through actually meets your team standards so that the review that you with your eyeballs is manageable. Then the pinnacle, that's final category, the pinnacle of what we've seen teams start to really just scratch the surface on is the AI software factory concept. This is where one or more stations in the software development lifecycle have completely automated workflows, at least for some subset of the work that goes through the software team. What that subset is depends on the level of sophistication of your

14:51AI software factory implementation, what type of software it is, what the climate of your organization is, and some of the easiest situations are those spelling typos or small UI adjustments. It doesn't really require any engineering, like I mentioned, just a little bit of adjustment in the code, just a little bit of programming. We reserve the heart problems for our own brains, the things that don't have obvious solutions. If it has an obvious solution, if it's small, if we've done that before, then we dispatch it into the AI software factory. The software factory fully automates our existing software development lifecycle, the existing quality standards. It does that for any of the changes that are compatible with the sophistication level of the software factory implementation that we've put in place for our team. That really saves time. It saves a lot of time for our development team to work on harder

15:57problems. That's what's possible today. I can't really predict what we're going to be next year, we're going to five years from now, but as we move forward, and as our industry learns how to leverage this generative AI technology and all of these coding tools, more and more software is going to come along, because businesses are realizing that they can do more and more with the teams that they have. Our teams can do more and more, and so we're not seeing. I mean, different companies are going to have layoffs and whatnot, but as an industry, we're not seeing the big contraction. We saw that some companies rise, some public laying off a whole bunch of people. I think those were just adjustments that individual companies, but as an industry, as a whole, what we're seeing is business leaders realizing that they can be more and more ambitious.

16:57As we go forward, the skill set that we all need to have is going to shift a little bit. The architecture skills are becoming more and more important. The structural engineering skills of software are becoming more and more important, and it just so happens that the speed of typing and the importance of that skill, maybe it might not be as important as it was for the first two decades of my career. I learned to be pretty good at typing and coding and all the re-sharp or shortcut keys and I really hone that skill, but as we go forward, man, I'm doing a heck of a lot more with voice dictation and being faster at it. As we move along, all kinds of the bounce of skills have always continued to change, and we're observing that. I'm doing a heck a lot more architecture, even in my own individual projects. As we go forward, I believe that for all of you, equip with the right tools and skills and processes. Every one of your teams

18:02can deliver world-class software results. No question. That's always been true, but in this time with generative AI, more than ever, we're learning together, the industry is shifting, and it's a fun time. So, happy coding, dear listener, and keep shipping. You've been listening to the AI Demops Podcast. You can find us on YouTube, Apple podcasts, Google Play, and everywhere else. Visit our sponsor ClearMeasure at ClearMeasure.com. And on behalf of your host, Jeffrey Palermo, thanks for listening and may God bless you.