SE Radio 737: Owen McGirr on Software Accessibility

0:00This is Software Engineering Radio, the podcast for professional developers on the web at sc-radio.net. SCRadio is brought to you by the IEEE Computer Society by IEEE Software Magazine, online at computer.org slash software. Welcome to Software Engineering Radio, I'm your host Gavin Henry, and today my guest is Owen McGurr. McGurr is a Donnagall-based software developer and entrepreneur who builds inclusive technology. He is the creator of Say It, an iOS app, and the Switchify app, which allows you to use an Android phone or tablet without touching the screen and compare with Switchify PC to control a Windows PC. In 2022 he was honoured in the MIT Technology Review innovators under 35 lists for creating Say It. Owen, welcome to Software Engineering Radio, is there anything I miss in your bio that you'd like to add? Thanks for having me on Gavin, I would only add that accessibility is not just an area I work in professionally.

1:06It is something I depend on every day. I have cerebral palsy, I am an AAC user, and I use assistive technology myself. So I am really looking forward to talking about accessibility from both sides. As a developer, building this kind of software, and as someone who depends on it every day. For me, this is not an abstract engineering topic. It affects whether I can work, communicate, and use the same tools as everyone else. That gives me a very practical view of what works, what looks good in a demo, but falls apart in real life, and where software can do better. Through the show we're going to be talking about what accessibility is. We want to understand the different ways of using Interact Software. We want to understand how this is currently handled between various operating systems. We're going to explore the technical challenges of creating the app, say it, and the Switchify app. And we're going to discuss what developers can do to improve accessibility.

2:11And there's some other podcast episodes that you might like to listen to after this. SE Radio Episode 329, which is Andrea Staphick on Accessibility for the visually impaired. That's a really good one. An episode 686 from France, so stiosed from W3C. I'm not sure if I pronounced that right. And there's a couple of articles from the IEEE Accessible Learning Labs about accessibility education through learning. Engineering Accessible Software, that's a good article. A brief survey of how things are done in continuous integration and automated testing. So that's something we're after. To lay the foundation, I'd like to start with an overview of what accessibility is. So what does accessibility mean? Accessibility means designing software, so people can use it in the way that works for them. Not only in the way the developer happened to imagine.

3:12That might mean using touch, keyboard, switch access, voice, screen readers, iGays, headtracking, or another access method entirely. It also means thinking about things like fatigue, speed, error recovery, cognitive load, and whether someone can complete a real task independently. To me, accessibility is not a separate feature or a layer you add at the end. It is a measure of whether the product can actually be used by the people who need it. Thanks. So what assumptions are normally made when creating apps or software? A lot of software assumes the user can see the screen clearly. Touch small targets accurately, move quickly, read comfortably, hear audio, and recover easily from mistakes. It assumes the user has a mouse, or direct touch, or two hands, or enough fine motor control to hit a small icon first time. It also assumes the user can react within whatever time the software decides is reasonable.

4:13One example I absolutely hate is a dialogue or warning that times out before I've had a fair chance to read it and respond. That kind of design assumes speed and motor control that not everyone has. There are also structural assumptions that focus will not matter. That a custom control does not need a proper role. That a drag gesture is fine as the only way to do something, or that a user can easily recover if they press the wrong thing. More generally, software often assumes the user interacts in the same way as the developer. Those assumptions are often invisible until someone with different access needs tries to use the product. Can accessibility be bolted on at a layer stage? Sometimes you can improve accessibility later, but it is much harder and usually less effective. If accessibility was tacked on, you know right away as a user. The labels might technically exist, but the flow feels awkward, focused jumps around, or the feature only works for the simplest case.

5:17There is also a legal and business side to this. Companies sometimes only take accessibility seriously after a complaint, a lawsuit, or a compliance deadline. But by then they are trying to retrofit accessibility onto design decisions that were already made. That is more expensive, more stressful, and usually produces a worse experience. If the structure of the app is wrong, if controls are custom without semantic meaning, or if flows depend on gestures only, you can end up fighting your own design. Accessibility works best when it is part of the design and engineering process from the start. Excellent, I am going to move on now to talk about different ways of using my interact with software. So what could people use to interact with software? There are things like iGays, Touch, Voice, if you can talk, keyboard, mouse, and things like switches. Can you take us through that?

6:17People might use Touch, mouse, keyboard, voice control, screen readers, switches, iGays, head tracking, facial gestures, sip and puff devices, or combinations of those. The important point is that there is no single default user. Good software should not assume that direct touch or a mouse pointer is the only valid input method. Can you give me an example of good accessibility? A concrete example of good accessibility is a checkout flow or sign up form that works from start to finish without a mouse. I can press tab and move through the fields in a sensible order. Each input has a real label, not just placeholder text. If there is an error, focus moves to the error or the error is announced, and I am told exactly what to fix. Buttons are real buttons, links are real links, and nothing important is hidden behind hover, drag, or a tiny icon with no label. If there is a timeout, I get enough warning and a way to extend it.

7:19The same idea applies outside the web too. A good app exposes its controls to the platform accessibility APIs, so a screen reader, switch system, keyboard, or automation tool can understand what is on screen. I would encourage developers to throw away the mouse for the day and try to use their own product with the keyboard or another access method. You quickly find out what actually works. And for some quick wins for us and developers. The quick wins are mostly about using the platform properly. Use native controls where possible, because a native button already knows it is a button. Can receive focus, can be activated from the keyboard, and exposes useful information to assistive technology. If you build a custom control, you have to provide the same name, role, state, focus behavior, and keyboard behavior yourself. Label every interactive element. Keep the focus order logical.

8:20Make tap targets large enough. Support dynamic text sizes. Do not use color alone to communicate meaning. Make errors clear and put focus somewhere useful when something goes wrong. Test the main workflow with only tab, shift tab, enter, and escape. Then test it with a screen reader. Then test it with switch style navigation if you can. A useful rule is, do not only test whether each component is accessible. Test whether the whole task is accessible. I have also been experimenting with Codex and Switchify recently. I wrote a harness where Codex looks at a screenshot, reasons about what is on screen, and chooses which switchify action to send over ADB. As if it only had switches. Some of the results gave me pause. If the AI could not understand the UX from what was visible on screen, that made me think I should rethink parts of the UX,

9:21because a real user might have the same uncertainty. That is not a replacement for disabled user testing, but it is a useful way to stress test assumptions. Automated checks are helpful, but they only catch part of the problem. Real workflows with real users are where you find the issues that matter. Now I'm going to move on to talk about apps that you created on. I'm you created say it. Are you using that to talk to me now or something else? Because you're generating this audio, aren't you? I am not using the original say it, iOS app for this call. I am using podium speak or podiumspeak.xyz, which is a web app I built for delivering longer spoken pieces. I originally created it for Dr. Claire Malone, a scientist and science communicator with cerebral palsy, because giving a talk or presentation is a different problem from everyday AAC conversation. In a conversation, you need speed and flexibility.

10:22In a talk, you need reliability, prepared sections, consistent speech, and a way to move through the script without losing your place. podium speak is built as a PWA, so the speech can be prepared and the talk can be presented from the browser in a more app-like way. For this podcast setup, the speech output is routed through OBS, so the call hears it as my microphone input, rather than picking it up through speakers in the room. Thank you. So let's explore say it and the other app you created called switchify. How does say it work? Say it started as an iOS AAC app, but the current direction is web-based. It is now built as a next JS progressive web app, so the user can install it and use it more like an app, while still getting the benefits of the web. The basic idea is that the user types what they want to say, and the app speaks it using text to speech. But the important part is not just calling a speech API. AAC design is about reducing the number of actions needed to say something useful.

11:25That means saved phrases, fast access to common sentences, categories that make sense to the user, picture support for buttons, and a predictable layout, so the user is not searching every time. It also means thinking about fatigue and conversation timing. If I have to press 20 things before I can answer, the conversation has already moved on. The web version gives me more flexibility across devices, and as a PWA, it can be closer to an app, while still being easier to update. The challenge is making that feel reliable enough for communication, because this is not just a productivity tool. For an AAC user, it can be their voice. How fast can users speak using it? It depends heavily on the person, the access method, and the prediction or phrase tools available. Typing with direct touch is very different from typing with one switch, iGa's, head tracking, or scanning. If someone is using a single switch, every character can involve waiting for the scan to reach the right item,

12:28selecting it, and then doing that again and again. Saved phrases, prediction, abbreviations, and good layout can make a huge difference because they reduce the number of selections. But I think words per minute is only one part of the story. In a real conversation, timing matters. If there is too much latency, or if it takes too long to repair a mistake, the AAC user can be left behind even if the system is technically working. So I think about speed in terms of participation. Can the person greet someone, answer a question, interrupt politely, make a joke, recover from an error, and keep enough energy to continue? That is a better measure than just raw typing speed. Can you give me some examples of the technical challenges you face creating here? For example, did you write native code? And was AI around when you built this? The hard parts were not only speaking text.

13:28The original iOS version of Say It was native and written in Swift. The current version is a TypeScript web app built with next JS as a progressive web app, so the technical problems are different now. On iOS, I was working inside Apple's native app model and speech APIs. On the web, I get more reach across devices, but I also have to deal with browser text-to-speech, offline behavior, installability, performance, storage, and consistency across devices. Browser speech support is useful, but it is not identical everywhere. So you have to think about voice availability, timing, volume, interruption, and what happens when a browser or device behaves differently. For AAC, latency matters because the delay changes the rhythm of conversation. The layout also matters because every extra action costs time and energy. Saved phrases, prediction, categories, button size, and recovery from mistakes are not just nice UI details.

14:30They affect whether someone can participate. When I started this, AI was not around in the way it is now. Today, AI can help with coding, refactoring, testing ideas, and exploring designs, but the core AAC decisions still have to come from real use. Because an AI does not know what it feels like to rely on the software as your voice. And the switchify app, why did you create it? Switchify is an Android Accessibility Service for people who cannot comfortably use a touchscreen. At the surface level, it lets someone control an Android phone or tablet with switches or camera-based facial gestures. Under the hood, it watches Android Accessibility events, reads the accessibility tree, filters the screen down to actionable nodes, and builds a scanning model over those nodes. If the app exposes good accessibility information, switchify can use item scanning to move through real UI elements. If the app does not expose useful nodes, it can fall back to point scan or radar scan,

15:35where the user chooses a position on the screen instead. That fallback is important because not every Android app is built well for accessibility. When the user selects something, switchify can dispatch taps, swipes, long presses, scrolling, system actions like home and back, and more complex gesture patterns. It also has an internal menu system for things like gestures, scrolling, media control, editing, and device controls. Input can come from physical switches, keyboard style switch events, or camera-based facial gestures. But internally, those inputs are mapped onto switchify actions such as select, next item, previous item, change direction, or pause. The newer PC side extends that idea beyond the phone. The Android app can pair with switchify PC and send authenticated commands for mouse movement, clicks, scrolling, typing, and window control,

16:36so the same access method can control a Windows computer. I created switchify because I wanted switch access that fitted how I actually used technology. I had been using iOS for about 10 years, and I could access a computer through Grid 3, but I wanted more choice and more control on Android. Android is a powerful platform, but the switch access experience did not feel flexible enough for me. I wanted something that could scan real UI elements when that worked. Fallback to point or radar scanning when it did not, support camera-based gestures, and let me configure actions around my own movement rather than forcing me into one model. It started as a personal need, but the more I built it, the more obvious it became that this was not just my problem. Many users need an access system that adapts to them, not the other way around. Switchify PC came from the same idea. If I already have a good access method on Android, why should that stop at the phone?

17:36I wanted the same switches or gestures to control a Windows PC as well. What is it running? Switchify Android is a native Android app written in Kotlin. The settings and normal appscreens use Jetpack Compose, but the core product is not really the settings UI. It is the Accessibility Service. That service is the long-running part that listens to Android Accessibility events, reads the Accessibility Tree, builds the list of actionable nodes, manages scan modes, draws overlays, handles switch input, and dispatches gestures back into Android. Internally, there are separate layers for scanning, switch events, access techniques, menus, gesture execution, camera switches, and PC control. For example, item scan works from Android Accessibility nodes, while point scan and radar scan are more spatial and coordinate-based.

18:36Camera-based switches use the Android Camera Stack, Camera X, and face gesture detection so facial movements can become switch actions. Settings, switch mappings, gesture patterns, and user preferences have to be stored reliably because a broken configuration can make the app hard to use. Switchify PC is a separate Windows companion written in CIS and .NET. It runs in the tray, uses Bluetooth for pairing and reconnect, validates authenticated protocol messages from Android, and turns those messages into Windows mouse, keyboard, text, media, window, and status actions. It also has to care about Windows-specific things like packaging, code signing, installation under program files, and UI access, because Accessibility software sometimes needs to interact with windows that ordinary apps cannot control. Could you draw across the screen easily? If you mean the Accessibility Service UI, the main framework is Android's Accessibility Service API.

19:39It gives you the special position in the system. You can observe Accessibility events, inspect the active window through Accessibility nodes, draw an Accessibility overlay, perform global actions like back or home, and dispatch gestures. That is a powerful starting point, but it is not a switch access framework by itself. The API gives you access to the screen structure and a way to place UI above other apps. Switchify still has to decide what should be highlighted, how scanning moves, what happens when the user selects something, and how overlays stay aligned with a screen that is constantly changing. So the answer is, Android gives you the Accessibility Service API, but the actual Access technique, scanning behavior, visual feedback, menu system, and recovery logic all have to be built on top of it. What frameworks were there, and what did you have to build from scratch? The main framework was Android's Accessibility Service API. It gives you primitives, Accessibility events, Accessibility node info, global actions, gesture dispatch, and the ability to draw an Accessibility overlay.

20:49But it does not give you a ready-made switch access product. I had to build the access layer on top of those primitives. The scanning model, the scan tree, highlighting, the menu system, the mapping from switch events to actions, point scan, radar scan, item scan, gesture lock, gesture repeat, and recovery when the user makes a mistake or the screen changes underneath them. The platform can tell me what nodes are on screen, but it does not tell me the best order to scan them, how to group them, how to escape from a bad state, or how to make the interaction understandable to someone using one or two switches. That is the layer switcher file has to provide. Are there any gotchas with setting it up before using it that you have to work around? I have tried to make setup as painless as possible because accessibility tools have a bootstrapping problem. The user may need accessibility before they can easily enable Accessibility. On Android, there are still things the platform requires, like enabling the Accessibility Service and granting camera permission if the user wants facial gesture switches.

21:55So the job is to guide the user through that with as little confusion as possible and then make the service reliable once it is enabled. The service also has to survive normal Android lifecycle behavior, device lock and unlock, app updates, and different manufacturer settings. For switchify PC, there is another setup layer. Pairing over Bluetooth, confirming the device, keeping the PC app running in the tray, and installing it properly so Windows allows the right kind of input automation. Those setup details sound boring, but they matter because if setup is too hard, the Accessibility feature might as well not exist for the person who needs it. Did AI play a role in creating this at all? Yes, a huge role. I would say AI helped with about 95% of the implementation work on recent switchify development. That does not mean the product idea came from AI, the need, the design direction, and the judgment about what is actually usable came from my own experience as a switch user and AAC user.

23:02But AI has been extremely useful for writing code, refactoring, generating tests, exploring edge cases, and moving faster through Android and Windows implementation details. I have also been experimenting with AI as a testing tool. One example is the Codex and Switchify harness I built. It takes a screenshot, asks Codex to reason about what is on screen, and then sends a Switchify action over ADB as if the agent only had switches. That has been useful because when the AI misunderstands the UX, it makes me ask whether the UX is actually clear enough. I do not see AI as a replacement for disabled user testing, but it can be very powerful when the person directing it understands the access problem deeply. Let's go into more detail about testing with AI. Can you take us through what AI is doing here? You mention it takes a screenshot. The important distinction is that the AI is not directly controlling Android.

24:02I wrote a parent harness that owns the emulator, ADB connection, screenshots, Switchify configuration, and every action sent to the device. The AI is used as the perception and decision making part of the test. I start by giving the harness a goal in ordinary language, such as opening YouTube or navigating to a particular control. The harness builds and installs the latest debug version of Switchify on an Android emulator. It checks that the package is installed, the accessibility service is enabled, and Switchify's private debug bridge is responding. It then backs up the user's existing Switchify settings and configures a known test profile using manual item scanning. The test is deliberately limited to an emulator. It does not run against somebody's real phone, and when the test finishes, it restores the Switchify settings and Switch mappings it backed up. There is even a separate cleanup process watching the runner, so the settings can still be restored if the main test process crashes.

25:02Once the environment is ready, the harness takes a real screenshot using ADB. The screenshot is passed directly to Codex as an image, so the model is reasoning from the same visual information available to a user. I do not give it Android's accessibility tree or let it inspect the interface with UI automator. Partly because that would give it information a Switch, user might not have, and partly because Android's automation APIs can interfere with an accessibility service while it is being tested. The screenshot is supported by a small amount of diagnostic context. The model is told which application is in the foreground. Whether the Switchify accessibility service is enabled, the screen dimensions, recent relevant log messages, and the sequence of Switch actions already performed. That action history matters because a single screenshot cannot always tell you whether scanning has just started, whether the highlight has moved, or whether the system has returned to the same state. I also give the model a technical operating guide for Switchify.

26:04It explains that scanning is hierarchical. The highlight might represent a row, a group within that row, or an individual actionable item. It explains that selecting a row may drill into it rather than activate something immediately. It describes the primary and secondary highlights, the escape states, keyboard scanning, and the Switchify action menu. Without that context, the model would naturally interpret the screen like a touch interface and assume that selecting something is equivalent to tapping it. For each step, I launch an isolated AI decision process. It receives the goal, the current screenshot, the operating guide, the diagnostic state, and the recent action history. It then has to return a strictly structured JSON response containing what it observed, what it thinks should happen next, why it made that decision, whether the goal has been achieved, and how confident it is. In the normal test profile, it can choose only five responses. Next, previous, select, capture, or stop.

27:05It cannot invent an ADB command. Tap a coordinate, swipe the screen, launch an app directly, or use Android back to escape when it gets stuck. Even if the model returns something outside that list, the parent harness rejects it before anything reaches the device. When it chooses an action, the harness sends an ADB broadcast to Switchify's debug testing bridge. Switchify converts that into its own internal Switch action, such as move to the next item or select the current item. That action goes through the same scanning manager and accessibility service logic used by Switchify itself. So the AI is not pretending to press the touch screen. It is interacting through Switchify's Switch control pathway. The harness then captures a new screenshot and the cycle repeats. Observe reason. Choose one Switch action. Execute it and observe the result. Each step uses a fresh AI process, so it cannot rely on an undocumented private memory of what happened.

28:07Any context it needs must come from the recorded screenshots, state and action history supplied by the harness. The model can also report strange behavior as a structured finding. It records what it expected, what actually happened, the visual evidence, the severity and even possible areas of the Android code worth investigating. The complete run contains every screenshot, decision, action, warning and finding, followed by a final screenshot and a summary. That makes a failed test reproducible instead of leaving me with only the AI got stuck. What interested me most was where the AI made the wrong assumption. This is not about claiming that an AI perfectly represents a disabled user. It does not. But when it repeatedly misunderstands whether a highlight is actionable, whether selecting has moved down a level, whether a menu is part of the expected workflow, or whether anything happened after a Switch press, that gives me pause. The model has the instructions and the visual evidence, yet the interface is still communicating something ambiguous.

29:11That makes the AI useful as a deliberately unfamiliar user. Its mistakes can expose places where I have relied on knowledge that exists in my own head because I designed the system. The result is not automatically proof of a user experience problem. But it gives me a precise point to investigate with real Switch users rather than assuming the interface is obvious simply because it is obvious to me. Are you mainly writing prompts or are there accessibility targeted tools you're using? Environment is everything. I am using a general purpose AI model rather than a specialized accessibility testing model, but I have built an accessibility specific environment around it. The prompt is only one part of that environment. You can write an enormous prompt explaining Switch access, but if the agent still has access to a mouse, touch screen coordinates or unrestricted ADB commands, it will naturally take the easiest route. It might complete the task, but it will not have tested the experience that a Switch user actually has.

30:12A successful result would therefore tell me very little about Switchify. The important work was deciding what the agent could see and what it was allowed to do. It sees a screenshot and a limited amount of device state. It receives a Switchify operating guide explaining scanning, highlight levels, menus, and escape behavior. Most importantly, it can normally choose only next, previous, select, capture, or stop. The parent harness rejects anything outside those actions, so I am not merely asking the model to imagine that it uses Switch's. I have removed the alternatives. The environment makes Switch access the only available way to progress. It is the AI equivalent of telling a developer to put the mouse away for the day, except the harness can actually enforce it. There are accessibility specific components involved, but they are components I created around the general AI's, Switchify's debug bridge, the manual scanning test profile, the action restrictions, the operating guide, and the evidence system.

31:13The model itself is general purpose. The accessibility knowledge and constraints come from the environment. That distinction matters beyond this project. Prompting can influence what an agent says it is doing, but the environment determines what it can really do. If we want meaningful accessibility testing from AI, we need accessible test environments. Representative input constraints and evidence of the route taken. Otherwise, the agent may reach the right destination through a pathway the actual user does not have. Are there any accessibility challenges using AI and agents? Honestly, the biggest accessibility challenge is very simple. I wish I could speak to the agent. Working with AI is supposed to feel conversational. A speaking user can explain an idea, interrupt when the agent misunderstands, add a missing detail and redirect it almost immediately. As an AAC user, every one of those interventions takes more work.

32:14I have to compose the message, navigate the interface, and make sure the agent has not moved on or opened something temporary while I'm still preparing my response. The irony is that the AI can generate and modify code extremely quickly, but communicating my intention to it can still be the slowest part of the process. The technology has accelerated the computer side of the conversation without necessarily making the human side equally accessible. Most AI products seem to assume one of two users. Somebody typing quickly on a keyboard or somebody speaking naturally into a microphone. AAC sits awkwardly between those two models. My communication is text-based, but that does not mean it should be treated like ordinary keyboard input. I may be using switches, scanning, predictive text, or generated speech, and each turn can take considerably longer. That is why timeouts are particularly bad. A temporary dialogue might be a minor irritation for somebody using a mouse, but for a switch or AAC user, it can make an interaction impossible.

33:17By the time I navigate to it or compose a response, it may already have disappeared. If a decision requires my input, the agent should pause and wait indefinitely rather than assuming silence means I am not there. A better agent interface would understand that communication speed is not the same thing as understanding. It would let me pause the agent while I compose, cue additional instructions, interrupt using an accessible switch action, and respond to prompts without a countdown. It would also present its current status in a form that works with scanning and AAC rather than relying on rapidly changing visual messages. So when I say I wish I could speak, I do not simply mean adding another voice control button. I mean I want the same immediacy in the conversation that speaking users already have. AI has enormous potential for disabled developers, but the interaction model must recognize different ways of communicating instead of treating speech and fast typing as the defaults. Have you found there's a lot of context switching when using AI?

34:20Agents are getting better at maintaining context and I tend to use massive threads rather than constantly starting again. A long thread becomes a kind of working memory. It contains the original idea, the decisions we made, corrections I gave the agent, and the reasons behind them. I also plan before asking the agent to implement something. That reduces context switching because I am not making every design decision reactively while the agent is already changing files. I try to establish the goal, constraints, and likely stages first, then let the agent work through them. This matters more to me as an AAC user because changing context is not just mentally disruptive. Every correction, repeated explanation, and new instruction has a communication cost. If an agent forgets why a decision was made, reconstructing that context may take me considerably longer than it would take a speaking user. There is still a limit to massive threads. They can accumulate outdated assumptions and the agent may give too much weight to something said much earlier.

35:24So, planning is important. The thread needs a clear direction and major decisions should be written down in the repository or another durable source rather than existing only in the conversation. Overall, context switching is improving as agents become more capable of carrying a task from beginning to end. My preference is to give an agent a substantial piece of work, keep the context together, and intervene only where my judgment is genuinely needed. The best agent experience is not one where I supervise every click. It is one where I establish intent. The agent maintains that intent across the work, and it returns with evidence of what it actually did. What could be done to make this experience better? The biggest improvement for me would be better control over the agent while it is working. Agents can now do much larger pieces of work independently, which is useful. But the controls around that work still assume somebody can react quickly with a mouse or keyboard.

36:24I need a reliable way to pause an agent, add information, redirect it, and then let it continue without losing its progress. The housing should freeze the task in a known state. It should not cancel the work, discard the context, or leave files half-changed. When I resume, the agent should briefly confirm what it was doing. Incorporate my new instruction and continue from that point. Interrupting is especially important. A speaking user can often stop someone mid-sentence as soon as they notice a misunderstanding. As an AAC user, I may recognize the problem immediately, but need time to compose the correction. By the time I send it, the agent may have made several more decisions based on the original misunderstanding. There should be an accessible hold action that stops new work immediately while still giving me unlimited time to explain why. Those controls must be properly keyboard and switch accessible. Tab and enter are your best friends, but only a focus order is predictable and every control can actually be reached.

37:28Pause, stop, resume, and redirect should have stable positions, clear labels, and configurable switch actions. They should not appear temporarily, move around the screen or disappear behind a dialogue. There also needs to be a distinction between pausing, stopping, and cancelling. Pausing means keep everything and weight. Stopping means finish the current safe operation and return control to me. Cancelling means a band in the task, and because that can lose work, it should be deliberate. At the moment, interfaces sometimes blur those actions together, which makes intervention risky. The agent should also recognize messages arriving while it is working. If I send a correction, it should not sit ignored until the entire original task has finished. The agent should reach a safe stopping point, read the new instruction, and decide whether it changes the current work. For an AAC user, the message itself should be treated as the interruption mechanism. I should not have to race to press a separate button before typing it.

38:30This is ultimately about preserving agency. I want the AI to work independently, but independence should not mean that it becomes difficult to stop. The ideal system gives the agent enough freedom to complete substantial work while ensuring that I can regain control through one reliable, accessible action at any moment. Is there a better way to train AI tools regarding accessibility? I think a better approach is to train and evaluate AI in constrained environments where accessibility is not just described in a prompt, but enforced by the tools available. If you tell an agent to behave like a switch user but still give it a mouse, touch screen coordinates or unrestricted automation, it will usually take the shortest route. That proves it can complete the task, but it does not prove that the accessible route works. The agent needs an environment in which next, previous, and select are genuinely its only controls. That is what I tried to create with the switchify testing harness.

39:31The model sees the screen, receives enough context to understand the scanning system and chooses one switch action at a time. The surrounding software enforces those restrictions. It cannot bypass a difficult menu by tapping directly on the target. If it becomes stuck, that difficulty becomes part of the test result. This principle can be applied to other access methods. An agent testing keyboard accessibility should be restricted to keyboard input. A screen reader test agent should primarily receive the semantic information and announcements available to a screen reader user, rather than a perfect visual representation of the page. An iGaze simulation should account for target size, dwell time, and the difficulty of precise selection. The environment should represent the capabilities and limitations of the input method being tested. I would still be careful with the word training. An AI operating under switch constraints does not become a switch user, and it cannot reproduce fatigue in voluntary movement, communication effort, or somebody's lived experience.

40:34The aim is not to replace testing by disabled people. It is to expose basic failures and ambiguous interactions earlier, before asking a person to spend their time finding problems that automated testing could have caught. The results should also be interpreted properly. If an agent fails, that does not automatically mean the interface is inaccessible. The model may have misunderstood the screen, but that misunderstanding is useful evidence. We can inspect the screenshots and action history, ask why the state was unclear, improve either the interface or the agent's accessibility model, and then test again. So the better way to teach AI about accessibility is through experience shaped by real constraints. Documentation explains the rules, but the environment determines whether the agent must follow them. Accessibility should become part of the agent's test architecture, not a paragraph added to the end of its prompt. How did you create switchify without having it in the flash space?

41:34For example, how do you program or control your IDE, etc. and what's that like as a user? I had been using iOS for about 10 years, and for desktop access I was using Grid 3 right up until a few days ago when I released switchify PC. So I used the access tools I already had, mainly Grid 3 to control the computer and build the first versions. The answer is, slowly and with a lot of patience. I was using existing assistive technology to build a new assistive technology tool. In a way, that also showed why switchify needed to exist, because the process made the barriers very obvious. Are there any differences between iOS Android Windows you've had to handle? Yes. Each platform exposes accessibility and input APIs differently, but one of the biggest differences is that touch is fundamentally different from a mouse. On Android and iOS, you are often trying to simulate direct interaction with a touch screen.

42:36Taps, swipes, long presses, multi-touch gestures and screen positions. On Windows, control is built around a pointer, clicks, keyboard input, windows, focus, hover states, drag operations and desktop UI conventions. That changes the design of the access system. A touch gesture is usually a bounded action at a point or along a path. A mouse interaction can involve moving through empty space, hovering before clicking, dragging between windows, using the wheel, or relying on focus in a way that mobile apps often do not. iOS has strong built-in accessibility features and consistency, but third-party developers have less freedom to build deep access tools. Android gives more flexibility through accessibility services and gesture dispatch, but device differences, OEM behavior and the quality of app accessibility trees can be challenging. Windows gives a lot of power for desktop control, but the range of apps and UI technologies is much wider, so consistency becomes the hard problem.

43:39What can each OS learn from one another? Android can learn from iOS's consistency and polish and built-in accessibility. One of the strengths of iOS is that the system accessibility features feel integrated and predictable across a lot of apps. iOS can learn from Android's openness and willingness to let developers build deeper access tools. Android accessibility services give developers more room to experiment with things like scanning, gesture dispatch, and alternative input. And that openness is important because not every user fits the built-in model. Windows can learn from mobile platforms about simpler, more predictable accessibility experiences, but mobile platforms can also learn from windows about the value of automation APIs and keyboard access. Across all of them, the biggest lesson is that platform vendors should treat accessibility APIs as core infrastructure. If an update breaks focus, changes gesture behavior or makes UI information harder to inspect, that is not a small regression for assistive technology users.

44:48All of them can do better at involving disabled users earlier, documenting APIs clearly and giving developers stable ways to understand and control the interface. We're now onto the final part of our show where we're going to talk about how we can help others. So what can developers do to improve accessibility? Start by treating accessibility as a normal engineering requirement, not a polishing task. Use semantic UI components so the platform can tell assistive technology what something is and what it does. Keep a logical focus order. Make sure every control has a name, role, and state. Support keyboard and assistive technology navigation. Respect user settings like text size and contrast, and test with screen readers and switch control. Tab and enter are your best friends. If you cannot get through the main workflow with those keys, that is assigned the structure of the app needs work. Automated accessibility checks are useful, but they mostly catch obvious problems.

45:52They will not tell you whether the workflow is exhausting, whether an error is recoverable, or whether the app makes sense with an indirect access method. Most importantly, do not wait until the end of a project to ask whether it is accessible. What can vendors do? Vendors can provide better platform APIs, better documentation, and better testing tools. A good example is gesture emulation on Android. It took Android 7 versions to add support for accessibility services to perform gestures. And until the platform exposes an API like that, developers cannot build certain kinds of access tools properly. Platform vendors decide what is possible. If an API does not exist, or if it is unreliable, every assistive technology developer has to work around that limitation and users pay the price. Vendors should make accessibility features reliable across updates, provide stable ways to inspect UI structure and perform user actions, avoid breaking assistive technology, and involve disabled users and specialist developers before design decisions are locked in.

46:59Accessibility should be part of product quality, not a compliance checkbox. Where can developers go to learn more? The W3C Web Content Accessibility Guidelines are a good foundation for web work. Apple, Google, and Microsoft all have accessibility developer documentation. Developers should also learn from disabled creators, AAC users, screen reader users, and switch users. The best learning happens when you test real workflows with people who use these technologies every day. Right, your best start wrapping up the show. I really enjoy chatting with you about accessibility. But if there was one thing a software engineer should remember from our show, what would you like that to be you in? The one thing I would want software engineers to remember is that accessibility is not optional. It is not a nice extra, and it is not something you add only if there is time at the end. It is part of whether the software works. Accessibility is affected by the controls you choose, the APIs you expose, the assumptions you make about input, timing, focus, error recovery, and how a user completes a real task.

48:09If you build the product assuming everyone uses it like you do, you will exclude people without meaning to. But if you build with accessibility in mind from the start, you usually end up with software that is clearer, more robust, and easier for everyone to use. Was there anything we missed that you would like to mention? I would emphasize that accessibility is not a niche issue. Everyone's ability changes over time, and accessible software is usually better software for everyone. Also, disabled people should not only be users at the end of the process. We should be involved in design, engineering, testing, and decision making. What apps do you recommend people look at for a great accessibility experience? No app is perfect, and I think that is important to say. I would look at the built-in accessibility systems on iOS, Android, and Windows first, because they show how much can be done at the platform level, but they all have trade-offs.

49:09I would also look at specialist tools such as AAC apps, screen readers, switch access tools, and apps that support multiple input methods well. The best examples are not perfect apps. They are apps where you can see real thought about how different users complete real tasks without being forced into one interaction style. Excellent. So if people want to find out more, how can they get in touch or reach out? People can visit my website at oinmageer.com or learn more about Switchify at Switchifyapp.com. For Switchify specifically, the website has contact details, links to the source code, and ways to suggest features. Owen, thank you for coming on the show. It's been a real pleasure. This is Gavin Henry for Software Engineering Radio. Thank you for listening. Thanks again, Gavin. I really appreciate the chance to talk about this. If listeners take one thing away, I hope it is that accessibility is not only about passing a checklist. It is about giving people a fair chance to use software in the way that works for them.

50:24Thanks for listening to SC Radio, an educational program brought to you by IJerplee Software Magazine. For more about the podcast including other episodes, visit our website at sc-radio.net. To provide feedback, you can comment on each episode on the website or reach us on LinkedIn, Facebook, Twitter, or through our Slack channel at scradio.slack.com. You can also email us at teamat sc-radio.net. This and all other episodes of SC Radio is licensed under Creative Commons license 2.5. Thanks for listening. Thanks for listening to SC Radio.