Omni-IO Skills: Harnessing Your Agent Omni-Native

0:00Welcome to Daily Papercast. Today's paper has been selected from the Hugging Face Daily Paper List of September 30, 2026, and it has gained 57 upvotes. The title is Omni-Io Skills, Harnessing Your Agent Omni-Native. This paper is authored by Yenlin Lee and Ming-Yang Hao from the National University of Singapore, and the corresponding author is Hao Fei from the University of Oxford. Let's dive into the introduction. Real-world tasks often require operating across multiple modalities. Take, for example, creating an online course. It starts with lecture recordings and reference documents, moves through content analysis and visual design, and culminates in slides, illustrations, narration, and an explainer video. So, Ashley, how are these tasks interconnected? The resulting artifacts are deeply interconnected, sharing facts, style, timing, and production

1:01constraints. For an Omni system to be effective, it must be capable of processing and producing combinations of text, images, audio, video, documents, 3D assets, and code, ensuring semantic and asset continuity. Got it. So how did recent Omni models address these needs and what challenges remain? Some foundation models in the Omni space have made progress by expanding understanding and generation capabilities through unified auto-regressive modeling and hybrid discrete continuous designs. However, they face persistent scaling pressures. The more modalities a model supports, the more challenges arise in reconciling representations, objectives, and fidelity requirements. This often necessitates new data, codecs, decoders, and alignment stages. It sounds like there's room for improvement. That's the complementary approach proposed in this paper. The authors introduce Omni IO skills, a plug-and-play agent Harness designed to make existing agents

2:02Omni native. This system layer can Harness continuously evolving capabilities and integrate them into coherent, application-level workflows without altering the core reasoning model of the host agent. How does it work exactly? It operates through hierarchical skills, standardized multimodal execution interfaces, dependency-aware orchestration, and a persistent asset registry. Multi-asset workflows are represented as declare execution graphs, which schedule operations concurrently and register outputs for reuse across multiple tasks and turns. Interesting. What types of skills does Omni IO incorporate? The implementation includes 27 skills covering 38 representative tasks across seven artifact modalities and four capability families, understanding, generation, reasoning, and retrieval. What's the practical impact of using Omni IO skills on foundation models like GPT5.6,

3:03Seoul and Claude Sonnet 5? The Harness raises the input support rates of these models from 40% and 38.89% to 100%. Furthermore, it increases the relative semantic quality coupled score from 26.99 to 74.94 for GPT5.6 Seoul and from 27.82 to 77.78 for Claude Sonnet 5, while strict structure score reaches 100% and 99.78% respectively. That's impressive. In summary, how does Omni IO skills advance the field? Omni IO skills provides a practical route to developing broad, available Omni systems without having to change the reasoning core of the host agent, ensuring continuity across modalities and tasks. That concludes the introduction section. Moving on to the method section of Omni IO skills, harnessing your agent Omni Native.

4:05Let's delve into how the authors propose and implement this innovative framework. The primary aim of Omni IO skills is to create a plug and play harness that can be integrated with existing general purpose agents to enhance their multimodal capabilities without needing to alter their core reasoning models. And how do they achieve this goal? The harness operates through several key components, hierarchical skills, a standardized multimodal execution interface, dependency-aware orchestration, and a persistent asset registry. Essentially, these elements work together to organize, execute, and manage multimodal workflows efficiently and effectively. Let's break down these components one by one, starting with hierarchical skills. What are they? And how do they function? There are hierarchical skills are the backbone of Omni IO skills. They are categorized into three levels, atomic skills, expert skills, and scenario skills. Atomic skills perform a single operation.

5:05Expert skills target a concrete deliverable by coordinating multiple atomic operations. And scenario skills address an application-specific context by organizing several expert skills around a task objective. Sounds like a robust structure. Can you give an example of how these skills interact in a real-world application? Sure. For instance, suppose a user wants to generate educational materials. The scenario skill for education sharing would orchestrate various tasks like video understanding, document analysis, and image generation to either atomic or expert skills. The workflow ensures that all educational content is cohesive and interrelated. Right. Now, what about the multimodal execution interface? How does it contribute to achieving the system's goals? The multimodal execution interface standardizes how tasks are executed across different modalities. It provides a consistent way to invoke capabilities, whether it's understanding, generation, or utility

6:05operations. This layer essentially converts semantic task specifications from the skill entry layer into executable operations, ensuring seamless integration of various tools and models. These seem crucial when dealing with multiple modalities. But how does the system handle dependencies and task coordination? That's where the dependency-aware orchestration comes in. Each skill's tasks are represented as declare execution graphs or DEGs. These graphs map out task dependencies and schedule operations in concurrent execution waves. This makes sure that each task is executed in the correct order. And successful outputs are registered for reuse in downstream tasks. I see. What is the significance of the asset registry in this framework? The asset registry is pivotal because it maintains a persistent record of all generated assets. This allows for cross-turn reuse and ensures that intermediate and final outputs can be referenced independently of their physical paths.

7:07Essentially, it provides a stable reference and continuity across multiple tasks and sessions, which is crucial for long-term and complex workflows. Now, let's discuss the practical implementation of these methods. How did the authors evaluate OmniIO skills? They evaluated the harness on UNIM90, a controlled subset of tasks from UNIM that cover text, image, audio, video, document, code, and 3D modalities. They used GPT 5.6-soul and Claude Sonic 5 as host agents to test the effectiveness of OmniIO skills in enhancing multimodal capabilities. What metrics did they use for evaluation? And what were the results? They reported metrics like input to support rate, semantic quality coupled score, SQCS, interleaved coherent score, ICS, strict structure score, STS, and lenient structure score, La Else. The results were impressive. OmniIO skills raised the input support rate to 100% for both GPT 5.6-soul and Claude

8:14Sonic 5. Besides, SQCS increased substantially, indicating improved semantic correctness and generation quality. STS and La Else also showed marked improvements, demonstrating the effectiveness of the harness and structural consistency across tasks. Those are significant improvements. It seems that OmniIO skills could be a game changer for multimodal agent capabilities. Indeed, the authors also provided qualitative case studies to showcase real-world applications, highlighting how the harness can produce consistent and high-quality outputs across different modalities. I think that covers most of the key components and evaluation methods. Anything else to add? To emphasize, OmniIO skills makes an existing agent omninative without altering its reasoning core. This modular and extensible approach leverages hierarchical skills, standardized interfaces, dependency-aware orchestration, and persistent asset tracking to create cohesive multimodal

9:16workflows. That wraps up our discussion on the method section. Stay tuned for the next part where we delve into the experiments and results. Let's move on to the experiments and results section of OmniIO skills harnessing your agent omninative. Ashley, can you walk us through the experimental setup? Certainly. The authors evaluated the effectiveness of OmniIO skills using two general-purpose agents, GPT 5.6-SOL and Claude Sonnet 5. They employed UniM90, a controlled subset of tasks from UniM that encompasses text, image, audio, video, document, code, and 3D modalities. What specific configurations were used for these agents during the evaluations? The experimental setup involved testing the agents under two configurations, the base agent configuration, which uses the agent's native capabilities without any additional enhancements,

10:16and the agent plus OmniIO skills configuration, which integrates the OmniIO skills into the agent's environment. And how did the authors ensure the controlled setup across all instances? To ensure a controlled setup, they maintained the same tool configurations, system prompts, and execution budgets across both configurations. The only variable was the inclusion of OmniIO skills. Make sense. Now what metrics did the authors use to evaluate the performance? They focused on several key metrics, input support rate, semantic quality coupled score or SQCS, interleaved coherent score or ICS, strict structure score or STS, and lenient structure score or LES. What does the input support rate measure? The input support rate, denoted by TOW, measures the proportion of instances where an agent can fully receive and process all input modalities.

11:17And what about SQCS, ICS, STEDS, and LES? SQCS evaluates semantic correctness and generation quality. ICS assesses the overall coherence of interleaved multimodal responses. STS measures the strict consistency of output structures, while LES evaluates modality level coverage. Those are comprehensive metrics. Can you tell us about the results? Yes, the results showed significant improvements when OmniIO skills were integrated. For GPT 5.6 Sol, the input support rate increased from 40% to 100%, and for Claude Sonnet 5, it rose from 38.89% to 100%. That's quite a jump. What changes were observed in the semantic quality coupled score? For GPT 5.6 Sol, the SQCS increased from 26.99 to 74.94, and for Claude Sonnet 5, it

12:19increased from 27.82 to 77.78. These substantial gains indicate improved semantic correctness and quality of generated outputs. Impressive. How did OmniIO skills affect the interleaved coherence score? The ICS for GPT 5.6 Sol increased from 34.61 to 93.98, while for Claude Sonnet 5, it rose from 32.00 to 83.28. Those are huge improvements in coherence. What about the structure scores? Both agents showed marked improvements in structure metrics as well. GPT 5.6 Sol achieved a strict structure score of 100.00 and a lenient structure score of 100.00. Claude Sonnet 5 reached 99.78 on the strict structure score and 100.00 on the lenient structure score. It seems the OmniIO skills really enhanced the multimodal capabilities of these agents.

13:22Any qualitative case studies in the paper? Yes, the authors provided two qualitative case studies, one involved generating an art tutorial and the other focused on product promotion. In these case studies, OmniIO skills orchestrated various tasks like video understanding, document analysis, and image generation to produce consistent and high quality outputs across different modalities. Can you give us a brief overview of one of these case studies? Sure. In the art tutorial generation case, OmniIO skills invoked several skills. The education sharing scenario skill to plan the tutorial, the video understanding atomic skill to extract drawing steps, the audio understanding atomic skill to recover accompanying explanations, and the image generation atomic skill to produce a series of tutorial images. The outcome was a cohesive and detailed art tutorial spanning multiple modalities. That's a great example of how OmniIO skills can manage complex workflows seamlessly.

14:25Anything else to note from the experiment and result section? Just a highlight that the harness level composition offered by OmniIO skills bridges substantial gaps across different host agents, delivering high quality and structurally consistent outputs. It demonstrates a modular and extensible approach to evolving multimodal capabilities in AI systems. That concludes the experiment section. Next up, we'll discuss the related work to understand the foundation and context for this research. Let's dive into the related work section to see how OmniIO skills builds on and differentiates from existing research. This paper places OmniIO skills within the broader context of advancements in multimodal understanding and generation models, agent harnesses, and skills frameworks. Starting with multimodal understanding and generation models, what foundational work does this paper reference? Early research and multimodal understanding and generation have largely separate pathways.

15:28Understanding models mapped images, audio and video into semantic representations suitable for language-based inference while generative models focused on recovering visual, acoustic or temporal details. However, recent Omni Foundation models have unified perception, reasoning, and generation within a shared context. What specific architectural approaches have been employed to achieve this unification? Two primary architectural roots have been explored. Unified auto-regressive models discretize diverse modalities, applying next token prediction over a common sequence, thus providing a uniform interface for mixed and interleaved content. Hybrid discrete continuous models, on the other hand, combine auto-regressive semantic reasoning with continuous representations, diffusion or flow objectives, and modality-specific decoders to retain media fidelity. Interesting. What are the trade-offs between these approaches? Unified auto-regressive models simplify the architecture and naturally support interleaving,

16:29but face challenges related to modality tokenization, sequence length, and perceptual detail loss. Hybrid designs maintain greater specialization but require managing multiple encoders, adapters, objectives, and decoders, forming a complex design spectrum. How does Omni-IO skills fit into this landscape? Omni-IO skills complements these foundation models by treating them, along with specialist models and media engines, as replaceable execution backends. It shifts the unification to the level of task execution and artifact flow, allowing applications to evolve independently of the underlying model stack. Next, let's look at Agent Harnises. What role do they play in this framework? Agent Harnises provide the operational substrate for sustained execution. They include mechanisms like action loops, tool access, context and state management, execution control, verification, and recovery. React, for instance, defines an interleaved reasoning, action observation loop for models

17:31to revise their plans based on environmental feedback, while the model context protocol standardizes external tool and resource exposure to models. And how do these concepts extend to multimodal agents? Multimodal agents use similar control loops to select and coordinate modality specialists. Systems like MM React connect language models with vision experts, and recent Omni-Agence extend this coordination to images, audio, and video through master agent delegation or active perception. What additional challenges do production workflows pose? Entrepreneur workflows require intermediate asset transfer, provenance, and cross-turn revision. Omni-IoSkills expands upon existing harnesses by connecting a general-purpose agent to multimodal backends, coordinating dependency-aware execution, and maintaining persistent artifact states for reuse. Let's discuss skills frameworks. How do they contribute to this approach? Skills frameworks add a procedural knowledge layer, recording task applicability, required

18:35inputs, execution modes, and expected outputs. Skills can preserve workflows, domain conventions, executable code, and composition patterns. For instance, SkillsBench benchmarks the effect of curated skills across diverse expert tasks, while SUAUA skill represents computer use procedures as parameterized execution and composition graphs. And what about their application to multimodal agents? The intersection of Skills Frameworks and Omni-Agent Research is underdeveloped for workflows requiring any to any understanding in generation with persistent artifact states. Omni-IoSkills bridges this gap by organizing multimodal procedures through hierarchical skills and enabling composable, traceable workflows with cross-turn asset reuse. To recap, Omni-IoSkills builds upon several branches of existing research. Total understanding and generation models, agent harnesses, and Skills Frameworks, to offer a comprehensive approach for enhancing multimodal agent capabilities.

19:39Exactly. By integrating these advancements, Omni-IoSkills provides a scalable, flexible, and extensible system that improves continuity and coherence across tasks without altering the core reasoning models of the host agents. That concludes the related work section. We've reached the final part of today's discussion on Omni-IoSkills, harnessing your agent Omni-Native. Ashley, can you summarize the key contributions and takeaways from this paper? Of course, Evan. This paper introduces a novel approach called Omni-IoSkills designed to extend the capability of existing general-purpose agents by making them Omni-Native. The main contributions include the development of hierarchical skills that coordinate multimodal workflows. A standardized multimodal execution interface, dependency-aware orchestration through declare execution graphs, and a persistent asset registry for artifact management and reuse. And in terms of practical impact, how does this approach enhance the performance of agents

20:41like GPT5.6-SOL and Claude Sonnet5? The practical impact is significant. The harness raises the input support rates of these models to 100%, dramatically improves the semantic quality coupled score, ensures interleaved coherence, and achieves high scores in output structure metrics. This shows that Omni-IoSkills effectively bridges gaps across modalities, ensuring that generated content is both semantically accurate and structurally sound. It sounds like Omni-IoSkills offers a robust and scalable solution for enhancing multimodal capabilities without requiring changes to the core reasoning models of host agents. Precisely, by focusing on a modular and extensible framework, Omni-IoSkills allows existing agents to leverage a broad spectrum of evolving capabilities, accommodating a wide range of application contexts, and ensuring continuity across tasks. That's a wrap on today's episode.

21:41Thank you, Ashley, for walking us through this fascinating paper. Thank you, Evan, and thanks to our listeners for joining us today. We hope you found today's discussion insightful. Make sure to join us for future episodes where we'll continue to explore groundbreaking research from the field of AI and multimodal systems. Remember, you can find the paper we discussed today and many others on the Hugging Face daily paper list. Until next time, stay curious and keep exploring. Goodbye!