University of Kwazulu-Natal

Establishing Trust In Ai Agents Ii Observability In Llm Agent

Bonisiwe Shabane
-
establishing trust in ai agents ii observability in llm agent

AI Agents are becoming the next big leap in artificial intelligence in 2025. From autonomous workflows to intelligent decision making, AI Agents will power numerous applications across industries. However, with this evolution comes the critical need for AI agent observability, especially when scaling these agents to meet enterprise needs. Without proper monitoring, tracing, and logging mechanisms, diagnosing issues, improving efficiency, and ensuring reliability in AI agent-driven applications will be challenging. An AI agent is an application that uses a combination of LLM capabilities, tools to connect to the external world, and high-level reasoning to achieve a desired end goal or state; Alternatively, agents can... Image credit: Google AI Agent Whitepaper.

For more information about AI agents, see: Typically, telemetry from applications is used to monitor and troubleshoot them. In the case of an AI agent, given its non-deterministic nature, telemetry is also used as a feedback loop to continuously learn from and improve the quality of the agent by using it as... Large language model (LLM) agents have demonstrated remarkable capabilities across various domains, gaining extensive attention from academia and industry. However, these agents raise significant concerns on AI safety due to their autonomous and non-deterministic behavior, as well as continuous evolving nature . From a DevOps perspective, enabling observability in agents is necessary to ensuring AI safety, as stakeholders can gain insights into the agents’ inner workings, allowing them to proactively understand the agents, detect anomalies, and...

Therefore, in this paper, we present a comprehensive taxonomy of AgentOps, identifying the artifacts and associated data that should be traced throughout the entire lifecycle of agents to achieve effective observability. The taxonomy is developed based on a systematic mapping study of existing AgentOps tools. Our taxonomy serves as a reference template for developers to design and implement AgentOps infrastructure that supports monitoring, logging, and analytics. thereby ensuring AI safety. An large language model (LLM) is a large-scale language model with tens of billions of parameters, pretrained on vast and diverse datasets, and applicable to various downstream tasks [1]. While LLMs demonstrate impressive capabilities, they also exhibit limitations in understanding and performing complex tasks.

This has led to an increasing demand for LLM agents (including agentic systems which are designed to prompt an LLM multiple times using agent-like design patterns and exhibit varying degrees of agent-like behavior), which... An LLM agent is an autonomous system powered by LLMs, capable of perceiving context, reasoning, planning, executing workflows, by leveraging external tools, knowledge bases and other agents to achieve human goals [3]. LLM agents have shown remarkable potential to enhance productivity across various domains, attracting widespread attention from academia and industry. For example, many agents are being successfully applied in the software engineering domain, such as Devin 111Devin, https://www.cognition.ai/blog/introducing-devin, ChatDev 222ChatDev. https://github.com/OpenBMB/ChatDev, SWE-agent 333SWE-agent, https://github.com/princeton-nlp/SWE-agent. Hereafter, we use ”agents” to refer specifically to LLM agents throughout this paper.

Despite the huge potential to enhance productivity, the adoption of LLM agents introduces unique challenges due to their inherent characteristics. Complex Artefacts and Pipelines: The agents are compound AI systems, integrating LLMs with various components (i.e., design time artifacts), such as context engine and external tools, and dynamically generating runtime artifacts, such as goals... The operational pipelines typically include context processing, reasoning and planning, workflow execution, and continuous evolution based on the feedback. Throughout these processes, the pipelines may leverage external tools, knowledge bases, and other agents to achieve human goals. Wenyue Hua, Xianjun Yang, Mingyu Jin, Zelong Li, Wei Cheng, Ruixiang Tang, Yongfeng Zhang [TrustAgent: Towards Safe and Trustworthy LLM-based Agents](https://aclanthology.org/2024.findings-emnlp.585/) (Hua et al., Findings 2024)

ACL materials are Copyright © 1963–2026 ACL; other materials are copyrighted by their respective copyright holders. Materials prior to 2016 here are licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 International License. Permission is granted to make copies for the purposes of teaching and research. Materials published in or after 2016 are licensed on a Creative Commons Attribution 4.0 International License. The ACL Anthology is managed and built by the ACL Anthology team of volunteers. Site last built on 26 January 2026 at 17:05 UTC with commit 113e74b.

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs. Key tips and resources for effective monitoring of LLM applications.

Observability becomes increasingly essential for maintaining quality, reliability, and performance. This guide provides a comprehensive approach to implementing effective LLM observability. Gal from Trace Loop walks through essential strategies and tools for monitoring LLM applications, with practical insights for teams at every stage of development. As applications grow in complexity, especially with the rise of autonomous agents, traditional monitoring methods prove insufficient. Effective LLM observability is built on Tracing, Metrics Definition, Quality Evaluation, and Actionable Insights. Use the interactive diagram below to explore each component and how they work together to create a comprehensive observability strategy.

People Also Search

AI Agents Are Becoming The Next Big Leap In Artificial

AI Agents are becoming the next big leap in artificial intelligence in 2025. From autonomous workflows to intelligent decision making, AI Agents will power numerous applications across industries. However, with this evolution comes the critical need for AI agent observability, especially when scaling these agents to meet enterprise needs. Without proper monitoring, tracing, and logging mechanisms,...

For More Information About AI Agents, See: Typically, Telemetry From

For more information about AI agents, see: Typically, telemetry from applications is used to monitor and troubleshoot them. In the case of an AI agent, given its non-deterministic nature, telemetry is also used as a feedback loop to continuously learn from and improve the quality of the agent by using it as... Large language model (LLM) agents have demonstrated remarkable capabilities across vario...

Therefore, In This Paper, We Present A Comprehensive Taxonomy Of

Therefore, in this paper, we present a comprehensive taxonomy of AgentOps, identifying the artifacts and associated data that should be traced throughout the entire lifecycle of agents to achieve effective observability. The taxonomy is developed based on a systematic mapping study of existing AgentOps tools. Our taxonomy serves as a reference template for developers to design and implement AgentO...

This Has Led To An Increasing Demand For LLM Agents

This has led to an increasing demand for LLM agents (including agentic systems which are designed to prompt an LLM multiple times using agent-like design patterns and exhibit varying degrees of agent-like behavior), which... An LLM agent is an autonomous system powered by LLMs, capable of perceiving context, reasoning, planning, executing workflows, by leveraging external tools, knowledge bases an...

Despite The Huge Potential To Enhance Productivity, The Adoption Of

Despite the huge potential to enhance productivity, the adoption of LLM agents introduces unique challenges due to their inherent characteristics. Complex Artefacts and Pipelines: The agents are compound AI systems, integrating LLMs with various components (i.e., design time artifacts), such as context engine and external tools, and dynamically generating runtime artifacts, such as goals... The op...