Over the last ten years, observability providers have been engaged in a battle over user interfaces. Fighting over the appearance and usability of their tools. AsdataThe process of data collection became standardized with initiatives like OpenTelemetry, leading to the main areas of competition shifting higher in the system architecture, specifically to the user interface (UI).
As vendors focused on visualization, dashboards, and workflows, a more significant movement began to take shape—unifying logs, traces, and metrics into one cohesive, exploratory interface.
This indicated that teams could observe all activities in one view, making it simpler to grasp what was occurring across their systems in real time. Observability, in many respects, turned into a user interface challenge, as the focus was on how efficiently humans could navigate and analyze intricate data to derive meaningful insights.
That approach benefited the industry greatly. However, it is now facing scrutiny due to increased complexity in current systems.
The consumer is changing
As intelligent AI systems become more prevalent throughout organizations, the main user of monitoring data is changing from a human operator to an automated system.
When this occurs, the benefit of refined processes featuring unified signals decreases, and the focus shifts lower in the system hierarchy.
The focus has shifted from how effectively an individual can handle telemetry data and identify the root cause. Instead, the key issue is whether the fundamental system possesses the appropriate data, the correct retention policies, and the necessary characteristics that allow machines to analyze and reason about it.
This shift has not completely arrived yet, but the path is clear. AI systems are already able to detect patterns and relationships within vast amounts of data, even though they continue to face challenges with genuine cause-and-effect analysis.
That divide is being actively addressed. Each majorcloud providerAnd the AI lab is focusing on agent abilities that extend beyond chat interfaces into self-directed decision-making. The more urgent question is whether current observability platforms are built for this upcoming era.
The compromises that were once reasonable are no longer valid.
Modern observability solutions were developed based on principles that were relevant when humans were the sole operators involved. In that setting, systems were created with the way engineers manually explore and resolve problems in mind.
As a result, data retention periods are brief, often lasting only a few days, since engineers typically do not require access to data from further back.
In the same way, sampling and rollups are bold approaches since an experienced professional can fill in the missing parts using their knowledge and judgment. Moreover, pricing models also reflect this situation, as they are designed for human-led, less frequent queries instead of ongoing analysis.
Each of these compromises was logical for people. They turn into disadvantages as soon as machines are relied upon to handle the analytical tasks.
Limited data retention periods hinder AI agents from identifying patterns, seasonal variations, and connections between events. An AI system that has access only to the past 72 hours of information, for instance, may not recognize that a specific surge in traffic occurs regularly according to a predictable schedule linked to seasonal changes.
Without a long-term perspective, AI systems remain trapped in the same reactive cycle that observability was designed to help companies break free from.
Aggressive data sampling introduces another challenge. Rollups and pre-aggregation eliminate the specific details that machines require for precise analysis.
A person examining a latency chart can decide if the underlying distribution is significant. An AI agent cannot take that shortcut. It requires high-quality data, as the signals it relies on are exactly those that sampling eliminates.
Next, there's the economic issue. Platforms that charge per query, limit simultaneous usage, or restrict access to specific human users are inherently incompatible with how AI agents function. AI agents don't perform a single query and analyze a graph. Instead, they conduct ongoing, parallel analysis across various dimensions at the same time.
A pricing structure that discourages extensive machine usage may result in unmanageable expenses or compel teams to limit the features they aim to provide.
These patterns also already shape the way foundational data is handledinfrastructure evolves. DatabaseObservability is becoming a primary focus in newly developed management systems.
Rather than isolating logs, metrics, and traces into separate systems or relying on extensive data sampling, they are built to manage and analyze full telemetry data sets at scale within one database layer.
This allows for the preservation and examination of all data, instead of relying on limited perspectives of system activity.
Making preparations for what is undoubtedly on the way
The positive aspect is that organizations don't have to wait for complete autonomous observability to begin getting ready. The necessary elements are already apparent and correspond to choices that leaders can make right now.
The importance of retention has increased compared to the past. If a platform only stores high-resolution data for a few days, it sets a limit on the potential of future AI agents before they are even implemented.
High-fidelity data is not an indulgence. The move towards sampling made sense when storage and processing power were the main constraints and humans were the sole users. As the cost of storing and analyzing raw telemetry keeps decreasing, retaining the original data becomes the more justifiable option.
Economics should match machine access behaviors. This involves looking beyond the listed cost of an observability solution, but rather how it bills for the high-concurrency, ongoing tasks that AI agents create.
Organizations that succeed in this area will be able to implement AI agents with assurance. Those that fail to do so will hinder their own progress.
We have highlighted the top AI tool.
This piece was created as part ofPro Perspectives, our channel showcases the top innovators and leaders in the technology sector today.
The opinions shared here belong to the author and may not reflect the views of Pro or Future plc. If you would like to contribute, click here for more information:https://www./pro/perspectives-how-to-submit
Liked this article? To discover more stories like this, follow us on MSN by clicking the +Follow button located at the top of this page.
No comments:
Post a Comment