Skip to main content

Setting up the code

In this walkthrough, we’ll be leveraging the Google ADK integration. Let’s create a virtual env first

Note: Use python3.12 to create virtual environments
This creates a virtual environment with name env. Activate it using the following command in your terminal
Once your virtual environment is active, you can run the following command to install all the necessary requirements for this walkthrough
Now, create a python script (say google_adk_futureagi.py) at your desired location and start by setting up the environment variables and imports
Initialize your trace provider and instrument Google ADK
Create your multi-agent system. First, let’s define the planner agent:
Define the researcher agent:
Define the critic agent:
Define the writer agent:
Create the root orchestrator agent:
Create the main execution function:
Create the main function with sample prompts:
Run your script:
Upon successful execution of the script, we see that a new project with the name of google-adk-new has been added in the Observe tab of the platform. Agent compass concepts When you click on the first project, you get directed to the LLM Tracing view where all the traces of your observe project are listed. Agent compass concepts Upon clicking of a trace, a drawer opens up that shows the trace tree and the details of the span selected. On top of them, the insights generated from Agent Compass are also shown in a collapsible accordion. You can toggle to see the expanded view of the same Agent compass concepts Inside the accordion are other headings each with their separate meaning. You will see these terms being used frequently. They should be interpreted as follows

Scores

Each of the metrics mentioned are the grounds on which the agent performance is evaluated out of a score of 5. They are as follows Agent compass concepts

Clickable metrics

These are the taxonomy metrics. They indicate under which metric your agent needs improvement and are decided by the compass itself (ex: Instruction Adherence, Incomplete task etc.)

Recommendation

This is a suggestion from the perspective of implementing a long term and robust fix. The recommendation may not always be the same as an immediate fix. In most of the cases, proceeding with the recommendation would be the best course of action

Immediate fix

This suggests a minimal functional fix. This fix may or may not necessarily align with the recommendation

Insights

Insights are high level overview of the complete trace execution. They do not change with the currently active taxonomy metric and give a bird’s eye view of what your agent did during execution

Description

The description conveys what went wrong during the agentic exection. It also answers what happened in the error

Evidence

Evidences are the supporting snippets from the LLM response that was generated during the agentic executions. They can help you uncover edge cases/unforeseen scenarios that might’ve been missed during the development phase

Root Causes

Indicates the underlying issue of an error occurence. This helps developers gain a better understanding of their agentic workflows

Spans

The list of affected spans. Each taxonomy metric can have different spans associated with it. You can click on the span to spot it in the trace tree

Sampling Rate

This is a special, user controlled parameter. It refers to what percentage of traces should the compass run on. Based on the sampling rate, the compass picks up traces at random to generate insights. Sampling rate can be configured in two simple steps mentioned below

Note: The adjusted/updated sampling rate will be applicable for upcoming traces only and not on the currently present or previously added traces
  • Step 1: Click on configure button on the top right corner of the observe screen Agent compass concepts
  • Step 2: Use the slider to adjust the sampling rate according to your needs. Click on update to save Agent compass concepts

Feed Tab

All the errors identified by the compass are grouped together and can be viewed under the Feed tab of the platform. The Feed tab shows all the errors identified by the compass in one place. The screen of the same looks like this Agent compass concepts Following terms are helpful in getting a better understanding of the feature

Cluster

Mulitple traces can have the same error. All those traces are grouped under a common cluster. The Error Name shown in the image above is essentailly the name of the cluster. The listing page of the tab provides options to filter the clusters based on project and age of the lastest error.

Events

This term is used to indicate the number of occurances of the particular error The number of times a particular error occured. The cycle of that is referred as trend (example: increasing, decreasing etc.) Clicking on each of the cluster takes us to a details page which gives more information about the error and the associated trace(s) with it. By default, the latest trace associated with the error cluster will be shown. There are also other features that will be explained one by one. Agent compass concepts Toggling between traces & filtering: The upper section of the page gives the options of toggling between traces, along with the information of when were the first and last occurences of the error. You can also able filter the data as per the time range of your liking. The graph displays the trends of the error Agent compass concepts Insights and Trace tree details: The next section shows the trace tree of the selected trace (latest affected trace by default). Along with it are the insights that were generated by the agent compass. On the right hand side, what we can see are the span attributes. Along with the metadata of the currently active span Agent compass concepts