Calls
Calls are the fundamental building block in Weave. They represent a single execution of a function, including:
- Inputs (arguments)
- Outputs (return value)
- Metadata (duration, exceptions, LLM usage, etc.)
Calls are similar to spans in the OpenTelemetry data model. A Call can:
- Belong to a Trace (a collection of calls in the same execution context)
- Have parent and child Calls, forming a tree structure
Creating Calls
There are three main ways to create Calls in Weave:
1. Automatic Tracking of LLM Libraries
Weave automatically tracks calls to common LLM libraries like openai
, anthropic
, cohere
, and mistral
. Simply call weave.init('project_name')
at the start of your program:
import weave
from openai import OpenAI
client = OpenAI()
# Initialize Weave Tracing
weave.init('intro-example')
response = client.chat.completions.create(
model="gpt-4",
messages=[
{
"role": "user",
"content": "How are you?"
}
],
temperature=0.8,
max_tokens=64,
top_p=1
)
2. Decorating Functions
However, often LLM applications have additional logic (such as pre/post processing, prompts, etc.) that you want to track.
Weave allows you to manually track these calls using the @weave.op
decorator. For example:
import weave
# Initialize Weave Tracing
weave.init('intro-example')
# Decorate your function
@weave.op
def my_function(name: str):
return f"Hello, {name}!"
# Call your function -- Weave will automatically track inputs and outputs
print(my_function.call("World"))
If your op is a method on a class, you need to pass the instance as the first argument to the op (see example below).
import weave
# Initialize Weave Tracing
weave.init("intro-example")
class MyClass:
# Decorate your method
@weave.op
def my_method(self, name: str):
return f"Hello, {name}!"
instance = MyClass()
# Call your method -- Weave will automatically track inputs and outputs
instance.my_method.call(instance, "World")
Call Display Name
Sometimes you may want to override the display name of a call. You can achieve this in one of three ways:
- Change the display name on a per-call basis. This uses the
Op.call
method to return aCall
object, which you can then use to set the display name usingCall.set_display_name
.
result, call = my_function.call("World")
call.set_display_name("My Custom Display Name")
- Change the display name for all Calls of a given Op:
@weave.op(call_display_name="My Custom Display Name")
def my_function(name: str):
return f"Hello, {name}!"
-
The
call_display_name
can also be a function that takes in aCall
object and returns a string. TheCall
object will be passed automatically when the function is called, so you can use it to dynamically generate names based on the function's name, call inputs, attributes, etc.-
One common use case is just appending a timestamp to the function's name.
from datetime import datetime
@weave.op(call_display_name=lambda call: f"{call.func_name}__{datetime.now()}")
def func():
return ... -
You can also get creative with custom attributes
def custom_attribute_name(call):
model = call.attributes["model"]
revision = call.attributes["revision"]
now = call.attributes["date"]
return f"{model}__{revision}__{now}"
@weave.op(call_display_name=custom_attribute_name)
def func():
return ...
with weave.attributes(
{
"model": "finetuned-llama-3.1-8b",
"revision": "v0.1.2",
"date": "2024-08-01",
}
):
func() # the display name will be "finetuned-llama-3.1-8b__v0.1.2__2024-08-01"
with weave.attributes(
{
"model": "finetuned-gpt-4o",
"revision": "v0.1.3",
"date": "2024-08-02",
}
):
func() # the display name will be "finetuned-gpt-4o__v0.1.3__2024-08-02"
-
Technical Note: "Calls" are produced by "Ops". An Op is a function or method that is decorated with @weave.op
.
By default, the Op's name is the function name, and the associated calls will have the same display name. The above example shows how to override the display name for all Calls of a given Op. Sometimes, users wish to override the name of the Op itself. This can be achieved in one of two ways:
- Set the
name
property of the Op before any calls are logged
my_function.name = "My Custom Op Name"
- Set the
name
option on the op decorator
@weave.op(name="My Custom Op Name)
Attributes
When calling tracked functions, you can add additional metadata to the call by using the weave.attributes
context manager. In the example below, we add an env
attribute to the call specified as 'production'
.
# ... continued from above ...
# Add additional "attributes" to the call
with weave.attributes({'env': 'production'}):
print(my_function.call("World"))
3. Manual Call Tracking
You can also manually create Calls using the API directly.
- Python
- HTTP API
import weave
# Initialize Weave Tracing
weave.init('intro-example')
def my_function(name: str):
# Start a call
call = weave.create_call(op="my_function", inputs={"name": name})
# ... your function code ...
# End a call
weave.finish_call(call, output="Hello, World!")
# Call your function
print(my_function("World"))
- Start a call: POST
/call/start
- End a call: POST
/call/end
curl -L 'https://trace.wandb.ai/call/start' \
-H 'Content-Type: application/json' \
-H 'Accept: application/json' \
-d '{
"start": {
"project_id": "string",
"id": "string",
"op_name": "string",
"display_name": "string",
"trace_id": "string",
"parent_id": "string",
"started_at": "2024-09-08T20:07:34.849Z",
"attributes": {},
"inputs": {},
"wb_run_id": "string"
}
}
Viewing Calls
- Web App
- Python
- HTTP API
To view a call in the web app:
- Navigate to your project's "Traces" tab
- Find the call you want to view in the list
- Click on the call to open its details page
The details page will show the call's inputs, outputs, runtime, and any additional attributes or metadata.
To view a call using the Python API, you can use the get_call
method:
import weave
# Initialize the client
client = weave.init("your-project-name")
# Get a specific call by its ID
call = client.get_call("call-uuid-here")
print(call)
To view a call using the Service API, you can make a request to the /call/read
endpoint.
curl -L 'https://trace.wandb.ai/call/read' \
-H 'Content-Type: application/json' \
-H 'Accept: application/json' \
-d '{
"project_id": "string",
"id": "string",
}'
Updating Calls
Calls are mostly immutable once created, however, there are a few mutations which are supported:
All of these mutations can be performed from the UI by navigating to the call detail page:
Set Display Name
- Python
- HTTP API
In order to set the display name of a call, you can use the Call.set_display_name
method.
import weave
# Initialize the client
client = weave.init("your-project-name")
# Get a specific call by its ID
call = client.get_call("call-uuid-here")
# Set the display name of the call
call.set_display_name("My Custom Display Name")
To set the display name of a call using the Service API, you can make a request to the /call/update
endpoint.
curl -L 'https://trace.wandb.ai/call/update' \
-H 'Content-Type: application/json' \
-H 'Accept: application/json' \
-d '{
"project_id": "string",
"call_id": "string",
"display_name": "string",
}'
Add Feedback
Please see the Feedback Documentation for more details.
Delete a Call
- Python
- HTTP API
To delete a call using the Python API, you can use the Call.delete
method.
import weave
# Initialize the client
client = weave.init("your-project-name")
# Get a specific call by its ID
call = client.get_call("call-uuid-here")
# Delete the call
call.delete()
To delete a call using the Service API, you can make a request to the /calls/delete
endpoint.
curl -L 'https://trace.wandb.ai/calls/delete' \
-H 'Content-Type: application/json' \
-H 'Accept: application/json' \
-d '{
"project_id": "string",
"call_ids": [
"string"
],
}'
Querying & Exporting Calls
The /calls
page of your project ("Traces" tab) contains a table view of all the Calls in your project. From there, you can:
- Sort
- Filter
- Export
The Export Modal (shown above) allows you to export your data in a number of formats, as well as shows the Python & CURL equivalents for the selected calls! The easiest way to get started is to construct a view in the UI, then learn more about the export API via the generated code snippets.
- Python
- HTTP API
To fetch calls using the Python API, you can use the client.calls
method:
import weave
# Initialize the client
client = weave.init("your-project-name")
# Fetch calls
calls = client.calls(filter=...)
Currently, it is easier to use the lower-level calls_query_stream
API as it is more flexible and powerful.
In the near future, we will move all functionality to the above client API.
import weave
# Initialize the client
client = weave.init("your-project-name")
calls = client.server.calls_query_stream({
"project_id": "",
"filter": {},
"query": {},
"sort_by": [],
})
The most powerful query layer is at the Service API. To fetch calls using the Service API, you can make a request to the /calls/stream_query
endpoint.
curl -L 'https://trace.wandb.ai/calls/stream_query' \
-H 'Content-Type: application/json' \
-H 'Accept: application/json' \
-d '{
"project_id": "string",
"filter": {
"op_names": [
"string"
],
"input_refs": [
"string"
],
"output_refs": [
"string"
],
"parent_ids": [
"string"
],
"trace_ids": [
"string"
],
"call_ids": [
"string"
],
"trace_roots_only": true,
"wb_user_ids": [
"string"
],
"wb_run_ids": [
"string"
]
},
"limit": 100,
"offset": 0,
"sort_by": [
{
"field": "string",
"direction": "asc"
}
],
"query": {
"$expr": {}
},
"include_costs": true,
"include_feedback": true,
"columns": [
"string"
],
"expand_columns": [
"string"
]
}'
Call FAQ
Call Schema
Please see the schema for a complete list of fields.
Property | Type | Description |
---|---|---|
id | string (uuid) | Unique identifier for the call |
project_id | string (optional) | Associated project identifier |
op_name | string | Name of the operation (can be a reference) |
display_name | string (optional) | User-friendly name for the call |
trace_id | string (uuid) | Identifier for the trace this call belongs to |
parent_id | string (uuid) | Identifier of the parent call |
started_at | datetime | Timestamp when the call started |
attributes | Dict[str, Any] | User-defined metadata about the call |
inputs | Dict[str, Any] | Input parameters for the call |
ended_at | datetime (optional) | Timestamp when the call ended |
exception | string (optional) | Error message if the call failed |
output | Any (optional) | Result of the call |
summary | Optional[SummaryMap] | Post-execution summary information |
wb_user_id | Optional[str] | Associated Weights & Biases user ID |
wb_run_id | Optional[str] | Associated Weights & Biases run ID |
deleted_at | datetime (optional) | Timestamp of call deletion, if applicable |
The table above outlines the key properties of a Call in Weave. Each property plays a crucial role in tracking and managing function calls:
- The
id
,trace_id
, andparent_id
fields help in organizing and relating calls within the system. - Timing information (
started_at
,ended_at
) allows for performance analysis. - The
attributes
andinputs
fields provide context for the call, whileoutput
andsummary
capture the results. - Integration with Weights & Biases is facilitated through
wb_user_id
andwb_run_id
.
This comprehensive set of properties enables detailed tracking and analysis of function calls throughout your project.
Calculated Fields:
- Cost
- Duration
- Status