Skip to main content

Costs

info

Custom costs are accessible via Python and REST queries. UI uptake is under development and expected to be complete by middle of October 2024

Adding a custom cost

You can add a custom cost by using the add_cost method. The three required fields are llm_id, prompt_token_cost, and completion_token_cost. llm_id is the name of the LLM (e.g. gpt-4o). prompt_token_cost and completion_token_cost are cost per token for the LLM (if the LLM prices were specified inper million tokens, make sure to convert the value). You can also set effective_date to a datetime, to make the cost effective at a specific date, this defaults to the current date.

import weave
from datetime import datetime

client = weave.init("my_custom_cost_model")

client.add_cost(
llm_id="your_model_name",
prompt_token_cost=0.01,
completion_token_cost=0.02
)

client.add_costs({
llm_id="your_model_name",
prompt_token_cost=10,
completion_token_cost=20,
# If for example I want to raise the price of the model after a certain date
effective_date=datetime(2025, 4, 22),
)

Querying for costs

You can query for costs by using the query_costs method. There are a few ways to query for costs, you can pass in a singular cost id, or a list of LLM model names.

import weave

client = weave.init("my_custom_cost_model")

costs = client.query_costs(llm_ids=["your_model_name"])

cost = client.query_costs(costs[0].id)

Purging a custom cost

You can purge a custom cost by using the purge_costs method. You pass in a list of cost ids, and the costs with those ids are purged.

import weave

client = weave.init("my_custom_cost_model")

costs = client.query_costs(llm_ids=["your_model_name"])
client.purge_costs([cost.id for cost in costs])

Calculating costs for a Project

You can calculate costs for a project by using our calls_query and adding include_costs=True with a little bit of setup.

import weave

weave.init("project_costs")
@weave.op()
def get_costs_for_project(project_name: str):
total_cost = 0
requests = 0

client = weave.init(project_name)
# Fetch all the calls in the project
calls = list(
client.get_calls(filter={"trace_roots_only": True}, include_costs=True)
)

for call in calls:
# If the call has costs, we add them to the total cost
if call.summary["weave"] is not None and call.summary["weave"].get("costs", None) is not None:
for k, cost in call.summary["weave"]["costs"].items():
requests += cost["requests"]
total_cost += cost["prompt_tokens_total_cost"]
total_cost += cost["completion_tokens_total_cost"]

# We return the total cost, requests, and calls
return {
"total_cost": total_cost,
"requests": requests,
"calls": len(calls),
}

# Since we decorated our function with @weave.op(),
# our totals are stored in weave for historic cost total calculations
get_costs_for_project("my_custom_cost_model")

Setting up a custom model with custom costs

Try our cookbook for a Setting up costs with a custom model or