OpenRouter

Openrouter.ai is a unified interface for many LLMs, supporting both foundational models like OpenAI GPT-4, Anthropic Claude, Google Gemini but also open source models like LLama-3, Mixtral and many more, some models are even offered for free.

OpenRouter offers a Rest API and an OpenAI SDK compatibility (docs) which Weave automatically detects and integrates with (see OpenRouter quick start for more details).

To switch your OpenAI SDK code to OpenRouter, switch out the API key to your OpenRouter API key, base_url to https://openrouter.ai/api/v1, and model to one of their many chat models. When you call weave.init(), provide a project name for your traces. If not specified, your default entity is used. To find or update your default entity, refer to [User Settings](https://docs.wandb.ai/guides/models/app/settings-page/user-settings/#default-team) in the W&B Models documentation.

import os
import openai
import weave

weave.init('openrouter-weave')

system_content = "You are a travel agent. Be descriptive and helpful."
user_content = "Tell me about San Francisco"

client = openai.OpenAI(
    api_key=os.environ.get("OPENROUTER_API_KEY"),
    base_url="https://openrouter.ai/api/v1",
)
chat_completion = client.chat.completions.create(
    extra_headers={
    "HTTP-Referer": $YOUR_SITE_URL, # Optional, for including your app on openrouter.ai rankings.
    "X-Title": $YOUR_APP_NAME, # Optional. Shows in rankings on openrouter.ai.
    },
    model="meta-llama/llama-3.1-8b-instruct:free",
    messages=[
        {"role": "system", "content": system_content},
        {"role": "user", "content": user_content},
    ],
    temperature=0.7,
    max_tokens=1024,
)
response = chat_completion.choices[0].message.content
print("Model response:\n", response)

While this is a simple example to get started, see our OpenAI guide for more details on how to integrate Weave with your own functions for more complex use cases.