Together AI

Together AI is a platform for building and finetuning generative AI models, focusing on Open Source LLMs, and allowing customers to fine-tune and host their own models.

info

Full Weave together python package support is currently in development

While full Weave support for the together python package is currently in development, Together supports the OpenAI SDK compatibility (docs) which Weave automatically detects and integrates with.

To switch to using the Together API, replace the API key with your Together API key, set base_url to https://api.together.xyz/v1, and set the model to one of their chat models. When you call weave.init(), provide a project name for your traces. If not specified, your default entity is used. To find or update your default entity, refer to [User Settings](https://docs.wandb.ai/guides/models/app/settings-page/user-settings/#default-team) in the W&B Models documentation.

import os
import openai
import weave

weave.init('together-weave')

system_content = "You are a travel agent. Be descriptive and helpful."
user_content = "Tell me about San Francisco"

client = openai.OpenAI(
    api_key=os.environ.get("TOGETHER_API_KEY"),
    base_url="https://api.together.xyz/v1",
)
chat_completion = client.chat.completions.create(
    model="mistralai/Mixtral-8x7B-Instruct-v0.1",
    messages=[
        {"role": "system", "content": system_content},
        {"role": "user", "content": user_content},
    ],
    temperature=0.7,
    max_tokens=1024,
)
response = chat_completion.choices[0].message.content
print("Together response:\n", response)

While this is a simple example to get started, see our OpenAI guide for more details on how to integrate Weave with your own functions for more complex use cases.