Together AI Deploy Integration

Overview

Deploy a containerized Together AI inference integration service with Docker. This skill covers building a production image that connects to Together's OpenAI-compatible API for running completions, embeddings, and image generation across 100+ open-source models. Includes environment configuration for model selection and batch processing, health checks that verify API key validity and model availability, and rolling update strategies for zero-downtime deployments serving real-time inference requests.

Docker Configuration

FROM python:3.12-slim AS builder
WORKDIR /app
COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt

FROM python:3.12-slim
RUN groupadd -r app && useradd -r -g app app
WORKDIR /app
COPY --from=builder /usr/local/lib/python3.12/site-packages /usr/local/lib/python3.12/site-packages
COPY --from=builder /usr/local/bin /usr/local/bin
COPY src/ ./src/
USER app
EXPOSE 8000
HEALTHCHECK --interval=30s --timeout=5s --retries=3 \
  CMD curl -f http://localhost:8000/health || exit 1
CMD ["python", "src/server.py"]

Environment Variables

export TOGETHER_API_KEY="tog_xxxxxxxxxxxx"
export TOGETHER_BASE_URL="https://api.together.xyz/v1"
export TOGETHER_DEFAULT_MODEL="meta-llama/Llama-3.1-8B-Instruct"
export TOGETHER_MAX_TOKENS="2048"
export LOG_LEVEL="info"
export PORT="8000"

Health Check Endpoint

import express from 'express';

const app = express();

app.get('/health', async (req, res) => {
  try {
    const response = await fetch(`${process.env.TOGETHER_BASE_URL}/models`, {
      headers: { 'Authorization': `Bearer ${process.env.TOGETHER_API_KEY}` },
    });
    if (!response.ok) throw new Error(`Together API returned ${response.status}`);
    res.json({ status: 'healthy', service: 'together-integration', model: process.env.TOGETHER_DEFAULT_MODEL, timestamp: new Date().toISOString() });
  } catch (error) {
    res.status(503).json({ status: 'unhealthy', error: (error as Error).message });
  }
});

Deployment Steps

Step 1: Build

docker build -t together-integration:latest .

Step 2: Run

docker run -d --name together-integration \
  -p 8000:8000 \
  -e TOGETHER_API_KEY -e TOGETHER_BASE_URL -e TOGETHER_DEFAULT_MODEL \
  together-integration:latest

Step 3: Verify

curl -s http://localhost:8000/health | jq .

Step 4: Rolling Update

docker build -t together-integration:v2 . && \
docker stop together-integration && \
docker rm together-integration && \
docker run -d --name together-integration -p 8000:8000 \
  -e TOGETHER_API_KEY -e TOGETHER_BASE_URL -e TOGETHER_DEFAULT_MODEL \
  together-integration:v2

Error Handling

Issue	Cause	Fix
`401 Unauthorized`	Invalid API key	Regenerate key at api.together.xyz/settings
`Model not found`	Wrong model ID string	List models with `GET /v1/models` or check docs
`429 Rate Limited`	Exceeding requests per minute	Implement backoff; use batch inference for 50% cost savings
`500 Server Error`	Model overloaded or unavailable	Retry with exponential backoff; try alternate model
Slow inference	Model cold start on first request	Use a smaller model or keep-alive with periodic requests

Resources

Next Steps

See together-webhooks-events.