# Holbrooke Capital — Inference API (agent onboarding)

> **OpenAI-compatible API.** Configure your client/agent with its **OpenAI** (a.k.a.
> "OpenAI-compatible" / "Custom OpenAI") provider, pointed at the base URL below.

## Connection settings
| Setting | Value |
|---|---|
| Provider type | **OpenAI / OpenAI-compatible** |
| Base URL | `https://inference.holbrookecapital.com/v1` |
| API key | your `sk-...` key |
| Models | `qwen-uncensored` · `gemma-4-31b` · `gemma4-26b-uncensored` |

## Authentication
Send `Authorization: Bearer <your sk-... key>` on all inference calls
(`/v1/chat/completions`) and on `/v1/models` — both return **401** without a key.

> The Ollama-style discovery endpoints (`/api/tags`, `/api/show`, `/api/version`) respond
> **keyless** so an agent can auto-detect the provider before a key is configured. Everything
> that actually runs inference requires the key.

**Put the key in your client's API-key field directly — not only an environment variable.** Some
agent frameworks do not read `OPENAI_API_KEY` for a "custom" provider and silently send a
placeholder, causing 401s. Set the key explicitly in the model/provider config. *(Hermes Desktop:
set `model.api_key` in `config.yaml` — its `custom` resolver does not read `OPENAI_API_KEY`.)*

## Endpoints
- `POST /v1/chat/completions` — chat (streaming via `"stream": true`, OpenAI SSE). Accepts the
  OpenAI `tools` / function-calling schema.
- `GET  /v1/models` — list models (key required).

## Models
| Model | Use | Max context |
|---|---|---|
| `qwen-uncensored` | unfiltered assistant, agent-friendly (recommended default) | **256K** tokens |
| `gemma-4-31b` | general-purpose assistant | **96K** tokens |
| `gemma4-26b-uncensored` | uncensored assistant (MoE 26B-A4B) | **128K** tokens |

## Chat completion
```
POST https://inference.holbrookecapital.com/v1/chat/completions
Authorization: Bearer YOUR_API_KEY
Content-Type: application/json

{ "model": "qwen-uncensored", "messages": [{"role":"user","content":"Hello"}], "stream": false }
```

## Python (OpenAI SDK)
```python
from openai import OpenAI
client = OpenAI(base_url="https://inference.holbrookecapital.com/v1", api_key="YOUR_API_KEY")
print(client.chat.completions.create(
    model="qwen-uncensored",
    messages=[{"role": "user", "content": "Hello"}],
).choices[0].message.content)
```

## Latency — what to expect
- **First request to an idle model** loads it on demand (~3–5s), then it stays warm — subsequent
  calls are fast (often <1s for short follow-ups).
- **Large prompts take longer to prefill.** Latency scales with prompt size: a fresh 20–30K-token
  agentic context is ~15–25s; short follow-ups are ~1s. This is expected for big contexts, not a
  failure. Use `"stream": true` to get the first token as soon as prefill finishes.

## Notes
- Standard OpenAI params apply: `temperature`, `top_p`, `max_tokens`, `stop`, `stream`, `tools`.
- Standard SDK clients (OpenAI Python/JS, etc.) and OpenAI-compatible agents work as-is.
- A `401` almost always means the key wasn't actually sent — set it in the client's api_key field,
  not only an env var (see Authentication).

## Get a key
Create a free account at https://holbrookecapital.com/signup, then put your
`sk-...` key in your client's **API-key field** (not only an environment variable):

```python
from openai import OpenAI
client = OpenAI(
    base_url="https://inference.holbrookecapital.com/v1",
    api_key="YOUR_SK_KEY_HERE"
)
```