Docs / Keys, sign-in and audit capture

Keys, sign-in and audit capture

The gateway in front of every deployment is one component with three placements. Node-local runs on the GPU instance itself, which is the default and involves nothing of ours in the request path. The hosted edge is a small pair we run, included in the plan. The private edge is a tiny always-on VM in your own AWS or Nebius account. This page covers what the gateway enforces: keys, identity-provider sign-in and audit capture. Stop and start covers what an edge does while the deployment sleeps.

Named keys

A deployment holds a collection of keys, not one. Each has a name, a public id for attribution and revocation, and optional limits the gateway enforces: requests per minute (a token bucket per key) and tokens per UTC day, counted from the engine's usage report. The key itself is shown once, at creation, and stored only encrypted. The last live key cannot be revoked; create another first.

# Create a key with limits (the key is in the response, once)
curl -X POST https://app.llmhangar.com/v1/deployments/$ID/keys \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"name": "analytics", "rate_limit_rpm": 60, "daily_token_limit": 2000000}'

# List (ids, names, limits, never material) and revoke by id
curl https://app.llmhangar.com/v1/deployments/$ID/keys -H "Authorization: Bearer $TOKEN"
curl -X DELETE https://app.llmhangar.com/v1/deployments/$ID/keys/key_3f9a1c2e -H "Authorization: Bearer $TOKEN"

On an edge placement a change reaches the gateway within seconds. On the node-local placement the collection is rendered into the gateway when the instance boots, so a change lands at the deployment's next start; the API response says which.

Sign-in with your identity provider

On an edge placement the gateway accepts JWTs your IdP issues beside the keys. Validation is strict: the issuer and audience must match exactly, the signature is checked against the issuer's JWKS with an explicit allowlist of asymmetric algorithms (never none, never HMAC), an expiry is required, and clock skew is bounded at five minutes. The JWKS is refreshed when an unknown key id appears, at most once a minute, and on a one-hour TTL; keys older than 24 hours are refused even when the issuer is unreachable. The token's subject and issuer land in the audit record. You manage endpoint identities through your own identity provider.

curl -X PUT https://app.llmhangar.com/v1/deployments/$ID/auth \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"jwt": {"issuer": "https://login.example.com/", "audience": "llm-endpoint",
              "jwks_url": "https://login.example.com/.well-known/jwks.json",
              "algorithms": ["RS256"], "clock_skew_seconds": 60}}'

Audit capture to your own bucket

With capture on, the gateway writes a record for every request and every response to a bucket you own (S3 or S3-compatible), with write-only credentials you mint. The control plane stores those credentials encrypted, hands them only to that deployment's gateway, and never holds content.

Record before forwarding. The request record is written to the gateway's local spool and synced to disk before the request reaches the model. The response record follows the last byte, with the digest of what was streamed and a flag when the stream ended early.
Hash chained and anchored. Records form one chain per gateway instance, each linking to the previous by hash. The chain head, the record count and the count of anything the gateway had to drop are posted to the control plane every 50 records or 30 seconds. A record removed or altered breaks the chain against the anchor.
Capture mode. metadata stores digests, sizes, timing, status, the key id or the IdP subject; full adds the prompt and the response up to 1 MB each.
Policy. best_effort keeps serving through a bucket outage (records spool locally up to 256 MB and upload later in order; beyond that the gap is counted and anchored). fail_closed refuses to serve what it cannot record: no unlogged inference.

curl -X PUT https://app.llmhangar.com/v1/deployments/$ID/audit \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"bucket": "acme-llm-audit", "prefix": "prod", "region": "eu-central-1",
       "access_key_id": "AKIA...", "secret_access_key": "...",
       "capture": "metadata", "policy": "fail_closed"}'

# The anchored chain heads per gateway instance
curl https://app.llmhangar.com/v1/deployments/$ID/audit/checkpoints -H "Authorization: Bearer $TOKEN"

Retention, deletion and object lock are the bucket's own settings. Verifying a chain means listing the objects under the prefix in order, recomputing each hash over the previous hash and the record, and comparing the last one with the anchored head.

Placement and failure policy

Keys are verified by the node-local gateway on every placement, so an edge that is down never becomes a way around the key check. What an edge adds while the instance is off, the sleeping page and wake on request, degrades to a plain connection error on the default placement. A hosted-edge deployment chooses at deploy whether its hostname fails over to the instance while the pair is down (failover_origin, the default) or stays down until the pair is back (fail_closed), which is the right choice when audit capture must not be bypassed. The private edge runs a digest-pinned image in your account with no access from us; upgrades are yours to trigger.