Skip to content

container.md

Latest commit

 

History

History
238 lines (185 loc) · 17.1 KB

container.md

File metadata and controls

238 lines (185 loc) · 17.1 KB
Raw

Run agentic-api in a container

The production image contains only the Rust gateway and its runtime libraries. It does not contain Python, vLLM, GPU libraries, model weights, Cargo, or the Rust toolchain. Run inference and PostgreSQL as external services.

Install from crates.io instead

Since 0.4.0 the gateway is also published to crates.io, so a container is not required on a host with a Rust toolchain (MSRV 1.98.1):

cargo install agentic-server --version 0.4.0 --locked

This installs two binaries: agentic, the harness CLI (agentic serve starts the gateway alone; agentic run claude|codex starts it and launches a harness), and agentic-server, the standalone server configured through the same environment variables as the container below. The container remains the recommended path for Kubernetes and for hosts without a toolchain.

Build the image

The multi-stage build pins its Rust and Debian bases by digest, uses BuildKit caches, and copies only agentic-server into the runtime stage. Dependabot proposes weekly digest updates so base-image changes remain explicit and reviewable.

DOCKER_BUILDKIT=1 docker build \
  --build-arg OCI_CREATED="$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
  --build-arg OCI_REVISION="$(git rev-parse HEAD)" \
  --build-arg OCI_VERSION="dev" \
  --tag agentic-api:dev \
  .

For a release build, pass the version being packaged instead of dev and tag the image to match, for example OCI_VERSION="0.4.0" with --tag agentic-api:0.4.0. That version tag (or, preferably, the digest recorded when the image is pushed) is what an environment overlay such as the one in the Kubernetes guide should pin.

CI also records the workflow name and run URL in the vLLM-compatible image labels. Local builds use local as the pipeline label unless OCI_BUILD_PIPELINE is supplied as a build argument.

Pass --build-arg CARGO_BUILD_JOBS=<n> if the builder needs a different concurrency limit. For multi-platform publishing, use a Buildx builder with native workers where available; emulation is slower.

Configure the gateway

The image starts agentic-server in standalone mode. At minimum, set LLM_API_BASE to an external OpenAI-compatible or vLLM endpoint. Production deployments should also set DATABASE_URL to an external PostgreSQL database.

Variable Default Purpose
LLM_API_BASE none Required upstream inference URL
GATEWAY_HOST 0.0.0.0 Listen address
GATEWAY_PORT 9000 Listen port
DATABASE_URL $AGENTIC_API_HOME/agentic_api.db SQLite or PostgreSQL persistence URL
AGENTIC_API_HOME /var/lib/agentic-api User configuration and default local-state directory
AGENTIC_FILES_STORAGE_DIR $AGENTIC_API_HOME/files Local file bytes; mount persistent writable storage when using the Files API
AGENTIC_FILE_SEARCH_SQLITE_VEC_EXTENSION_PATH unset Trusted sqlite-vec loadable extension for semantic search with a file-backed SQLite database
AGENTIC_MAX_REQUEST_BODY_SIZE_BYTES 10485760 Maximum serialized request size for HTTP bodies and WebSocket messages; must be greater than zero
POSTGRES_MAX_CONNECTIONS 10 Maximum PostgreSQL connections per gateway replica
POSTGRES_ACQUIRE_TIMEOUT_SECONDS 30 Maximum wait for a PostgreSQL pool connection
POSTGRES_LOCK_TIMEOUT_SECONDS 5 Maximum wait for a PostgreSQL row or table lock
POSTGRES_MIGRATION_TIMEOUT_SECONDS 300 Lock and statement timeout while running startup migrations
POSTGRES_STATEMENT_TIMEOUT_SECONDS 30 Maximum runtime for normal PostgreSQL statements
POSTGRES_IDLE_TIMEOUT_SECONDS 600 Recycle idle PostgreSQL connections; 0 disables
POSTGRES_MAX_LIFETIME_SECONDS 1800 Recycle PostgreSQL connections after this lifetime; 0 disables
OPENAI_API_KEY none Credential sent to the upstream service when the client does not supply one
OIDC_ISSUER none Optional OIDC issuer for inbound bearer-token authentication
OIDC_AUDIENCE none Required token audience when OIDC_ISSUER is set
SKIP_LLM_READY_CHECK false Skip startup and recurring upstream probes for providers without /health; PostgreSQL readiness remains enforced
CORS_ALLOWED_ORIGINS none Comma-separated browser origins

The container entrypoint rejects percent-encoded SQLite paths because SQLx decodes them before opening the database. Use a literal filesystem path or PostgreSQL instead.

Do not put credentials into the image or Docker build arguments. Inject them at runtime through a secret manager.

docker run --rm --name agentic-api \
  --publish 127.0.0.1:9000:9000 \
  --env LLM_API_BASE=https://vllm.example.com \
  --env DATABASE_URL=postgresql://agentic-api@postgres.example.com/agentic_api \
  --env OPENAI_API_KEY \
  agentic-api:dev

Inbound authentication is disabled by default. OPENAI_API_KEY is an upstream credential, not a password for callers, so keep the port bound to loopback unless an authenticated ingress protects it or OIDC is enabled.

To enable OIDC bearer authentication, configure both variables:

docker run --rm --name agentic-api \
  --publish 127.0.0.1:9000:9000 \
  --env LLM_API_BASE=https://vllm.example.com \
  --env OPENAI_API_KEY \
  --env OIDC_ISSUER=https://identity.example.com \
  --env OIDC_AUDIENCE=agentic-api \
  agentic-api:dev

The gateway discovers the provider and its JSON Web Key Set before listening. /health and /ready remain public; all /v1/* routes then require an OIDC Authorization: Bearer token. The identity token is consumed by the gateway, and OPENAI_API_KEY supplies the upstream inference credential. See OIDC bearer authentication for the validation and key-rotation contract. For a runnable GitHub-backed setup, follow GitHub authentication with Dex.

If the upstream is running on the Docker host, use http://host.docker.internal:<port> on Docker Desktop. On Linux, add --add-host host.docker.internal:host-gateway.

PostgreSQL production settings

Use the TLS settings required by the managed database provider. Prefer certificate and hostname verification when the provider supplies a CA certificate:

DATABASE_URL='postgresql://agentic-api:password@postgres.example.com/agentic_api?sslmode=verify-full&sslrootcert=/run/secrets/postgres-ca.pem'

sslmode=require encrypts the connection but does not verify the server hostname. Mount private CA certificates and client keys from runtime secrets; do not copy them into the image.

AGENTIC_MAX_REQUEST_BODY_SIZE_BYTES bounds the encoded request the gateway will read — JSON overhead, replayed conversation history, and base64 image attachments included. It is unrelated to the upstream model's token context limit, so raising it does not raise what the inference server will accept. Because inline base64 attachments cost roughly a third more than the source image, deployments that replay image-bearing conversations may need more than the 10 MiB default. Requests above the limit receive 413 Payload Too Large; oversized WebSocket messages and frames are rejected by the transport, which closes the connection. The --max-request-body-size-bytes argument takes precedence over this variable, which in turn overrides server.max_request_body_size_bytes in the configuration file.

AGENTIC_RATE_LIMIT_REQUESTS_PER_MINUTE turns on a per-caller request rate limit, and AGENTIC_RATE_LIMIT_BURST (default: the per-minute rate) sets how many requests a caller may send at once. Both are off by default. The limit is kept in each container's memory, so with N replicas a caller can send up to N times the rate. Without OIDC, callers are told apart by the peer address the container sees, so behind a proxy or load balancer that does not preserve client addresses all callers share one bucket. The --rate-limit-requests-per-minute and --rate-limit-burst arguments take precedence over these variables, which in turn override server.rate_limit_requests_per_minute and server.rate_limit_burst. See Rate limits.

Size the pool across the whole deployment, not one process. Keep replicas * POSTGRES_MAX_CONNECTIONS below the managed database connection limit, with capacity reserved for migrations, administration, and failover. The acquire timeout bounds how long a request waits when the pool is exhausted. Lock and statement timeouts prevent a stalled transaction, slow query, or hot conversation from holding a connection indefinitely. Idle and lifetime recycling protect against stale connections and can be disabled with 0 only when the provider recommends it.

Connect directly to PostgreSQL or use a session-pooling proxy. Transaction-pooling proxies are unsupported because the gateway configures search_path, lock timeouts, and statement timeouts for the lifetime of each pooled session.

Each gateway replica runs the embedded SQLx migrations during startup. PostgreSQL advisory locking serializes concurrent migration attempts, so replica startup is repeatable and only proceeds after the schema is ready. Migration lock waits and statements use the finite migration timeout rather than the shorter application lock timeout. A migration failure or timeout prevents that replica from serving traffic.

The gateway pins every PostgreSQL pool connection to the single schema on search_path that already contains the persistence tables or SQLx migration history. This prevents an earlier empty schema from receiving shadow tables or capturing runtime queries. Startup fails instead of guessing when persistence objects exist in more than one search_path schema. Do not grant untrusted roles CREATE on any schema in the gateway role's search_path.

This release does not read data stored by earlier releases. Before migrating, startup checks whether a database migrated by an earlier release holds any conversations, items, or responses. A database without such rows migrates normally; one with them is handled as follows:

  • The default SQLite database ($AGENTIC_API_HOME/agentic_api.db, used when DATABASE_URL is not set) is dropped and recreated at the latest schema. Its stored conversations and responses are discarded, and a warning is logged.
  • Any other database, including PostgreSQL and an explicit SQLite DATABASE_URL, is rejected. Startup fails with an error asking the operator to reset the database or configure a new one; the stored data is left unchanged.

To upgrade a deployment that uses PostgreSQL or an explicit DATABASE_URL, stop every replica of the earlier release, then either point DATABASE_URL at a new empty database or drop all of the gateway's tables (responses, items, conversations, message_threads, the file_search_* tables, and _sqlx_migrations), according to the deployment's data-retention policy. Back up the database first if its contents must be kept.

On a new PostgreSQL database, the first startup applies the migrations and then widens timestamp and sequence columns to 64-bit integers.

Stored requests now fail if their response or conversation state cannot be persisted. For streaming requests, the gateway sends an error event instead of response.completed. Most client responses use the generic message failed to persist response; the underlying database error is written only to gateway logs. The exception is a conversation conflict: a turn that answers the conversation's pending tool call is refused when another turn, which had to answer the same call, was stored while it ran, with status 400, type invalid_request_error, code conversation_locked, and param conversation. No part of the refused turn is persisted, so the client can retry the request against the conversation's latest state. This prevents clients from receiving a response ID that cannot be continued after a lock timeout or other database failure without exposing database schema or constraint details.

The file search migration (0008_file_search.sql) adds four tables for files, vector stores, attachments, and chunks. Supervisor-managed deployments must apply this migration and grant the runtime role SELECT, INSERT, UPDATE, and DELETE on the new tables before starting the upgraded gateway. File metadata and embeddings use the same database as conversation state. Uploaded bytes use the local filesystem configured by AGENTIC_FILES_STORAGE_DIR. Mount persistent writable storage for the Files API, including PostgreSQL deployments. Replicas serving the same files must share that directory. See file search.

AGENTIC_API_SCHEMA_READY keeps schema changes under supervisor control. Startup performs a read-only compatibility check and fails if the database was created by an earlier release, if required persistence columns, types, nullability, primary/foreign-key/CHECK constraints, or the conversation sequence index are missing, or if persistence integer columns still need widening. Every startup requires message_threads from migration 0012_message_threads.sql, including its state/checkpoint/lease CHECK constraints and SELECT/INSERT/UPDATE/DELETE privileges, even when clients send no thread requests. Apply the repository migrations through 0011_file_search_vec_map.sql to an empty schema with a DDL-capable migration role, then apply 0012_message_threads.sql and widen the integer columns in one transaction before starting the DML-only gateway role. Run the example below from the repository root with psql -v ON_ERROR_STOP=1 so any statement failure stops the script:

BEGIN;
\i crates/agentic-server-core/migrations/0012_message_threads.sql
ALTER TABLE conversations
    ALTER COLUMN created_at TYPE BIGINT USING created_at::BIGINT;
ALTER TABLE items
    ALTER COLUMN created_at TYPE BIGINT USING created_at::BIGINT,
    ALTER COLUMN seq TYPE BIGINT USING seq::BIGINT;
ALTER TABLE responses
    ALTER COLUMN created_at TYPE BIGINT USING created_at::BIGINT;
COMMIT;

Enable automated backups and point-in-time recovery in the managed database service according to the deployment's recovery-point and recovery-time requirements. Test restores separately; gateway replicas do not create database backups.

Smoke test

The liveness probe reports whether the gateway process is serving traffic. Startup performs one rollback-only functional write through the conversation, item, and response tables to verify the configured role and persistence policies. Readiness then checks the upstream inference service and runs a one-second read-only database, schema, and privilege probe, avoiding write amplification from orchestrator polling.

curl --fail http://127.0.0.1:9000/health
curl --fail http://127.0.0.1:9000/ready

The container CI workflow builds the image, verifies that build tools are absent, launches the gateway against a mock upstream, checks both probes, and exercises a stored Responses API request through SQLite persistence. HTTP streaming and WebSockets use the same gateway binary and exposed port; the image does not add a transport proxy.

On SIGTERM, the gateway stops accepting connections and gives in-flight requests up to eight seconds to drain before closing the remaining connections. Set an orchestrator termination grace period longer than eight seconds; the default 30-second Kubernetes grace period and the documented 10-second Docker stop timeout both satisfy this requirement.

Kubernetes and OpenShift security context

The image defaults to UID 10001 and GID 0. Its working directory is setgid and the entrypoint uses a group-cooperative umask, so new SQLite files remain writable when OpenShift replaces the UID while retaining the group-0 permission model. Do not set a fixed runAsUser when the cluster assigns arbitrary UIDs.

A volume mounted at /var/lib/agentic-api hides the ownership and mode stored in the image. For SQLite or local file storage, configure the storage class or pod-level fsGroup so the mounted directory is writable by a supplemental group assigned to the container. The example below uses group 0 to match the image; if the cluster assigns a different permitted supplemental group, use that group and ensure the volume root is group-writable and setgid. PostgreSQL deployments that do not use the Files API do not need this pod-level filesystem setting.

Volumes initialized by an older image may contain SQLite files without group-write permission. Before rotating to an arbitrary UID, repair those volumes once as an administrator with chmod -R g+rwX /var/lib/agentic-api.

spec:
  securityContext:
    fsGroup: 0
    fsGroupChangePolicy: OnRootMismatch
  containers:
    - name: agentic-api
      securityContext:
        allowPrivilegeEscalation: false
        capabilities:
          drop: ["ALL"]
        runAsNonRoot: true
        seccompProfile:
          type: RuntimeDefault

Mount persistent writable storage at /var/lib/agentic-api when using SQLite or the Files API. With PostgreSQL, the file volume can instead be mounted at AGENTIC_FILES_STORAGE_DIR; replicas accessing the same files must share it. Back up that directory together with the database.

For a replicated PostgreSQL deployment, use the Kubernetes manifests and operational guide.