@arizeai/ax-client is a TypeScript client for the Arize AX REST API. It is still under active development and is subject to change.
Arize is an AI engineering platform. It helps engineers develop, evaluate, and observe AI applications and agents.
Arize has both Enterprise and OSS products to support this goal:
- Arize AX — an enterprise AI engineering platform from development to production, with an embedded AI Copilot
- Phoenix — a lightweight, open-source project for tracing, prompt engineering, and evaluation
- OpenInference — an open-source instrumentation package to trace LLM applications across models and frameworks
- Tracing - Trace your LLM application's runtime using OpenTelemetry-based instrumentation.
- Evaluation - Leverage LLMs to benchmark your application's performance using response and retrieval evals.
- Datasets - Create versioned datasets of examples for experimentation, evaluation, and fine-tuning.
- Experiments - Track and evaluate changes to prompts, LLMs, and retrieval.
- Playground- Optimize prompts, compare models, adjust parameters, and replay traced LLM calls.
- Prompt Management- Manage and test prompt changes systematically using version control, tagging, and experimentation.
- Overview
- Key Features
- Installation
- Configuration
- Datasets
- Experiments
- Prompts
- Evaluators
- Tasks
- Spans
- Traces
- Annotation Configs
- Annotation Queues
- AI Integrations
- Integrations
- Projects
- API Keys
- Roles
- Role bindings
- Resource restrictions
- Organizations
- Organization memberships
- Users
- Spaces
- Space memberships
- Webhooks
- Webhook subscriptions
- REST endpoints
# or yarn, pnpm, bun, etc.
npm install @arizeai/ax-clientThe client will automatically read environment variables from your environment, if available.
The following environment variables are used:
ARIZE_API_KEY- The API key to use for authentication.ARIZE_BASE_URL- The base URL of the Arize AX API.
Alternatively, you can pass configuration options to the client directly, and they will be prioritized over environment variables and default values.
The @arizeai/ax-client package allows you to create and manage datasets and their examples.
import { listDatasets } from "@arizeai/ax-client";
const datasets = await listDatasets({ space: "my-space" });
console.log(datasets);Create a dataset by providing a space (name or ID), name, and array of examples (each containing at least one property).
import { createDataset } from "@arizeai/ax-client";
const dataset = await createDataset({
name: "my-dataset",
space: "my-space",
examples: [{ question: "What is 2+2?", answer: "4", topic: "math" }],
});import { getDataset } from "@arizeai/ax-client";
const dataset = await getDataset({ dataset: "my-dataset", space: "my-space" });
console.log(dataset);import { updateDataset } from "@arizeai/ax-client";
const dataset = await updateDataset({
dataset: "my-dataset",
space: "my-space",
name: "my-renamed-dataset",
});import { deleteDataset } from "@arizeai/ax-client";
await deleteDataset({ dataset: "my-dataset", space: "my-space" });Lists the examples of a dataset. filter is an optional SQL-like expression
for more complex queries.
import { listDatasetExamples } from "@arizeai/ax-client";
const examples = await listDatasetExamples({
dataset: "my-dataset",
space: "my-space",
});
console.log(examples);
const filtered = await listDatasetExamples({
dataset: "my-dataset",
space: "my-space",
filter: "question = 'What is 2+2?'",
});
console.log(filtered);import { appendExamples } from "@arizeai/ax-client";
const result = await appendExamples({
dataset: "my-dataset",
space: "my-space",
examples: [{ question: "What is 2+2?", answer: "4", topic: "math" }],
});
console.log(result.exampleIds); // IDs of the inserted examplesUpdating examples creates a new version of the dataset.
import { updateExamples } from "@arizeai/ax-client";
const dataset = await updateExamples({
dataset: "my-dataset",
space: "my-space",
examples: [{ id: "your_example_id", question: "What is 2+2?", answer: "4" }],
newVersionName: "your_new_version_name",
});import { annotateDatasetExamples } from "@arizeai/ax-client";
await annotateDatasetExamples({
space: "my-space",
dataset: "my-dataset",
annotations: [
{
recordId: "example_id_abc123",
values: [
{ name: "quality", score: 0.9 },
{ name: "topic", label: "science" },
],
},
],
});Examples are removed in place from the given version; no new version is created. The delete is partial-tolerant and idempotent — the result reports which IDs were deleted and which were not.
import { deleteDatasetExamples } from "@arizeai/ax-client";
const result = await deleteDatasetExamples({
dataset: "my-dataset",
space: "my-space",
datasetVersionId: "your_dataset_version_id",
examples: ["example_id_1", "example_id_2"],
});
console.log(
result.completed,
result.deletedExampleIds,
result.notDeletedExampleIds,
);The @arizeai/ax-client package allows you to create and manage experiments and their runs.
import { listExperiments } from "@arizeai/ax-client";
const experiments = await listExperiments({
dataset: "my-dataset",
space: "my-space",
});
console.log(experiments);import { createExperiment } from "@arizeai/ax-client";
const experiment = await createExperiment({
experimentName: "my-experiment",
dataset: "my-dataset",
space: "my-space",
experimentRuns: [],
});
console.log(experiment);import { getExperiment } from "@arizeai/ax-client";
// Using names
const experiment = await getExperiment({
experiment: "my-experiment",
dataset: "my-dataset",
space: "my-space",
});
// Using an ID directly
const byId = await getExperiment({ experiment: "your_experiment_id" });import { deleteExperiment } from "@arizeai/ax-client";
await deleteExperiment({
experiment: "my-experiment",
dataset: "my-dataset",
space: "my-space",
});List experiment runs by providing an experiment name or ID (space and dataset context are required when using names), optionally narrowed by an SQL-like filter over run columns, evaluations (eval.<name>.score), and annotations (annotation.<name>.*). Runs are returned in stable ID ascending order — pass pagination.nextCursor back as cursor to fetch the next page, keeping the filter unchanged while paging.
import { listExperimentRuns } from "@arizeai/ax-client";
const experimentRuns = await listExperimentRuns({
experiment: "my-experiment",
dataset: "my-dataset",
space: "my-space",
});
const filtered = await listExperimentRuns({
experiment: "my-experiment",
dataset: "my-dataset",
space: "my-space",
filter: "eval.quality.score < 0.5",
limit: 50,
});
console.log(filtered.data);Append between 1 and 1000 new runs to an existing experiment. Each run must include exampleId (the ID of an example from the experiment's dataset) and output. The response includes the updated experiment and the generated run IDs in input order.
import { appendExperimentRuns } from "@arizeai/ax-client";
const result = await appendExperimentRuns({
space: "my-space",
dataset: "my-dataset",
experiment: "my-experiment",
experimentRuns: [{ exampleId: "ex_abc123", output: "The answer is 42" }],
});
console.log(result.runIds); // IDs of the appended runsimport { annotateExperimentRuns } from "@arizeai/ax-client";
await annotateExperimentRuns({
space: "my-space",
dataset: "my-dataset",
experiment: "my-experiment",
annotations: [
{
recordId: "run_id_abc123",
values: [
{ name: "accuracy", label: "correct", score: 1.0 },
{ name: "notes", text: "Well-structured output" },
],
},
],
});The @arizeai/ax-client package allows you to create and manage prompts, their versions, and labels.
A prompt is created together with its initial version.
import { createPrompt } from "@arizeai/ax-client";
const prompt = await createPrompt({
space: "my-space",
name: "customer-support",
description: "A prompt for customer support interactions",
version: {
commitMessage: "Initial version",
inputVariableFormat: "F_STRING",
provider: "OPEN_AI",
model: "gpt-4",
messages: [
{ role: "SYSTEM", content: "You are a helpful assistant." },
{ role: "USER", content: "Hello, {name}!" },
],
},
});By default the latest version is returned. Pass versionId or label to resolve a specific version instead.
import { getPrompt } from "@arizeai/ax-client";
// Latest version
const prompt = await getPrompt({
prompt: "customer-support",
space: "my-space",
});
// Version tagged with the "production" label
const productionPrompt = await getPrompt({
prompt: "customer-support",
space: "my-space",
label: "production",
});import { listPrompts } from "@arizeai/ax-client";
const { data, pagination } = await listPrompts({ space: "my-space" });
console.log(data.map((p) => p.name));import { updatePrompt } from "@arizeai/ax-client";
const updated = await updatePrompt({
prompt: "customer-support",
space: "my-space",
description: "Updated description for the prompt",
});import { deletePrompt } from "@arizeai/ax-client";
await deletePrompt({ prompt: "customer-support", space: "my-space" });You can list all versions of a prompt, retrieve a specific version by its ID, and create new versions.
import {
listPromptVersions,
getPromptVersion,
createPromptVersion,
} from "@arizeai/ax-client";
// List versions
const { data: versions } = await listPromptVersions({
prompt: "customer-support",
space: "my-space",
});
// Get a specific version by its ID (pure ID, no name resolution)
const version = await getPromptVersion({ versionId: "your-version-id" });
// Create a new version
const newVersion = await createPromptVersion({
prompt: "customer-support",
space: "my-space",
commitMessage: "Updated system prompt",
inputVariableFormat: "F_STRING",
provider: "OPEN_AI",
model: "gpt-4",
messages: [
{ role: "SYSTEM", content: "You are a helpful assistant." },
{ role: "USER", content: "Hello, {name}!" },
],
});Labels let you tag a specific prompt version with a named alias (e.g. "production", "staging").
import {
getPromptVersionByLabel,
setPromptVersionLabels,
deletePromptVersionLabel,
} from "@arizeai/ax-client";
// Resolve the version a label currently points to
const version = await getPromptVersionByLabel({
prompt: "customer-support",
space: "my-space",
labelName: "production",
});
// Set labels on a version (replaces all existing labels)
const { labels } = await setPromptVersionLabels({
versionId: "your-version-id",
labels: ["production", "staging"],
});
// Remove a single label from a version
await deletePromptVersionLabel({
versionId: "your-version-id",
labelName: "staging",
});The @arizeai/ax-client package allows you to create and manage LLM-as-a-judge (template), code, and remote evaluators, along with their versions.
import { listEvaluators } from "@arizeai/ax-client";
const evaluators = await listEvaluators({ space: "my-space" });
console.log(evaluators);import { createTemplateEvaluator } from "@arizeai/ax-client";
const evaluator = await createTemplateEvaluator({
name: "Relevance",
space: "my-space",
commitMessage: "Initial version",
templateConfig: {
name: "Relevance",
template:
"Is the response relevant?\nQuery: {{query}}\nResponse: {{response}}",
includeExplanations: true,
useFunctionCalling: true,
classificationChoices: { relevant: 1, irrelevant: 0 },
direction: "MAXIMIZE",
llmConfig: {
aiIntegrationId: "QUlJbnRlZ3JhdGlvbjphYmMxMjM=",
modelName: "gpt-4o",
invocationParameters: { temperature: 0 },
providerParameters: {},
},
},
});import { createCodeEvaluator } from "@arizeai/ax-client";
const evaluator = await createCodeEvaluator({
name: "JSON Parseable",
space: "my-space",
commitMessage: "Initial version",
codeConfig: {
type: "MANAGED",
name: "json_parseable",
managedEvaluator: "JSON_PARSEABLE",
variables: ["output"],
},
});A remote evaluator calls a customer-hosted HTTP endpoint via an EVALUATOR integration.
Requires the enableRemoteEvalTasks feature flag on the account.
import { createRemoteEvaluator } from "@arizeai/ax-client";
const evaluator = await createRemoteEvaluator({
name: "My Remote Eval",
space: "my-space",
commitMessage: "Initial version",
integrationId: "<evaluator-integration-id>",
});import { getEvaluator } from "@arizeai/ax-client";
const evaluator = await getEvaluator({
evaluator: "Relevance",
space: "my-space",
});
console.log(evaluator);import { updateEvaluator } from "@arizeai/ax-client";
const evaluator = await updateEvaluator({
evaluator: "Relevance",
space: "my-space",
name: "Updated Evaluator Name",
});import { deleteEvaluator } from "@arizeai/ax-client";
await deleteEvaluator({ evaluator: "Relevance", space: "my-space" });You can list all versions of an evaluator, retrieve a specific version by its ID, and create new versions.
import {
listEvaluatorVersions,
getEvaluatorVersion,
createTemplateEvaluatorVersion,
} from "@arizeai/ax-client";
// List versions
const versions = await listEvaluatorVersions({
evaluator: "Relevance",
space: "my-space",
});
// Get a specific version by its ID
const version = await getEvaluatorVersion({ versionId: "your-version-id" });
// Create a new template version (use createCodeEvaluatorVersion for code evaluators,
// or createRemoteEvaluatorVersion for remote evaluators)
const newVersion = await createTemplateEvaluatorVersion({
evaluator: "Relevance",
space: "my-space",
commitMessage: "Updated prompt template",
templateConfig: {
name: "Relevance",
template: "Rate the relevance.\nQuery: {{query}}\nResponse: {{response}}",
includeExplanations: true,
useFunctionCalling: true,
classificationChoices: { relevant: 1, irrelevant: 0 },
direction: "MAXIMIZE",
llmConfig: {
aiIntegrationId: "QUlJbnRlZ3JhdGlvbjphYmMxMjM=",
modelName: "gpt-4o",
invocationParameters: { temperature: 0 },
providerParameters: {},
},
},
});
// Create a new remote version (switches to a different EVALUATOR integration)
import { createRemoteEvaluatorVersion } from "@arizeai/ax-client";
const remoteVersion = await createRemoteEvaluatorVersion({
evaluator: "My Remote Eval",
space: "my-space",
commitMessage: "Switch endpoint",
integrationId: "<evaluator-integration-id>",
});import { deleteEvaluatorVersions } from "@arizeai/ax-client";
await deleteEvaluatorVersions({
evaluator: "Relevance",
space: "my-space",
versionIds: ["your_version_id_1", "your_version_id_2"],
});The @arizeai/ax-client package allows you to create and manage scheduled or on-demand tasks (server-side evaluations and experiments) and monitor their runs.
Tip: Prefer the narrowly-typed
createEvaluationTaskandcreateRunExperimentTaskhelpers shown below. A lower-levelcreateTaskis also exported if you need to construct a request payload directly.
import { listTasks } from "@arizeai/ax-client";
const tasks = await listTasks({ space: "my-space", name: "prod" });
console.log(tasks);Each evaluator is either a span-shape entry (with columnMappings / queryFilter) or a
trace/session-shape entry (with queryMappings). Supply one shape per evaluator; they are
mutually exclusive via the SpanEvaluatorInput | TraceOrSessionEvaluatorInput union.
Span-granularity task:
import { createEvaluationTask } from "@arizeai/ax-client";
const task = await createEvaluationTask({
name: "Weekly Quality Check",
type: "TEMPLATE_EVALUATION",
space: "my-space",
project: "my-project",
queryFilter: "span_kind = 'LLM'",
evaluators: [
{
evaluatorId: "your_evaluator_id",
columnMappings: {
input: "attributes.input.value",
output: "attributes.output.value",
},
},
],
});Trace/session-granularity task (multi-span query):
const task = await createEvaluationTask({
name: "Trace Quality Check",
type: "TEMPLATE_EVALUATION",
space: "my-space",
project: "my-project",
queryFilters: {
filters: [
{ id: "A", filter: "span_kind = 'LLM'" },
{ id: "B", filter: "span_kind = 'RETRIEVER'" },
],
expression: "A AND B",
},
evaluators: [
{
evaluatorId: "your_evaluator_id",
queryMappings: [
{
variableName: "input",
queryIds: ["A"],
attributePath: "attributes.input.value",
},
{
variableName: "output",
queryIds: ["B"],
attributePath: "attributes.output.value",
},
],
},
],
});import { createRunExperimentTask } from "@arizeai/ax-client";
const task = await createRunExperimentTask({
name: "GPT-4o Baseline Task",
dataset: "my-dataset",
space: "my-space",
runConfiguration: {
experiment_type: "LLM_GENERATION",
aiIntegration: "my-openai-integration",
model_name: "gpt-4o",
input_variable_format: "F_STRING",
messages: [
{ role: "SYSTEM", content: "You are a helpful assistant." },
{ role: "USER", content: "Answer: {question}" },
],
},
});import { getTask } from "@arizeai/ax-client";
// By ID
const task = await getTask({ task: "your_task_id" });
// By name (requires space)
const byName = await getTask({ task: "My Task", space: "my-space" });import { updateTask, deleteTask } from "@arizeai/ax-client";
// Basic update — rename and adjust sampling rate
await updateTask({
task: "your_task_id",
name: "Renamed Task",
samplingRate: 0.5,
});
// Switch to trace/session shape (multi-span query)
await updateTask({
task: "your_task_id",
queryFilters: {
filters: [{ id: "A", filter: "span_kind = 'LLM'" }],
expression: "A",
},
evaluators: [
{
evaluatorId: "your_evaluator_id",
queryMappings: [
{
variableName: "input",
queryIds: ["A"],
attributePath: "attributes.input.value",
},
],
},
],
});
await deleteTask({ task: "your_task_id" });import {
triggerTaskRun,
waitForTaskRun,
listTaskRuns,
getTaskRun,
cancelTaskRun,
} from "@arizeai/ax-client";
// Trigger a run, then poll until it finishes
const run = await triggerTaskRun({ task: "My Task", space: "my-space" });
const finalRun = await waitForTaskRun({
runId: run.id,
pollInterval: 3_000, // poll every 3 seconds
timeout: 5 * 60_000, // give up after 5 minutes
});
console.log(finalRun.status); // "COMPLETED" | "FAILED" | "CANCELLED"
// List, inspect, or cancel runs
const { data: runs } = await listTaskRuns({
task: "My Task",
space: "my-space",
});
const single = await getTaskRun({ runId: run.id });
await cancelTaskRun({ runId: run.id });The @arizeai/ax-client package allows you to list, annotate, and delete spans within a project.
import { listSpans } from "@arizeai/ax-client";
// By project ID
const spans = await listSpans({ project: "your_project_id" });
// By project name (requires space)
const byName = await listSpans({ project: "My Project", space: "my-space" });Set includedColumns to return only selected columns. Set excludedColumns
to omit selected columns, such as a large embedding vector. Do not set both.
Fixed span fields are always returned.
const spans = await listSpans({
project: "your_project_id",
excludedColumns: ["attributes.embedding.vectors"],
});granularity selects what each recordId identifies: a span (SPAN, the
default), a trace's root span (TRACE), or a session (SESSION, written to
the root span of the session's earliest trace). Up to 1000 records may be
annotated per request for SPAN/TRACE; up to 100 for SESSION.
import { annotateSpans } from "@arizeai/ax-client";
await annotateSpans({
space: "my-space",
project: "my-project",
annotations: [
{
recordId: "c3Bhbl9pZF9hYmMxMjM=", // base64-encoded span ID
values: [
{ name: "quality", score: 0.9 },
{ name: "topic", label: "science" },
],
},
],
});startTime/endTime bound the lookup window and are both optional: when
omitted, endTime defaults to now and startTime defaults to 31 days before
that (7 days for SESSION). If the record is older than the default window,
pass startTime explicitly or the record won't be found.
// Session-granularity example
await annotateSpans({
space: "my-space",
project: "my-project",
granularity: "SESSION",
startTime: new Date(Date.now() - 7 * 24 * 60 * 60 * 1000),
endTime: new Date(),
annotations: [
{
recordId: "20144", // session ID
values: [{ name: "quality", label: "good" }],
},
],
});import { deleteSpans } from "@arizeai/ax-client";
// By project name (requires space)
await deleteSpans({
project: "My Project",
space: "my-space",
spanIds: ["a1b2c3d4e5f6a7b8", "f8e7d6c5b4a39281"],
});The @arizeai/ax-client package allows you to list traces within a project. Each trace carries its full (flat) list of spans plus lightweight roll-up metadata; reconstruct the trace tree client-side using each span's parentId.
import { listTraces } from "@arizeai/ax-client";
// By project ID
const { data: traces, pagination } = await listTraces({
project: "your_project_id",
});
// By project name (requires space), with a filter and time range
const filtered = await listTraces({
project: "My Project",
space: "my-space",
filter: "status_code = 'ERROR'",
startTime: new Date("2024-01-01T00:00:00Z"),
endTime: new Date("2024-01-02T00:00:00Z"),
limit: 50,
});
// Page through results using the opaque cursor
const next = await listTraces({
project: "your_project_id",
cursor: pagination.nextCursor,
});The filter uses the same SQL-like syntax as listSpans, but the semantics are
trace-contains-match: a trace is returned when any of its spans matches the
filter (the matching span is usually a child, not the root).
Each response page also has an overall span limit, so a trace's span list can
be incomplete even when spansTruncated is false. Narrow the time window or
call listSpans with a trace_id filter to retrieve the trace's spans directly.
The @arizeai/ax-client package allows you to create and manage annotation configs, which define the annotations that can be applied to spans and examples.
import { listAnnotationConfigs } from "@arizeai/ax-client";
const annotationConfigs = await listAnnotationConfigs({ space: "my-space" });
console.log(annotationConfigs);import { createAnnotationConfig } from "@arizeai/ax-client";
const annotationConfig = await createAnnotationConfig({
name: "Accuracy",
space: "my-space",
type: "CATEGORICAL",
values: [
{ label: "accurate", score: 1 },
{ label: "inaccurate", score: 0 },
],
optimizationDirection: "MAXIMIZE",
});import { getAnnotationConfig } from "@arizeai/ax-client";
const annotationConfig = await getAnnotationConfig({
annotationConfig: "Accuracy",
space: "my-space",
});There is a dedicated update function per annotation config type. Each one must be called with the stored config's type — a config's type is immutable and cannot be changed. Any fields you omit are left unchanged.
import { updateCategoricalAnnotationConfig } from "@arizeai/ax-client";
const annotationConfig = await updateCategoricalAnnotationConfig({
annotationConfig: "Accuracy",
space: "my-space",
name: "Accuracy v2",
values: [
{ label: "accurate", score: 1 },
{ label: "inaccurate", score: 0 },
],
optimizationDirection: "MAXIMIZE",
});import { updateContinuousAnnotationConfig } from "@arizeai/ax-client";
const annotationConfig = await updateContinuousAnnotationConfig({
annotationConfig: "Accuracy",
space: "my-space",
name: "Accuracy v2",
minimumScore: 0,
maximumScore: 10,
optimizationDirection: "MAXIMIZE",
});import { updateFreeformAnnotationConfig } from "@arizeai/ax-client";
const annotationConfig = await updateFreeformAnnotationConfig({
annotationConfig: "Notes",
space: "my-space",
name: "Notes v2",
});import { deleteAnnotationConfig } from "@arizeai/ax-client";
await deleteAnnotationConfig({
annotationConfig: "Accuracy",
space: "my-space",
});The @arizeai/ax-client package allows you to create and manage annotation queues, add records to them, and annotate or assign those records.
import { listAnnotationQueues } from "@arizeai/ax-client";
const annotationQueues = await listAnnotationQueues({ space: "my-space" });
console.log(annotationQueues);import { createAnnotationQueue } from "@arizeai/ax-client";
const queue = await createAnnotationQueue({
name: "Quality Review Queue",
spaceId: "your_space_id",
annotationConfigIds: ["ac_abc123"],
assignmentMethod: "ALL",
});import {
getAnnotationQueue,
updateAnnotationQueue,
deleteAnnotationQueue,
} from "@arizeai/ax-client";
const queue = await getAnnotationQueue({
annotationQueue: "my_queue",
space: "my-space",
});
await updateAnnotationQueue({
annotationQueue: "my_queue",
space: "my-space",
name: "Updated Queue Name",
});
await deleteAnnotationQueue({ annotationQueue: "my_queue", space: "my-space" });import {
addAnnotationQueueRecords,
listAnnotationQueueRecords,
deleteAnnotationQueueRecords,
annotateAnnotationQueueRecord,
assignAnnotationQueueRecord,
} from "@arizeai/ax-client";
// Add records to the queue from span and session sources
await addAnnotationQueueRecords({
annotationQueue: "my_queue",
space: "my-space",
recordSources: [
{
recordType: "SPAN",
projectId: "proj_abc123",
startTime: "2024-01-15T00:00:00Z",
endTime: "2024-01-15T23:59:59Z",
spanIds: ["span_abc123"],
},
{
recordType: "SESSION",
projectId: "proj_abc123",
startTime: "2024-01-15T00:00:00Z",
endTime: "2024-01-15T23:59:59Z",
sessionIds: ["session_abc123"],
},
],
});
// List records
const records = await listAnnotationQueueRecords({
annotationQueue: "my_queue",
space: "my-space",
});
// Assign a record to the current user
await assignAnnotationQueueRecord({
annotationQueue: "my_queue",
space: "my-space",
annotationQueueRecordId: "aqr_abc123",
});
// Submit annotations for a record
await annotateAnnotationQueueRecord({
annotationQueue: "my_queue",
space: "my-space",
annotationQueueRecordId: "aqr_abc123",
annotations: [
{ name: "accuracy", label: "correct", score: 1.0 },
{ name: "quality", text: "Well-structured response" },
],
});
// Remove records
await deleteAnnotationQueueRecords({
annotationQueue: "my_queue",
space: "my-space",
recordIds: ["aqr_abc123", "aqr_def456"],
});The @arizeai/ax-client package allows you to create and manage AI integrations (LLM provider credentials used by evaluators, tasks, and the playground).
import { listAiIntegrations } from "@arizeai/ax-client";
const integrations = await listAiIntegrations({ space: "my-space" });
console.log(integrations);import { createAiIntegration } from "@arizeai/ax-client";
const integration = await createAiIntegration({
name: "Production OpenAI",
provider: "OPEN_AI",
apiKey: "sk-...",
modelNames: ["gpt-4o", "gpt-4o-mini"],
enableDefaultModels: true,
});import { getAiIntegration } from "@arizeai/ax-client";
// By ID
const integration = await getAiIntegration({
integration: "your_integration_id",
});
// By name (requires space)
const byName = await getAiIntegration({
integration: "Production OpenAI",
space: "my-space",
});import { updateAiIntegration } from "@arizeai/ax-client";
const integration = await updateAiIntegration({
integration: "Production OpenAI",
space: "my-space",
name: "Updated OpenAI",
modelNames: ["gpt-4o"],
});import { deleteAiIntegration } from "@arizeai/ax-client";
await deleteAiIntegration({
integration: "Production OpenAI",
space: "my-space",
});The @arizeai/ax-client package allows you to create and manage integrations on the polymorphic /v2/integrations surface, covering both LLM (model-provider) and AGENT (customer-hosted endpoint) integrations. Integration names are unique per account and type, so type is required when resolving a name; an ID always works on its own. These functions are in alpha and emit a one-time pre-release warning.
type is an optional filter. Omit it to list integrations of every type in one call; each item carries its type discriminator.
import { listIntegrations } from "@arizeai/ax-client";
// Only agent integrations
const agents = await listIntegrations({ type: "AGENT", space: "my-space" });
// Every type, discriminated by `integration.type`
const all = await listIntegrations({});The type field selects the config shape.
import { createIntegration } from "@arizeai/ax-client";
// LLM integration
const llm = await createIntegration({
type: "LLM",
name: "Production OpenAI",
config: { provider: "OPEN_AI", apiKey: "sk-..." },
});
// Agent integration
const agent = await createIntegration({
type: "AGENT",
name: "My Support Agent",
config: {
endpoint: "https://agent.example.com/replay",
inputSchema: { type: "object", properties: { input: { type: "string" } } },
requestPresets: [{ name: "default", config: { input: "hello" } }],
},
});config.provider selects the LLM config shape. Secrets are write-only and never returned.
| Provider | Required fields | Optional fields |
|---|---|---|
OPEN_AI, ANTHROPIC, GEMINI |
apiKey |
isFunctionCallingEnabled |
AWS_BEDROCK |
auth |
modelNames, isDefaultModelsEnabled |
VERTEX_AI |
projectId, location, projectAccessLabel |
|
CUSTOM |
baseUrl |
apiKey, headers, modelNames, isDefaultModelsEnabled, isFunctionCallingEnabled |
NVIDIA_NIM |
none | baseUrl, apiKey, headers, modelNames, isDefaultModelsEnabled, isFunctionCallingEnabled |
LITELLM |
baseUrl, apiKey |
headers, modelNames, isFunctionCallingEnabled |
FIREWORKS, TOGETHER_AI |
apiKey |
modelNames, isDefaultModelsEnabled, isFunctionCallingEnabled |
// Self-hosted LiteLLM proxy: the endpoint and virtual key are both required.
const litellm = await createIntegration({
type: "LLM",
name: "Team LiteLLM",
config: {
provider: "LITELLM",
baseUrl: "https://litellm.internal:4000",
apiKey: "sk-...",
},
});
// Hosted providers such as Fireworks AI and Together AI need only the API key.
// Arize resolves the models the key can reach; list `modelNames` for
// fine-tunes or dedicated deployments that cannot be discovered.
const together = await createIntegration({
type: "LLM",
name: "Together AI",
config: { provider: "TOGETHER_AI", apiKey: "..." },
});import { getIntegration } from "@arizeai/ax-client";
// By ID
const integration = await getIntegration({
integration: "your_integration_id",
});
// By name (requires type)
const byName = await getIntegration({
integration: "Production OpenAI",
type: "LLM",
});type is required (it selects the update shape) and is immutable server-side. Provide at least one updatable field; omitted fields are preserved.
import { updateIntegration } from "@arizeai/ax-client";
const integration = await updateIntegration({
integration: "Production OpenAI",
type: "LLM",
config: { provider: "OPEN_AI", apiKey: "sk-new-key" },
});import { deleteIntegration } from "@arizeai/ax-client";
await deleteIntegration({
integration: "My Support Agent",
type: "AGENT",
});The @arizeai/ax-client package allows you to create and manage projects.
import { createProject } from "@arizeai/ax-client";
const project = await createProject({
space: "my-space",
name: "my-project",
});
console.log(project);import { getProject } from "@arizeai/ax-client";
const project = await getProject({
project: "my-project",
space: "my-space",
});
console.log(project);import { listProjects } from "@arizeai/ax-client";
const { projects } = await listProjects({ space: "my-space" });
console.log(projects);import { updateProject } from "@arizeai/ax-client";
const project = await updateProject({
project: "my-project",
space: "my-space",
name: "my-renamed-project",
});
console.log(project);import { deleteProject } from "@arizeai/ax-client";
await deleteProject({ project: "my-project", space: "my-space" });The @arizeai/ax-client package allows you to create, list, delete, revoke, and refresh API keys.
import { createApiKey } from "@arizeai/ax-client";
const apiKey = await createApiKey({ keyType: "USER", name: "CI pipeline key" });
// Store apiKey.key securely — the full key value is only returned once
console.log(apiKey.key);Service keys are backed by a dedicated bot user scoped to one or more organizations and spaces:
const apiKey = await createApiKey({
keyType: "SERVICE",
name: "service-key",
organizations: [
{
orgId: "T3JnMTIz",
spaces: [{ spaceId: "U3BhY2UxMjM", role: { name: "MEMBER" } }],
},
],
expiresAt: new Date("2027-01-01"),
});
// apiKey.botUser — the bot user created for this key, with resolved role assignmentsimport { listApiKeys } from "@arizeai/ax-client";
const { data } = await listApiKeys();
console.log(data.map((k) => k.name));
// Filter by key type or status
const { data: userKeys } = await listApiKeys({ keyType: "USER" });
// List service keys for a specific space
const { data: serviceKeys } = await listApiKeys({
keyType: "SERVICE",
spaceId: "U3BhY2U6MTIzNDU=",
});
// List service keys created by a specific user (any caller with space access)
const { data: byUser } = await listApiKeys({
spaceId: "U3BhY2U6MTIzNDU=",
userId: "VXNlcjoxMjM0NQ==",
});Sets the key's status to revoked and deactivates it immediately. This operation is irreversible; revoking an already-revoked key is a no-op and still succeeds.
import { revokeApiKey } from "@arizeai/ax-client";
await revokeApiKey({ apiKeyId: "your-api-key-id" });Atomically revokes an existing key and issues a replacement with the same metadata (name, description, key type). The old key is invalidated and the new key is activated in a single transaction.
import { refreshApiKey } from "@arizeai/ax-client";
const refreshed = await refreshApiKey({ apiKeyId: "your-api-key-id" });
// Store refreshed.key securely — the full key value is only returned once
console.log(refreshed.key);Supply expiresAt to set an expiration on the replacement key:
const refreshed = await refreshApiKey({
apiKeyId: "your-api-key-id",
expiresAt: new Date("2027-01-01"),
});Supply gracePeriodSeconds to keep the old key valid for a window after the refresh, giving your services time to adopt the new key before the old one is invalidated:
const refreshed = await refreshApiKey({
apiKeyId: "your-api-key-id",
gracePeriodSeconds: 3600, // old key stays valid for 1 hour
});
// Store refreshed.key securely — the full key value is only returned once
console.log(refreshed.key);The @arizeai/ax-client package allows you to create and manage custom roles.
import { createRole } from "@arizeai/ax-client";
const role = await createRole({
name: "AI Engineer",
permissions: ["PROJECT_READ", "DATASET_READ", "DATASET_CREATE"],
description: "Can read and create datasets and experiments.",
});import { listRoles } from "@arizeai/ax-client";
const { data } = await listRoles();
console.log(data.map((r) => r.name));
// Filter to only predefined (system) roles
const { data: predefined } = await listRoles({ isPredefined: true });import { getRole } from "@arizeai/ax-client";
const role = await getRole({ roleId: "your-role-id" });import { updateRole } from "@arizeai/ax-client";
const role = await updateRole({
roleId: "your-role-id",
permissions: ["PROJECT_READ", "DATASET_READ"],
});import { deleteRole } from "@arizeai/ax-client";
await deleteRole({ roleId: "your-role-id" });The @arizeai/ax-client package allows you to bind a role to a user on a specific resource (e.g. a project or space).
import { createRoleBinding } from "@arizeai/ax-client";
const binding = await createRoleBinding({
userId: "VXNlcjoxMjM0NQ==",
roleId: "Um9sZTphYmMxMjM=",
resourceType: "PROJECT",
resourceId: "UHJvamVjdDphYmMxMjM=",
});
console.log(binding);userId is the ID of the user to bind the role to. To bind a service key, pass its bot
user's ID instead of your own — it is returned as botUser.id from createApiKey (see
Creating an API key above).
import { listRoleBindings } from "@arizeai/ax-client";
const bindings = await listRoleBindings({ resourceType: "SPACE" });
console.log(bindings);import { getRoleBinding } from "@arizeai/ax-client";
const binding = await getRoleBinding({ bindingId: "Um9sZUJpbmRpbmc6YWJjMTIz" });import { updateRoleBinding } from "@arizeai/ax-client";
const binding = await updateRoleBinding({
bindingId: "Um9sZUJpbmRpbmc6YWJjMTIz",
roleId: "Um9sZTphYmMxMjM=",
});import { deleteRoleBinding } from "@arizeai/ax-client";
await deleteRoleBinding({ bindingId: "Um9sZUJpbmRpbmc6YWJjMTIz" });The @arizeai/ax-client package allows you to restrict access to a resource so that only explicitly bound users can access it, and to lift that restriction.
import { restrictResource, unrestrictResource } from "@arizeai/ax-client";
// Restrict a resource
const restriction = await restrictResource({ resourceId: "your_project_id" });
console.log(restriction);
// Lift the restriction
await unrestrictResource({ resourceId: "your_project_id" });The @arizeai/ax-client package allows you to manage organizations.
import { listOrganizations } from "@arizeai/ax-client";
const result = await listOrganizations();
console.log(result.organizations);import { getOrganization } from "@arizeai/ax-client";
const org = await getOrganization({ organization: "your-organization-name" });
console.log(org);import { createOrganization } from "@arizeai/ax-client";
const org = await createOrganization({
name: "your-organization-name",
description: "Optional description",
});
console.log(org);import { updateOrganization } from "@arizeai/ax-client";
const org = await updateOrganization({
organization: "your-organization-name",
name: "your-organization-name-updated",
});
console.log(org);Warning: This operation is irreversible and deletes the organization and all resources that belong to it, including all spaces and their contents.
import { deleteOrganization } from "@arizeai/ax-client";
await deleteOrganization({ organization: "your-organization-name" });import { addOrganizationUser } from "@arizeai/ax-client";
const membership = await addOrganizationUser({
organizationId: "org_abc123",
userId: "VXNlcjoxMjM0NQ==",
role: { type: "PREDEFINED", name: "MEMBER" },
});
console.log(membership);import { removeOrganizationUser } from "@arizeai/ax-client";
await removeOrganizationUser({
organizationId: "org_abc123",
userId: "VXNlcjoxMjM0NQ==",
});The @arizeai/ax-client package allows you to manage users.
Note: Unlike organizations, users are identified by opaque ID only — not by name. User display names are not unique within an account, so all functions require the user's ID.
import { listUsers } from "@arizeai/ax-client";
const result = await listUsers();
console.log(result.data);import { getUser } from "@arizeai/ax-client";
const user = await getUser({ userId: "VXNlcjoxMjM0NQ==" });
console.log(user);The inviteMode parameter controls how the user is invited:
"EMAIL_LINK"— sends the user an email with a verification link."TEMPORARY_PASSWORD"— issues a one-time password returned in the response."NONE"— pre-provisions an SSO user directly; no invitation is sent and the user is immediately active via the configured identity provider.
Returns a UserCreated object (HTTP 201) for a new user, or a User object
(HTTP 200) when an existing invited user is returned as-is (idempotent — the
invitation is not resent).
import { createUser } from "@arizeai/ax-client";
const user = await createUser({
name: "Jane Smith",
email: "jane.smith@example.com",
role: { type: "PREDEFINED", name: "MEMBER" },
inviteMode: "EMAIL_LINK",
});
console.log(user);import { updateUser } from "@arizeai/ax-client";
const user = await updateUser({
userId: "VXNlcjoxMjM0NQ==",
name: "Jane Smith Updated",
isDeveloper: true,
});
console.log(user);Warning: This operation permanently blocks the user from the account. Blocked users cannot be re-invited — inactive is a terminal state. The operation cascades to organization memberships, space memberships, API keys, and role bindings.
import { deleteUser } from "@arizeai/ax-client";
await deleteUser({ userId: "VXNlcjoxMjM0NQ==" });import { resendInvitation } from "@arizeai/ax-client";
await resendInvitation({ userId: "VXNlcjoxMjM0NQ==" });import { resetPassword } from "@arizeai/ax-client";
await resetPassword({ userId: "VXNlcjoxMjM0NQ==" });Delete multiple users by ID and/or email. Each entry's outcome is returned individually.
import { bulkDeleteUsers } from "@arizeai/ax-client";
const results = await bulkDeleteUsers({
userIds: ["VXNlcjoxMjM0NQ=="],
emails: ["jane.smith@example.com"],
});
for (const r of results) {
console.log(r.userId, r.status, r.error ?? "");
}The @arizeai/ax-client package allows you to create and manage spaces.
import { listSpaces } from "@arizeai/ax-client";
const spaces = await listSpaces();
console.log(spaces);import { getSpace } from "@arizeai/ax-client";
const space = await getSpace({ space: "my-space" });
console.log(space);import { createSpace } from "@arizeai/ax-client";
const space = await createSpace({
organizationId: "T3JnYW5pemF0aW9uOmFiYzEyMw==",
name: "your_space_name",
isPrivate: true, // omit or set false for a public space (default)
});Private spaces are visible only to their members and account/org/space admins.
A warning is logged when isPrivate: true to remind you to add members before
the space becomes inaccessible to other users.
import { updateSpace } from "@arizeai/ax-client";
const space = await updateSpace({
space: "my-space",
name: "updated_space_name",
isPrivate: false, // omit to preserve the current visibility
});import { deleteSpace } from "@arizeai/ax-client";
await deleteSpace({ space: "my-space" });import { addSpaceUser } from "@arizeai/ax-client";
const membership = await addSpaceUser({
spaceId: "spc_abc123",
userId: "VXNlcjoxMjM0NQ==",
role: { type: "PREDEFINED", name: "MEMBER" },
});
console.log(membership);import { removeSpaceUser } from "@arizeai/ax-client";
await removeSpaceUser({
spaceId: "spc_abc123",
userId: "VXNlcjoxMjM0NQ==",
});Webhooks are alpha: every call logs a one-time warning and the API may change. A webhook is an organization-owned HTTPS destination. Pass a webhook ID, or a name together with organization.
import { listWebhooks } from "@arizeai/ax-client";
const webhooks = await listWebhooks({ organization: "my-org" });
console.log(webhooks.data);import { createWebhook } from "@arizeai/ax-client";
const webhook = await createWebhook({
organization: "my-org",
name: "deploy-notifier",
url: "https://example.com/hooks/arize",
authType: "HMAC_SHA256",
});
// Only returned here. Store it now; afterwards only signingSecretHint is readable.
console.log(webhook.signingSecret);authToken and headers are write-only and never come back in any response.
import {
getWebhook,
updateWebhook,
testWebhook,
deleteWebhook,
} from "@arizeai/ax-client";
const webhook = await getWebhook({
webhook: "deploy-notifier",
organization: "my-org",
});
// Omitted fields keep their value; pass `description: null` to clear it.
await updateWebhook({
webhook: "deploy-notifier",
organization: "my-org",
timeoutMs: 10000,
});
const outcome = await testWebhook({
webhook: "deploy-notifier",
organization: "my-org",
});
console.log(outcome.statusCode, outcome.errorMessage);
await deleteWebhook({ webhook: "deploy-notifier", organization: "my-org" });import { listWebhookDeliveryAttempts } from "@arizeai/ax-client";
const attempts = await listWebhookDeliveryAttempts({
webhook: "deploy-notifier",
organization: "my-org",
limit: 100, // up to 500
});
console.log(attempts.data);A subscription delivers one event from one prompt or evaluator to one webhook. Subscriptions have no name and are addressed by ID.
import { createWebhookSubscription } from "@arizeai/ax-client";
const subscription = await createWebhookSubscription({
webhook: "deploy-notifier",
organization: "my-org",
sourceType: "PROMPT",
sourceId: "your_prompt_id",
event: "PROMPT_VERSION_CREATED",
});import {
listWebhookSubscriptions,
getWebhookSubscription,
deleteWebhookSubscription,
} from "@arizeai/ax-client";
// sourceType and sourceId go together; omit both to list every readable subscription.
const subscriptions = await listWebhookSubscriptions({
sourceType: "PROMPT",
sourceId: "your_prompt_id",
});
const subscription = await getWebhookSubscription({
subscriptionId: "your_subscription_id",
});
await deleteWebhookSubscription({ subscriptionId: "your_subscription_id" });It is recommended to use the methods in this package. If more control is desired, you can use the client directly. The client provides a type-safe fetch for the entire Arize AX REST API.
import { createClient } from "@arizeai/ax-client";
const client = createClient();
// Get all datasets
const response = await client.GET("/v2/datasets");