UmamiAI Platform user guide

Step-by-step guide for using the entire platform

This guide walks end users through the full UmamiAI lifecycle: organization setup, agent and workflow configuration, knowledge + tools, run monitoring, approvals, team collaboration, API keys, webhooks, and day-to-day operations.

Last updated: February 25, 2026

Start here

If you are brand new to the platform, follow this exact sequence first.

  1. 1
    Create or access your account

    Use Sign up to create a new account, or Sign in if your admin already invited you. If you received an invite link, open it first to attach your membership to the correct organization.

  2. 2
    Complete onboarding

    Create your organization (or confirm the invited org), then create your first agent so the platform can route you into the main console experience.

  3. 3
    Verify access to key sections

    Open Dashboard, Agents, Runs, and Approvals. Owners/Admins should also verify Team, API Keys, and Webhooks are visible and working.

  4. 4
    Pick your first success path

    For business users: create and run an agent. For technical users: create an API key and test the Platform API. For operations users: configure webhooks and approvals.

Roles and permissions

Understanding access levels helps prevent confusion during setup and approval workflows.

Ownerhighest access
  • Manage org settings and members
  • Create/revoke API keys and webhooks
  • Approve/reject sensitive actions
  • Configure agents, tools, and workflows
Adminoperations + config
  • Most configuration and operational actions
  • Can invite members (org policy dependent)
  • Common choice for platform operators
  • Typically allowed to manage API/webhooks
Memberday-to-day usage
  • Run agents and workflows
  • View resources permitted by org policies
  • Can participate in operational review
  • Less access to sensitive configuration

Onboarding

The onboarding flow is designed to get you to a usable console quickly.

  1. 1
    Step 1 — Organization

    Enter a clear organization name. Use a stable naming convention (for example, “Acme Operations” instead of temporary project nicknames) so audits, exports, and team invites remain easy to track later.

  2. 2
    Step 2 — First agent

    Create a first agent with a simple, testable purpose such as summarization, triage, extraction, or QA. Keep the first version focused; you can add tools and workflow orchestration after the baseline run works.

  3. 3
    Step 3 — Ready

    You will land on the Dashboard. From here, continue into Agents to refine the agent, or into Team/API Keys if another teammate or external integration needs access.

Dashboard

Use the dashboard as your operational overview before drilling into detailed pages.

  1. 1
    Review activity and status cards

    Check high-level metrics and recent activity to confirm your org is active and no queue/backlog issue is preventing runs from completing.

  2. 2
    Jump to the next action

    Use the dashboard links to open Agents, Runs, Workflows, Approvals, or Team depending on your current objective.

  3. 3
    Use dashboard as a daily checklist

    Many teams begin each day by scanning pending approvals, failed runs, and webhook delivery status before making changes to agents or workflows.

Agents and versions

Agents are versioned execution units. Use versions to keep behavior reproducible while improving prompts and tool access over time.

Create your first production-ready agent
  1. 1
    Open Org → Agents and create a new agent

    Give it a business-readable name (for example: “Sales Weekly Summarizer” or “Invoice Triage Agent”).

  2. 2
    Add instructions with hard boundaries

    Be explicit about what the agent should do, what outputs are expected, what it must not do, and when it should request approval.

  3. 3
    Attach tools only when needed

    Start with the minimum set of tools. Add additional tools after you have validated the baseline prompt behavior and run outputs.

  4. 4
    Save and run a test

    Use realistic sample input (not placeholder text) so you can validate formatting, accuracy, and runtime behavior under real constraints.

Versioning workflow (recommended)
  1. Create v1 with a narrow task and no risky external actions.
  2. Test run output quality and edge cases.
  3. Create a new version when changing prompts, model settings, or tool access.
  4. Document what changed (purpose, risk, expected output differences).
  5. Promote the version only after reviewers confirm expected behavior.
Tip: Keep reusable format instructions inside the agent version, and pass task-specific context in each run payload.

Knowledge (RAG)

Knowledge pages let you manage documents and retrieval inputs used by agents and workflows.

  1. 1
    Open Org → Knowledge

    Review documents already added to your org. Confirm the source set is correct before relying on retrieval in production agents.

  2. 2
    Add or update documents

    Upload source material or connect content according to your org process. Use clear titles and ownership tags so operators can identify stale documents quickly.

  3. 3
    Validate retrieval relevance

    Run a knowledge search or a test run using representative queries. Confirm the returned chunks are accurate, current, and safe to use in downstream outputs.

  4. 4
    Retire stale knowledge

    Remove, replace, or supersede old documents when business rules change to avoid incorrect answers and accidental policy drift.

Tools

Tools extend agents with structured actions (e.g., HTTP calls, external systems). Treat tool setup like production integration work.

  1. 1
    Open Org → Tools and create a tool

    Define the tool name and purpose so reviewers understand exactly what action the agent may take using this integration.

  2. 2
    Configure endpoint/method/schema

    Provide the request details (URL, method, headers, payload schema) and validate the expected inputs/outputs before enabling the tool on agents.

  3. 3
    Store secrets securely

    Never paste credentials into prompts. Use the platform’s tool secret handling so tokens and keys are encrypted and not shown in normal UI views.

  4. 4
    Apply approvals for risky actions

    If the tool can mutate external systems, enable approvals so humans review intent before execution completes.

Runs

Runs are your primary execution record for prompts, tool calls, outputs, and debugging.

Create and monitor a run
  1. 1
    Open Org → Runs and start a new run

    Select the target agent and provide the input payload or prompt. Use realistic production-like examples for testing.

  2. 2
    Track run status

    Watch the run move through statuses as steps execute. If a tool requires approval, the run may pause until a reviewer acts.

  3. 3
    Inspect run steps and outputs

    Open the run detail to inspect each step, payloads, tool interactions, and final output for validation or debugging.

When a run fails
  1. Check if the run is waiting on an approval.
  2. Review the tool request/response and input payload.
  3. Confirm the relevant tool secret and endpoint are valid.
  4. Verify required knowledge documents/chunks exist.
  5. Re-run with narrower input to isolate the issue.

Workflows

Workflows coordinate multi-step execution. Use them for repeatable, auditable automation that spans prompts and tools.

  1. 1
    Open Org → Workflows and create a workflow

    Name the workflow by business outcome (for example: “Weekly pipeline digest + anomaly review”) and define a clear owner for operational accountability.

  2. 2
    Add steps in execution order

    Each step can call an agent or tool. Keep each step focused and clearly named so approvals and logs are easy to interpret later.

  3. 3
    Reference earlier outputs

    Pass context from previous steps into later steps using template references. This keeps workflows deterministic and reduces manual copying between runs.

  4. 4
    Test with realistic payloads

    Run the workflow with representative input and validate intermediate outputs, approval pauses, and final artifacts before broader rollout.

Step reference syntax

Use a template token like this inside later steps to reference the output of an earlier step.

{{step.N}}   # Example: {{step.1}}

Approvals

Approvals are the human control plane for sensitive or high-impact actions.

  1. 1
    Open Org → Approvals

    Use filters to focus on pending items, workflow-only items, or agent-run items depending on your triage responsibilities.

  2. 2
    Sort by step when reviewing workflows

    Use Step ↑ or Step ↓ to review workflow approvals in execution order. This helps reviewers understand context before approving downstream actions.

  3. 3
    Open details and verify intent

    Inspect the action payload, related run/workflow, and any tool context. Confirm the requested action matches policy and expected business logic.

  4. 4
    Approve or reject

    Submit the decision. The page preserves your filters/sort settings so you can continue triaging without losing context.

Team and invites

Use the Team page to scale access safely while preserving role boundaries.

Invite a teammate
  1. 1
    Go to Org → Team

    Review current members and pending invites before creating a new invite.

  2. 2
    Create invite

    Enter the teammate email and choose the minimum role they need right now. You can elevate later if required.

  3. 3
    Track invite acceptance

    Pending invites remain visible so admins can resend, revoke, or confirm whether the user joined.

Recommended access pattern
  • Keep only a small set of owners/admins.
  • Use member role for most day-to-day operators.
  • Review memberships regularly during audits.
  • Revoke access immediately when responsibilities change.

API keys

API keys let external systems call your UmamiAI Platform API securely.

  1. 1
    Open Org → API Keys

    Only owners/admins should create keys. Review existing keys and scopes to avoid issuing duplicates for the same integration.

  2. 2
    Create a scoped key

    Provide a name that identifies the integration (for example, “crm-sync-prod”), choose the minimum scopes, and create the key.

  3. 3
    Copy the plaintext key once

    The full key is shown only at creation time. Store it immediately in a secret manager, CI/CD secret store, or runtime environment variable.

  4. 4
    Test with the Platform API

    Use the docs and developer guide curl examples to verify auth, create a run, fetch results, and confirm your integration path before production rollout.

# Example API auth header
curl -sS "$UMAMIAI_BASE_URL/api/v1/agent-runs/$RUN_ID" \
  -H "Authorization: Bearer umami_..."

Webhooks

Webhooks notify your systems when events happen in the platform (for example run completion or approvals).

  1. 1
    Open Org → Webhooks and add an endpoint

    Provide a reachable HTTPS endpoint in your system. Use separate endpoints for staging and production where possible.

  2. 2
    Store the signing secret

    Copy the webhook signing secret and store it securely. Your receiver will use it to verify each delivery signature.

  3. 3
    Implement signature verification

    Validate the delivery using the signature header before trusting the payload. Reject requests that fail verification or come from unexpected sources.

  4. 4
    Test and monitor deliveries

    Trigger a platform event, inspect webhook delivery attempts, and verify your receiver handles retries and idempotency safely.

Typical delivery headers include X-Umami-Event, X-Umami-Delivery, and X-Umami-Signature.

Audit and operations

Use the platform’s records to support troubleshooting, governance, and operational review.

Operational review checklist
  • Pending approvals are triaged and not stale
  • Failed runs are investigated and re-tested
  • Webhook deliveries are succeeding / retried safely
  • API keys are scoped correctly and old keys revoked
  • Knowledge sources are current and owned
When preparing for an audit or handoff
  • Document who owns each agent and workflow
  • List critical tools and their approval requirements
  • Review team membership and least-privilege roles
  • Export / capture evidence from runs and approvals
  • Capture current integration endpoints and key rotations

Best practices for a production-ready setup

These practices reduce incidents and speed up onboarding for new teammates.

Design patterns
  • Start narrow: one agent, one workflow, one clear outcome.
  • Version all meaningful changes to prompts/tools.
  • Prefer structured outputs where downstream systems depend on parsing.
  • Add approvals before enabling destructive or external mutations.
Operations patterns
  • Use environment-specific webhooks and API keys.
  • Rotate keys regularly and when staff changes occur.
  • Keep knowledge fresh; retire stale documents explicitly.
  • Use pending approval queues as a daily ops ritual.

Troubleshooting and common issues

Use these quick checks before escalating to your platform owner or developer team.

I cannot see API Keys or Webhooks pages

Your role may not have permission. Confirm you are an owner/admin in the current organization and that you are in the correct org (top-level console route often includes the org ID).

A run is stuck or paused

Check the Approvals queue first. If no approval is pending, open the run and inspect the last step/tool response for missing inputs, bad secrets, or endpoint failures.

Webhook events are not arriving

Confirm the endpoint URL is correct, publicly reachable, and returns a success status. Review delivery logs and signature verification logic. Ensure your receiver handles retries idempotently.

Agent output quality changed unexpectedly

Check whether the agent version changed, a tool configuration changed, or knowledge documents were updated. Compare recent runs with known-good runs to isolate the difference.

Continue with technical integration

After your operators understand the platform, use the Platform API documentation and integration guide to connect external systems using scoped API keys and signed webhooks.