Self-hosted AI operations

FlexInferPrivate AI, operated like real infrastructure

Run and customize open models on your own GPUs, behind an OpenAI-compatible gateway. Loom governs how agents reach tools and context, MentatLab turns model calls into observable DAG workflows, and fi-fhir handles healthcare parsing, mapping, and delivery.

OpenAI-compatible APIGPU-aware, scale-to-zeroMCP-governed agent access

Explore FlexInfer Open playground

Platform Docs

Built and operated by Cody Blevins. Selected work

Fleet pulse

What this site is wired to, right now.

3 of 4 live· 4 min ago

Last mergedby Mills · titles sanitized

Live systems

Three systems you can read right now

Each row reads the same sanitized public API as the page it links to, and says whether its number is live or a saved snapshot.

Open the systems hub

Factory
The Mills delivery pipeline: every run's stages, gates, retries, and cost over the last 30 days.
420merged of 869 runs
Live
Trace factory runs
Inference plane
Open models on self-hosted GPUs behind an OpenAI-compatible gateway, with the engine and hardware of each one that is warm.
4warm of 20 models
Live
Open the inference plane
Cluster
The K3s cluster that serves this site: node readiness, GPUs, and the health score FlexDeck computes from them.
15/15nodes ready
Live
Open the cluster view

Measured on the clustersnapshot

Verified FlexInfer benchmark results

Serving throughput across language, embedding, and image workloads. The live feed takes over when available; otherwise this shows the latest verified cluster snapshot.

Newest record Sep 2, 2026

142.1tok/s recorded peak

FlexInfer docs

llama.cppRadeon VII

qwen3-1p7b-tools-radeonvii

23.8tok/s

10 samples320 tok

vllmRadeon RX 7900 XTX

gemma4-26b-a4b-gptq-5930k

142.1tok/s

49 samples3136 tok

vllmRadeon RX 7900 XTX

gemma4-26b-a4b-gptq

139.3tok/s

47 samples3008 tok

vllmcandidate

qwen35-35b-clean-gptq-64k-graph-candidate

79.4tok/s

21 samples672 tok

vllm

qwen3-14b-abliterated-v2

74.9tok/s

6 samples768 tok

teiCPU

BAAI/bge-large-en-v1.5

203.3emb/s

2 GiB memory

Snapshot scope. Each card is one run of the FlexInfer benchmarker against a deployed model: throughput, sample and token counts, and the GPU that deployment is pinned to. Candidates are tuning experiments, not serving lanes. Records from the older benchmarker keep only backend, hardware class, memory, and throughput. Treat these as operational baselines, not a cross-vendor leaderboard.

Inspect the source records

Platform posture

Four products, one operating boundary

The stack is intentionally layered: run and customize models inside your boundary, govern how agents reach tools and context, orchestrate repeatable DAG workflows over direct API models, and use fi-fhir when the workload is sensitive healthcare ETL instead of generic demo data.

Explore Platform

Private inference and model customization

FlexInfer

Kubernetes-native model lifecycle, OpenAI-compatible routing, GPU-aware runtime controls, quantization, adapters, and artifact delivery for private or hybrid inference.

Deployment:: Model runtime placement, scheduling, caching, activation, and adapter workflows stay inside your cluster boundary.
Integration:: Applications hit standard inference APIs while runtime operations stay inside your network and observability stack.

Docs Playground Product

MCP, context, and agent fleet coordination

Loom

Loom plus Loom Core coordinate MCP servers, editor config, agent sessions, tiered context, Weaver routing, sandbox execution, Mills autonomous delivery, RBAC, audit, HUD, and mobile fleet visibility.

Deployment:: Centralizes MCP server lifecycle, daemon routing, context memory, and policy boundaries for internal tools and agent access.
Integration:: Defines how agents reach internal systems with bounded context, auditable routing, and least-privilege intent.

Docs Playground Product

DAG agent orchestration

MentatLab

Mission Control for DAG-based agent workflows over direct API model calls, internal services, and private inference endpoints such as FlexInfer-hosted models.

Deployment:: UI, gateway, and orchestrator services run inside your Kubernetes footprint alongside private model and integration services.
Integration:: Workflows call direct model/API nodes and MCP-governed tools without depending on frontier coding harness semantics.

Docs Playground Product

Healthcare ingestion and transformation

fi-fhir

Healthcare-focused ingestion and transformation workflows across HL7v2, CSV, EDI X12, and CDA/CCDA with profile-driven, testable data handling.

Deployment:: Data transformation pipeline runs in your controlled environment and deployment topology.
Integration:: Source Profiles, semantic events, validation, routing, and terminology mapping isolate source variability while preserving operational traceability.

Docs Playground Product

Integration points

How the products connect

Use these contracts to map deployment and integration boundaries before implementation. The platform page carries the full boundary detail for each pair.

View platform architecture

Runtime and context boundary

FlexInfer + Loom

FlexInfer runs private model workloads in-cluster while Loom governs MCP routing, context memory, policy boundaries, and agent/tool access.

FlexInfer docs Loom Core docs

Control plane and operator UX

Loom + MentatLab

Loom provides policy-governed context and tool routing while MentatLab delivers mission-control UX for DAG design, execution planning, and run visibility.

MentatLab product Loom Core product

Inference and sensitive-data integration

FlexInfer + fi-fhir

fi-fhir handles profile-driven healthcare transformation while FlexInfer provides private runtime execution for downstream model workflows and AI-assisted integration development.

fi-fhir docs fi-fhir playground

Core stack

Loom suite, FlexInfer, fi-fhir, and MentatLab

FlexInfer runs the models, Loom governs context and policy, MentatLab gives operators the DAG view, and fi-fhir carries the healthcare workload. Product pages explain what each component does, docs explain the interfaces, and the playground checks configuration.

Agent and DAG orchestration UI

MentatLab mission control

Mission Control interface for building, validating, monitoring, and executing DAG-based agent workflows in private environments.

Loom Core governs context routing and policy boundaries. MentatLab provides the DAG design and run-visibility layer over direct API model calls, internal services, and private FlexInfer endpoints.

Product page Docs Code

Mobile fleet monitoring (iOS/iPad)

Loom Companion

SwiftUI app for fleet monitoring, session management, real-time alerts, and lightweight operator control from iPhone and iPad.

Loom Core exposes a frozen v1 mobile API (18 endpoints) with OAuth PKCE auth; Companion consumes it for on-the-go visibility into agents, sessions, and infrastructure.

Product page Docs Code

Component directory

FlexInfer
Inference control plane for private and hybrid workloads: model lifecycle, serverless activation, routing, and GPU-aware placement.
Docs Playground Product
Loom Core
Single-entry MCP and context control plane: registry sync, tiered memory, Weaver orchestration, proxy + daemon routing, and controlled execution.
Docs Playground Product
Loom
VS Code extension for multi-platform MCP config sync, skills registry, and agent context workflows.
Docs Playground
Loom Zed
Zed integration for Loom workflows: consistent servers, shared registry, and repeatable profiles.
Docs
fi-fhir
Healthcare integration tooling: Source Profiles, parsing pipeline, and HL7v2 to FHIR workflows.
Docs Playground Product

Enterprise capabilities

Operator controls already shipping across the platform stack

MCP routing, role-based access control, sandbox execution, and tracing. 4 controls are available today and 2 are in progress; each card carries its status.

FlexInfer product Capability map

MCP Gateway

Available

Centralized MCP routing via loom proxy with streamable HTTP transport, bearer/OIDC/mTLS auth, and hub failover.

Single context ingress with controlled routing, auditing, and automatic local fallback.

Role-based Access Control (RBAC)

Available

Role-aware permissions for MCP tool access with audit trail, cost tracking, and OAuth 2.1.

Enforces least-privilege context access with auditable decision logs across teams and environments.

Sandbox Executor (Docker + K8s)

Available

mcp-devbox sandbox runtime with Docker and Kubernetes backends for isolated agent execution.

Runs builds, tests, and automation in controlled containers with consistent isolation and audit trails.

Operational Foundations

Available

OpenTelemetry tracing across the MCP server fleet, JSON log correlation, observability stack, and deployment controls.

Distributed tracing, structured logging, and deployment procedures for investigating runtime behavior.

HUD Cost Dashboard

In progress

Cost monitoring integration: loom/cost-stats RPC, CostMonitor polling, SSE events, and OverviewPanel KPI tile.

HUD RBAC + Audit Visibility

In progress

RBAC config RPC, denied-calls ring buffer, ServersPanel RBAC sub-tab, and OverviewPanel badge.

Hands-on

Try it in the browser

The playground checks configuration against documented schemas. Labs and the project library show the systems behind the products.

Playground

Check a configuration

Schema-backed config editing, CLI generation, and sample transformations. Runtime behavior, permissions, and capacity still need checks in your deployment.

Open playground

Systems