Connecting to the inference metrics service…
Recent host utilization
CPU · Memory · GPU · Disk. Uncollected measurements are unavailable, not zero.
Current resource mix
ML observability
ML inference
Runtime measurements only. No accuracy score is claimed without labeled ground truth.
Request outcomes
Latency · recent generations
Token throughput
Service health
About this page
Metrics come only from the active self-hosted inference gateway. No prompts, answers, identities, IP addresses, credentials, filenames, device identifiers or private network addresses are published. This page does not generate AI requests.
Refreshes every 30 seconds while visible. Counters reset when the gateway restarts; latency and token statistics cover the latest 512 generations, and resource history covers up to 288 samples collected every 15 seconds. Requests made directly to the model API bypass gateway inference counters.