Monitoring
The traces and metrics an instance can send to a collector.
Describes Ronne AI Marketplace 0.4.0.
Monitoring
The health endpoint: GET /api/health answers 200 when Ronne can serve, with each check:
{"status":"ok","checks":{"database":"ok","storage":"ok"}}and 503 when it can't, in the API's error shape: setup_required before setup, database_unavailable when the database doesn't answer within 3 seconds, storage_unavailable when the storage folder can't take a write (its details.checks says which). Docker's health check, the services and the install scripts wait on it, and a load balancer can too. It says nothing else (no version, path or timing): it's public. The reason a check failed is in the log (db.unavailable, storage.unavailable).
Metrics and traces go to an OpenTelemetry collector you run, over OTLP/HTTP. They're off unless you set its address: then, and only then, Ronne loads OpenTelemetry. Set it in the settings file or the environment (with Docker, in the .env file next to compose.yaml, then docker compose up -d), and restart Ronne:
OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4318 OTEL_EXPORTER_OTLP_HEADERS=authorization=Bearer%20… # optional, the collector's OTEL_SERVICE_NAME=ronne-marketplace # optional; this is the default OTEL_RESOURCE_ATTRIBUTES=deployment.environment=production
OpenTelemetry's standard OTEL_* variables work as it documents them: a signal's own endpoint (OTEL_EXPORTER_OTLP_TRACES_ENDPOINT, …_METRICS_ENDPOINT), the sampler (OTEL_TRACES_SAMPLER), the export interval (OTEL_METRIC_EXPORT_INTERVAL, 60 s by default), and OTEL_EXPORTER_OTLP_PROTOCOL=http/json to send traces as JSON instead of protobuf (metrics always go as JSON). OTEL_SDK_DISABLED=true turns it off with an address set. Next.js's own NEXT_OTEL_VERBOSE is turned off: its extra spans would carry the filled-in paths, private names included. A collector that's down or slow never slows a request: exports retry in the background and are then dropped, with one telemetry.export_failed line a minute at most.
A collector that takes them, keeps traces in its own log and offers the metrics to Prometheus at :8889/metrics (otel-collector.yaml):
receivers:
otlp:
protocols:
http:
endpoint: 0.0.0.0:4318
processors:
batch: {}
exporters:
# Metrics for Prometheus to scrape, at http://<collector>:8889/metrics.
prometheus:
endpoint: 0.0.0.0:8889
# Traces to the collector's own log: replace with your tracing backend's exporter.
debug:
verbosity: basic
service:
pipelines:
traces:
receivers: [otlp]
processors: [batch]
exporters: [debug]
metrics:
receivers: [otlp]
processors: [batch]
exporters: [prometheus]
docker run -p 4318:4318 -p 8889:8889 \ -v ./otel-collector.yaml:/etc/otelcol-contrib/config.yaml:ro \ ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-contrib:0.162.0
The metrics, each with these attributes only:
| Metric | Kind | Attributes |
|---|---|---|
http.server.request.duration | histogram, seconds | http.request.method, http.route, http.response.status_code |
ronne.db.query.duration | histogram, seconds | db.system.name, db.operation.name |
ronne.sign_ins | counter | outcome (success, failure, rate_limited) |
ronne.tokens.exchanged | counter | outcome (success, failure, rate_limited) |
ronne.releases | counter | |
ronne.feed.builds | counter | tool, cached |
ronne.feed.build.duration | histogram, seconds | tool |
ronne.job.duration | histogram, seconds | job, result (ok, failed) |
ronne.drafts.uploaded | counter | outcome (accepted, refused) |
ronne.rate_limited | counter | limit (sign_in, upload, usage, token_exchange) |
ronne.health | gauge, 1 or 0 | check (database, storage) |
The traces: Next.js's own spans for each request, named by the route's pattern; a span for each database query, with its SQL and its placeholders, never the values; storage.put and storage.get with their byte count; and ronne.release, ronne.feed.build and ronne.draft.upload. An API route's span carries ronne.request_id, the X-Request-Id its answer had.
Never sent: user ids, emails, people's IP addresses, user agents, tokens, a query's values, or the names of items, scopes and workspaces. A request's path and URL aren't sent (an item page's path names the item), nor an error's message. Spans carry only the attributes Ronne lists; any other a library adds is dropped. Each request is its own trace: a caller's traceparent, tracestate and baggage are ignored, so a trace never joins a proxy's or a client's. The resource says service.name, service.version, deployment.* and what you set in OTEL_RESOURCE_ATTRIBUTES. Nothing goes anywhere but your collector.
Alerts worth having: the health check failing (ronne.health at 0, or the endpoint's 503); a rise in 5xx answers (http.server.request.duration by http.response.status_code); a rise in failed or limited sign-ins (ronne.sign_ins, ronne.rate_limited); slow queries (ronne.db.query.duration).
Logs and traces together: with tracing on, each log line written during a request carries traceId and spanId (Logs), so a line leads to its trace, and a trace's ronne.request_id to the lines.