095b1aa360
* [WEB-7447] feat: migrate CE telemetry from OTLP traces to OTLP metrics Replace span-based tracing (tracer.py) with OTLP observable gauges, mirroring the approach already used in plane-ee. Key changes: - Add otlp_endpoints.py — shared gRPC/HTTP endpoint helpers - Add telemetry_metrics.py — push_instance_metrics task using MeterProvider + observable gauges (service name: plane-ce-api) - User count excludes bots (is_bot=False) - Page count excludes bot-owned private pages only - Domain derived from WEB_URL env var - Celery beat entry replaced with timedelta schedule + configurable METRICS_PUSH_INTERVAL_MINUTES (default 360 min) - Add explicit opentelemetry-exporter-otlp-proto-grpc dep - Delete tracer.py and telemetry.py (no longer needed) Co-authored-by: Plane AI <noreply@plane.so> * fix: address review comments on CE telemetry metrics - harden grpc_endpoint_from_url for scheme-less OTLP_ENDPOINT values (e.g. "telemetry.plane.so:4317") by prepending "//" before urlparse - fix WEB_URL domain extraction for scheme-less values with same approach - replace N+1 workspace count queries (6×N) with 6 batched annotate(Count) aggregation queries — reduces DB load significantly at WORKSPACE_METRICS_LIMIT - add deterministic ordering (order_by created_at) to workspace slice - harden METRICS_PUSH_INTERVAL_MINUTES env parsing with try/except guard and positive-value validation to avoid crash on malformed input Co-authored-by: Plane AI <noreply@plane.so> * fix: cap METRICS_PUSH_INTERVAL_MINUTES to prevent timedelta overflow Add upper-bound check (10_000_000 minutes) and catch OverflowError alongside ValueError so an arbitrarily large env value cannot crash worker startup via timedelta(minutes=...) OverflowError. Co-authored-by: Plane AI <noreply@plane.so> --------- Co-authored-by: Plane AI <noreply@plane.so>
59 lines
2.1 KiB
Python
59 lines
2.1 KiB
Python
# Copyright (c) 2023-present Plane Software, Inc. and contributors
|
|
# SPDX-License-Identifier: AGPL-3.0-only
|
|
# See the LICENSE file for details.
|
|
|
|
"""
|
|
Shared OTLP endpoint helpers so metrics and traces use the same collector
|
|
when both are enabled. One URL (OTLP_ENDPOINT) is enough: same as traces
|
|
(e.g. https://telemetry.plane.so or https://telemetry.plane.town behind
|
|
nginx ingress with gRPC backend).
|
|
"""
|
|
|
|
import os
|
|
from urllib.parse import urlparse
|
|
|
|
# When no port in URL: https -> 443 (ingress), http -> 4317 (OTLP gRPC default)
|
|
OTLP_GRPC_DEFAULT_PORT = "4317"
|
|
HTTPS_DEFAULT_PORT = "443"
|
|
|
|
_DEFAULT_OTLP_ENDPOINT = "https://telemetry.plane.so"
|
|
|
|
|
|
def grpc_endpoint_from_url(url: str) -> str:
|
|
"""
|
|
Derive gRPC host:port from OTLP_ENDPOINT URL.
|
|
- https://telemetry.plane.so -> telemetry.plane.so:443 (nginx ingress)
|
|
- https://telemetry.plane.town -> telemetry.plane.town:443 (dev)
|
|
- telemetry.plane.so:4317 -> telemetry.plane.so:4317 (scheme-less with port)
|
|
- telemetry.plane.so -> telemetry.plane.so:4317 (scheme-less, default gRPC port)
|
|
- Explicit port in URL is always preserved.
|
|
"""
|
|
# urlparse needs a scheme to correctly populate hostname/netloc.
|
|
# Scheme-less values like "host:port" are misread as scheme="host", path="port".
|
|
if "://" not in url:
|
|
url = "//" + url
|
|
parsed = urlparse(url)
|
|
host = parsed.hostname or "telemetry.plane.so"
|
|
if parsed.port is not None:
|
|
port = str(parsed.port)
|
|
elif parsed.scheme == "https":
|
|
port = HTTPS_DEFAULT_PORT
|
|
else:
|
|
port = OTLP_GRPC_DEFAULT_PORT
|
|
return f"{host}:{port}"
|
|
|
|
|
|
def get_otlp_grpc_endpoint() -> str:
|
|
"""
|
|
Return the gRPC endpoint (host:port) for OTLP traces and metrics.
|
|
Derived from OTLP_ENDPOINT so the same URL works for both (e.g. collector
|
|
behind nginx ingress with gRPC backend on 443).
|
|
"""
|
|
base = os.environ.get("OTLP_ENDPOINT", _DEFAULT_OTLP_ENDPOINT)
|
|
return grpc_endpoint_from_url(base)
|
|
|
|
|
|
def get_otlp_http_metrics_url() -> str:
|
|
"""Return the HTTP URL for OTLP metrics (OTLP_ENDPOINT + /v1/metrics)."""
|
|
base = os.environ.get("OTLP_ENDPOINT", _DEFAULT_OTLP_ENDPOINT)
|
|
return f"{base.rstrip('/')}/v1/metrics"
|