Commit graph

7 commits

Author SHA1 Message Date
Manish Gupta
095b1aa360
[WEB-7447] feat: migrate CE telemetry from OTLP traces to OTLP metrics (#9156)
* [WEB-7447] feat: migrate CE telemetry from OTLP traces to OTLP metrics

Replace span-based tracing (tracer.py) with OTLP observable gauges,
mirroring the approach already used in plane-ee. Key changes:

- Add otlp_endpoints.py — shared gRPC/HTTP endpoint helpers
- Add telemetry_metrics.py — push_instance_metrics task using
  MeterProvider + observable gauges (service name: plane-ce-api)
- User count excludes bots (is_bot=False)
- Page count excludes bot-owned private pages only
- Domain derived from WEB_URL env var
- Celery beat entry replaced with timedelta schedule +
  configurable METRICS_PUSH_INTERVAL_MINUTES (default 360 min)
- Add explicit opentelemetry-exporter-otlp-proto-grpc dep
- Delete tracer.py and telemetry.py (no longer needed)

Co-authored-by: Plane AI <noreply@plane.so>

* fix: address review comments on CE telemetry metrics

- harden grpc_endpoint_from_url for scheme-less OTLP_ENDPOINT values
  (e.g. "telemetry.plane.so:4317") by prepending "//" before urlparse
- fix WEB_URL domain extraction for scheme-less values with same approach
- replace N+1 workspace count queries (6×N) with 6 batched annotate(Count)
  aggregation queries — reduces DB load significantly at WORKSPACE_METRICS_LIMIT
- add deterministic ordering (order_by created_at) to workspace slice
- harden METRICS_PUSH_INTERVAL_MINUTES env parsing with try/except guard
  and positive-value validation to avoid crash on malformed input

Co-authored-by: Plane AI <noreply@plane.so>

* fix: cap METRICS_PUSH_INTERVAL_MINUTES to prevent timedelta overflow

Add upper-bound check (10_000_000 minutes) and catch OverflowError alongside
ValueError so an arbitrarily large env value cannot crash worker startup via
timedelta(minutes=...) OverflowError.

Co-authored-by: Plane AI <noreply@plane.so>

---------

Co-authored-by: Plane AI <noreply@plane.so>
2026-05-28 18:34:27 +05:30
sriram veeraghanta
9f77ea5ebb
fix: Add docker pytest runner and fix bugs the suite surfaced (#9138)
* chore(api): add docker compose test runner

Adds docker-compose-test.yml at the repo root that boots an isolated
postgres / valkey / rabbitmq / minio stack with health checks and tmpfs
data dirs, then runs pytest against it and exits. Includes a usage doc
under apps/api/tests/RUNNING_TESTS.md and a pointer in AGENTS.md.

Prereq: ./setup.sh (generates apps/api/.env).

Usage:
  docker compose -f docker-compose-test.yml up --build \
    --abort-on-container-exit --exit-code-from api-tests
  docker compose -f docker-compose-test.yml down -v

* fix(api): correct bugs surfaced by the pytest suite

Five small bugs caught by enabling the pytest contract suite end-to-end.
Each is independently justifiable:

- api/serializers/cycle.py + api/views/cycle.py: CycleCreateSerializer.validate
  required project_id in the request body, but the view only ever passes
  it through the URL kwarg. Cycle create/update via the public API was
  returning 400 "Project ID is required". Read project_id from
  serializer context (passed by the view) in addition to body/instance.

- app/views/api.py: ApiTokenEndpoint.get(pk) and patch(pk) did not filter
  out is_service=True tokens, so a user could read and modify service
  tokens through the user token endpoint. The list mode and delete
  already filter is_service=False; aligned the other two.

- bgtasks/work_item_link_task.py: validate_url_ip checked hostname before
  scheme, so file:///etc/passwd raised "No hostname found" instead of
  the documented "Only HTTP and HTTPS" error. Swapped the order so the
  scheme guard matches the docstring intent.

- utils/path_validator.py: get_allowed_hosts used `WEB_URL or APP_BASE_URL`
  so when both are configured to different hosts (the standard local
  setup: WEB_URL=:8000, APP_BASE_URL=:3000), only one was added to the
  allow-list. Redirects to APP_BASE_URL then had their next_path stripped
  because the host wasn't allowed. Include every configured base URL.

* chore(api): align pytest tests with current behavior, clear warnings

Test-side fixes paired with the product fixes in the previous commit, plus
deprecation cleanup that drops the test run from 104 warnings to 0.

Tests:
- tests/contract/api/test_cycles.py: project fixture sets cycle_view=True;
  the Project model defaults the flag to False, so cycle create/update
  always tripped "Cycles are not enabled for this project".
- tests/contract/app/test_authentication.py: next_path uses "/workspaces"
  (validate_next_path rejects values without a leading slash and returns
  empty, which dropped the path from the redirect URL).
- tests/unit/bg_tasks/test_copy_s3_objects.py: mocked sync_with_external_service
  now returns description_json; the task unconditionally writes the value
  back to the Issue, and Issue.description_json is NOT NULL on UPDATE.
- tests/unit/utils/test_url.py: three length-limit tests placed the URL at
  char 970+ on a single line, which contains_url truncates away as ReDoS
  defense (500-char per-line cap). Restructured to keep test intent intact
  while staying inside the per-line window.

Warning cleanup (104 → 0):
- settings/common.py: removed USE_L10N=True (deprecated in Django 4.0,
  removed in 5.0; default is True).
- celery.py, settings/local.py, settings/production.py: pythonjsonlogger
  moved jsonlogger → json; update the import / formatter path.
2026-05-26 01:21:37 +05:30
sriram veeraghanta
02d0ee3e0f
chore: add copyright (#8584)
* feat: adding new copyright info on all files

* chore: adding CI
2026-01-27 13:54:22 +05:30
sriram veeraghanta
9237f568dd
[WEB-5044] fix: ruff lint and format errors (#7868)
* fix: lint errors

* fix: file formatting

* fix: code refactor
2025-09-29 19:15:32 +05:30
Nikhil
69c688b017
[WEB-4873]: Add webhook log cleanup task and update Celery schedule (#7772) 2025-09-18 20:09:01 +05:30
Nikhil
935e4b5c33
[WEB-4720] chore: refactor and extend cleanup tasks for logs and versions (#7604)
* Refactor and extend cleanup tasks for logs and versions

- Consolidate API log deletion into cleanup_task.py - Add tasks to
delete old email logs, page versions, and issue description versions -
Update Celery schedule and imports for new tasks

* chore: update cleanup task with mongo changes

* fix: update log deletion task name for clarity

* fix: enhance MongoDB archival error handling in cleanup task

- Added a parameter to check MongoDB availability in the flush_to_mongo_and_delete function.
- Implemented error logging for MongoDB archival failures.
- Updated calls to flush_to_mongo_and_delete to include the new parameter.

* fix: correct parameter name in cleanup task function call

- Updated the parameter name from 'mode' to 'model' in the process_cleanup_task function to ensure consistency and clarity in the code.

* fix: improve MongoDB connection parameter handling in MongoConnection class

- Replaced direct access to settings with getattr for MONGO_DB_URL and MONGO_DB_DATABASE to enhance robustness.
- Added warning logging for missing MongoDB connection parameters.
2025-08-24 15:13:49 +05:30
sriram veeraghanta
fdbe4c2ca6
chore: rename server to api (#7342) 2025-07-04 15:32:21 +05:30
Renamed from apps/server/plane/celery.py (Browse further)