Commit graph

8 commits

Author SHA1 Message Date
sriram veeraghanta
edf2475413
refactor: logging with retention + API token hardening (#9148)
* fix: harden API token handling against rate-limit tampering and plaintext logging

- Make `allowed_rate_limit` read-only on APITokenSerializer so users can no
  longer raise their own API token rate limit via PATCH (GHSA-xfgr-2x3f-g2cf).
- Stop persisting API keys in plaintext in APITokenLogMiddleware: store a
  SHA-256 hash as the token identifier and redact sensitive request headers
  (X-Api-Key, Authorization, Cookie) before logging (GHSA-r5p8-cj3q-38cc).

* refactor: remove MongoDB log sink and add per-log-type retention

Logs are now written to and cleared from PostgreSQL only; MongoDB is no
longer used as a log sink or archive.

- Drop the MongoDB write/archival paths from the API request logger, the
  webhook log writer, and the cleanup tasks; Postgres is the sole sink.
- Cleanup tasks now hard-delete expired rows in batches via `all_objects`
  (rows are removed immediately, not soft-deleted).
- Add env-backed, per-log-type retention settings: API activity logs
  (API_ACTIVITY_LOG_RETENTION_DAYS, default 14), webhook logs
  (WEBHOOK_LOG_RETENTION_DAYS, default 14), email logs
  (EMAIL_LOG_RETENTION_DAYS, default 7). HARD_DELETE_AFTER_DAYS no longer
  drives any log cleanup.
- Delete settings/mongo.py, remove MONGO_DB_* settings and the plane.mongo
  loggers, and drop the pymongo dependency.

* chore: gitignore local advisories.md notes file

* fix: use keyed HMAC-SHA256 for API token log identifier

Address CodeQL "weak hashing of sensitive data" by hashing the API key with
a SECRET_KEY-keyed HMAC instead of a bare SHA-256. The identifier is a
non-reversible tokenization of a high-entropy key (not password storage);
keying it also prevents precomputing the digest from a known key value.

* chore: address review feedback on log cleanup and request logging

- process_logs accepts extra kwargs so jobs enqueued by an older release
  (with a mongo_log arg) don't fail during a rolling deploy.
- Log-cleanup batch delete failures are logged and skipped rather than
  aborting the run, so a single bad batch can't block the rest.
- Extend logger middleware test to assert Authorization and Cookie headers
  are redacted; add a test that a failing cleanup batch is swallowed.

* fix: fall back to default when a log retention env value is invalid

Negative (or unparseable) retention values would compute a future cutoff and
delete every log row. The retention settings now fall back to their defaults
in that case via a shared `_retention_days` helper.
2026-05-27 16:00:05 +05:30
sriram veeraghanta
02d0ee3e0f
chore: add copyright (#8584)
* feat: adding new copyright info on all files

* chore: adding CI
2026-01-27 13:54:22 +05:30
Nikhil
1a9ebc8b68
[WEB-5071] chore: implement webhook logging to MongoDB and fallback to database (#7896)
* feat: implement webhook logging to MongoDB and fallback to database (#7887)

- Added a new function `save_webhook_log` to log webhook requests and responses to MongoDB, with a fallback to the database if the MongoDB save fails.
- Updated the `webhook_send_task` to utilize the new logging function.
- Modified the `get_webhook_logs_queryset` to order logs by creation date and adjusted the chunk size for iteration.

* refactor: clean up log data formatting in save_webhook_log function

* fix: update retry_count type in save_webhook_log function
2025-10-15 14:53:43 +05:30
sriram veeraghanta
9237f568dd
[WEB-5044] fix: ruff lint and format errors (#7868)
* fix: lint errors

* fix: file formatting

* fix: code refactor
2025-09-29 19:15:32 +05:30
Nikhil
69c688b017
[WEB-4873]: Add webhook log cleanup task and update Celery schedule (#7772) 2025-09-18 20:09:01 +05:30
Nikhil
e2c0d0f23c
[WEB-4813] fix: ensure all identifiers in log transformations are converted to strings (#7682)
* fix: ensure all identifiers in log transformations are converted to strings

* Update apps/api/plane/bgtasks/cleanup_task.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-09-01 14:12:34 +05:30
Nikhil
489a6e1e94
[WEB-4796] fix: update MongoDB collection check to use 'is not None' for better clarity #7671 2025-08-29 14:25:46 +05:30
Nikhil
935e4b5c33
[WEB-4720] chore: refactor and extend cleanup tasks for logs and versions (#7604)
* Refactor and extend cleanup tasks for logs and versions

- Consolidate API log deletion into cleanup_task.py - Add tasks to
delete old email logs, page versions, and issue description versions -
Update Celery schedule and imports for new tasks

* chore: update cleanup task with mongo changes

* fix: update log deletion task name for clarity

* fix: enhance MongoDB archival error handling in cleanup task

- Added a parameter to check MongoDB availability in the flush_to_mongo_and_delete function.
- Implemented error logging for MongoDB archival failures.
- Updated calls to flush_to_mongo_and_delete to include the new parameter.

* fix: correct parameter name in cleanup task function call

- Updated the parameter name from 'mode' to 'model' in the process_cleanup_task function to ensure consistency and clarity in the code.

* fix: improve MongoDB connection parameter handling in MongoConnection class

- Replaced direct access to settings with getattr for MONGO_DB_URL and MONGO_DB_DATABASE to enhance robustness.
- Added warning logging for missing MongoDB connection parameters.
2025-08-24 15:13:49 +05:30