2026-01-27 08:24:22 +00:00
|
|
|
# Copyright (c) 2023-present Plane Software, Inc. and contributors
|
|
|
|
|
# SPDX-License-Identifier: AGPL-3.0-only
|
|
|
|
|
# See the LICENSE file for details.
|
|
|
|
|
|
2025-12-09 19:31:10 +00:00
|
|
|
# Python imports
|
|
|
|
|
import logging
|
refactor: logging with retention + API token hardening (#9148)
* fix: harden API token handling against rate-limit tampering and plaintext logging
- Make `allowed_rate_limit` read-only on APITokenSerializer so users can no
longer raise their own API token rate limit via PATCH (GHSA-xfgr-2x3f-g2cf).
- Stop persisting API keys in plaintext in APITokenLogMiddleware: store a
SHA-256 hash as the token identifier and redact sensitive request headers
(X-Api-Key, Authorization, Cookie) before logging (GHSA-r5p8-cj3q-38cc).
* refactor: remove MongoDB log sink and add per-log-type retention
Logs are now written to and cleared from PostgreSQL only; MongoDB is no
longer used as a log sink or archive.
- Drop the MongoDB write/archival paths from the API request logger, the
webhook log writer, and the cleanup tasks; Postgres is the sole sink.
- Cleanup tasks now hard-delete expired rows in batches via `all_objects`
(rows are removed immediately, not soft-deleted).
- Add env-backed, per-log-type retention settings: API activity logs
(API_ACTIVITY_LOG_RETENTION_DAYS, default 14), webhook logs
(WEBHOOK_LOG_RETENTION_DAYS, default 14), email logs
(EMAIL_LOG_RETENTION_DAYS, default 7). HARD_DELETE_AFTER_DAYS no longer
drives any log cleanup.
- Delete settings/mongo.py, remove MONGO_DB_* settings and the plane.mongo
loggers, and drop the pymongo dependency.
* chore: gitignore local advisories.md notes file
* fix: use keyed HMAC-SHA256 for API token log identifier
Address CodeQL "weak hashing of sensitive data" by hashing the API key with
a SECRET_KEY-keyed HMAC instead of a bare SHA-256. The identifier is a
non-reversible tokenization of a high-entropy key (not password storage);
keying it also prevents precomputing the digest from a known key value.
* chore: address review feedback on log cleanup and request logging
- process_logs accepts extra kwargs so jobs enqueued by an older release
(with a mongo_log arg) don't fail during a rolling deploy.
- Log-cleanup batch delete failures are logged and skipped rather than
aborting the run, so a single bad batch can't block the rest.
- Extend logger middleware test to assert Authorization and Cookie headers
are redacted; add a test that a failing cleanup batch is swallowed.
* fix: fall back to default when a log retention env value is invalid
Negative (or unparseable) retention values would compute a future cutoff and
delete every log row. The retention settings now fall back to their defaults
in that case via a shared `_retention_days` helper.
2026-05-27 10:30:05 +00:00
|
|
|
from typing import Dict, Any
|
2025-12-09 19:31:10 +00:00
|
|
|
|
|
|
|
|
# Third party imports
|
|
|
|
|
from celery import shared_task
|
|
|
|
|
|
|
|
|
|
# Django imports
|
|
|
|
|
from plane.utils.exception_logger import log_exception
|
|
|
|
|
from plane.db.models import APIActivityLog
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
logger = logging.getLogger("plane.worker")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def log_to_postgres(log_data: Dict[str, Any]) -> bool:
|
|
|
|
|
"""
|
refactor: logging with retention + API token hardening (#9148)
* fix: harden API token handling against rate-limit tampering and plaintext logging
- Make `allowed_rate_limit` read-only on APITokenSerializer so users can no
longer raise their own API token rate limit via PATCH (GHSA-xfgr-2x3f-g2cf).
- Stop persisting API keys in plaintext in APITokenLogMiddleware: store a
SHA-256 hash as the token identifier and redact sensitive request headers
(X-Api-Key, Authorization, Cookie) before logging (GHSA-r5p8-cj3q-38cc).
* refactor: remove MongoDB log sink and add per-log-type retention
Logs are now written to and cleared from PostgreSQL only; MongoDB is no
longer used as a log sink or archive.
- Drop the MongoDB write/archival paths from the API request logger, the
webhook log writer, and the cleanup tasks; Postgres is the sole sink.
- Cleanup tasks now hard-delete expired rows in batches via `all_objects`
(rows are removed immediately, not soft-deleted).
- Add env-backed, per-log-type retention settings: API activity logs
(API_ACTIVITY_LOG_RETENTION_DAYS, default 14), webhook logs
(WEBHOOK_LOG_RETENTION_DAYS, default 14), email logs
(EMAIL_LOG_RETENTION_DAYS, default 7). HARD_DELETE_AFTER_DAYS no longer
drives any log cleanup.
- Delete settings/mongo.py, remove MONGO_DB_* settings and the plane.mongo
loggers, and drop the pymongo dependency.
* chore: gitignore local advisories.md notes file
* fix: use keyed HMAC-SHA256 for API token log identifier
Address CodeQL "weak hashing of sensitive data" by hashing the API key with
a SECRET_KEY-keyed HMAC instead of a bare SHA-256. The identifier is a
non-reversible tokenization of a high-entropy key (not password storage);
keying it also prevents precomputing the digest from a known key value.
* chore: address review feedback on log cleanup and request logging
- process_logs accepts extra kwargs so jobs enqueued by an older release
(with a mongo_log arg) don't fail during a rolling deploy.
- Log-cleanup batch delete failures are logged and skipped rather than
aborting the run, so a single bad batch can't block the rest.
- Extend logger middleware test to assert Authorization and Cookie headers
are redacted; add a test that a failing cleanup batch is swallowed.
* fix: fall back to default when a log retention env value is invalid
Negative (or unparseable) retention values would compute a future cutoff and
delete every log row. The retention settings now fall back to their defaults
in that case via a shared `_retention_days` helper.
2026-05-27 10:30:05 +00:00
|
|
|
Persist an external API request log to PostgreSQL.
|
2025-12-09 19:31:10 +00:00
|
|
|
"""
|
|
|
|
|
try:
|
|
|
|
|
APIActivityLog.objects.create(**log_data)
|
|
|
|
|
return True
|
|
|
|
|
except Exception as e:
|
|
|
|
|
log_exception(e)
|
|
|
|
|
return False
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@shared_task
|
refactor: logging with retention + API token hardening (#9148)
* fix: harden API token handling against rate-limit tampering and plaintext logging
- Make `allowed_rate_limit` read-only on APITokenSerializer so users can no
longer raise their own API token rate limit via PATCH (GHSA-xfgr-2x3f-g2cf).
- Stop persisting API keys in plaintext in APITokenLogMiddleware: store a
SHA-256 hash as the token identifier and redact sensitive request headers
(X-Api-Key, Authorization, Cookie) before logging (GHSA-r5p8-cj3q-38cc).
* refactor: remove MongoDB log sink and add per-log-type retention
Logs are now written to and cleared from PostgreSQL only; MongoDB is no
longer used as a log sink or archive.
- Drop the MongoDB write/archival paths from the API request logger, the
webhook log writer, and the cleanup tasks; Postgres is the sole sink.
- Cleanup tasks now hard-delete expired rows in batches via `all_objects`
(rows are removed immediately, not soft-deleted).
- Add env-backed, per-log-type retention settings: API activity logs
(API_ACTIVITY_LOG_RETENTION_DAYS, default 14), webhook logs
(WEBHOOK_LOG_RETENTION_DAYS, default 14), email logs
(EMAIL_LOG_RETENTION_DAYS, default 7). HARD_DELETE_AFTER_DAYS no longer
drives any log cleanup.
- Delete settings/mongo.py, remove MONGO_DB_* settings and the plane.mongo
loggers, and drop the pymongo dependency.
* chore: gitignore local advisories.md notes file
* fix: use keyed HMAC-SHA256 for API token log identifier
Address CodeQL "weak hashing of sensitive data" by hashing the API key with
a SECRET_KEY-keyed HMAC instead of a bare SHA-256. The identifier is a
non-reversible tokenization of a high-entropy key (not password storage);
keying it also prevents precomputing the digest from a known key value.
* chore: address review feedback on log cleanup and request logging
- process_logs accepts extra kwargs so jobs enqueued by an older release
(with a mongo_log arg) don't fail during a rolling deploy.
- Log-cleanup batch delete failures are logged and skipped rather than
aborting the run, so a single bad batch can't block the rest.
- Extend logger middleware test to assert Authorization and Cookie headers
are redacted; add a test that a failing cleanup batch is swallowed.
* fix: fall back to default when a log retention env value is invalid
Negative (or unparseable) retention values would compute a future cutoff and
delete every log row. The retention settings now fall back to their defaults
in that case via a shared `_retention_days` helper.
2026-05-27 10:30:05 +00:00
|
|
|
def process_logs(log_data: Dict[str, Any], **_: Any) -> None:
|
2025-12-09 19:31:10 +00:00
|
|
|
"""
|
refactor: logging with retention + API token hardening (#9148)
* fix: harden API token handling against rate-limit tampering and plaintext logging
- Make `allowed_rate_limit` read-only on APITokenSerializer so users can no
longer raise their own API token rate limit via PATCH (GHSA-xfgr-2x3f-g2cf).
- Stop persisting API keys in plaintext in APITokenLogMiddleware: store a
SHA-256 hash as the token identifier and redact sensitive request headers
(X-Api-Key, Authorization, Cookie) before logging (GHSA-r5p8-cj3q-38cc).
* refactor: remove MongoDB log sink and add per-log-type retention
Logs are now written to and cleared from PostgreSQL only; MongoDB is no
longer used as a log sink or archive.
- Drop the MongoDB write/archival paths from the API request logger, the
webhook log writer, and the cleanup tasks; Postgres is the sole sink.
- Cleanup tasks now hard-delete expired rows in batches via `all_objects`
(rows are removed immediately, not soft-deleted).
- Add env-backed, per-log-type retention settings: API activity logs
(API_ACTIVITY_LOG_RETENTION_DAYS, default 14), webhook logs
(WEBHOOK_LOG_RETENTION_DAYS, default 14), email logs
(EMAIL_LOG_RETENTION_DAYS, default 7). HARD_DELETE_AFTER_DAYS no longer
drives any log cleanup.
- Delete settings/mongo.py, remove MONGO_DB_* settings and the plane.mongo
loggers, and drop the pymongo dependency.
* chore: gitignore local advisories.md notes file
* fix: use keyed HMAC-SHA256 for API token log identifier
Address CodeQL "weak hashing of sensitive data" by hashing the API key with
a SECRET_KEY-keyed HMAC instead of a bare SHA-256. The identifier is a
non-reversible tokenization of a high-entropy key (not password storage);
keying it also prevents precomputing the digest from a known key value.
* chore: address review feedback on log cleanup and request logging
- process_logs accepts extra kwargs so jobs enqueued by an older release
(with a mongo_log arg) don't fail during a rolling deploy.
- Log-cleanup batch delete failures are logged and skipped rather than
aborting the run, so a single bad batch can't block the rest.
- Extend logger middleware test to assert Authorization and Cookie headers
are redacted; add a test that a failing cleanup batch is swallowed.
* fix: fall back to default when a log retention env value is invalid
Negative (or unparseable) retention values would compute a future cutoff and
delete every log row. The retention settings now fall back to their defaults
in that case via a shared `_retention_days` helper.
2026-05-27 10:30:05 +00:00
|
|
|
Persist external API request logs to PostgreSQL.
|
2025-12-09 19:31:10 +00:00
|
|
|
|
refactor: logging with retention + API token hardening (#9148)
* fix: harden API token handling against rate-limit tampering and plaintext logging
- Make `allowed_rate_limit` read-only on APITokenSerializer so users can no
longer raise their own API token rate limit via PATCH (GHSA-xfgr-2x3f-g2cf).
- Stop persisting API keys in plaintext in APITokenLogMiddleware: store a
SHA-256 hash as the token identifier and redact sensitive request headers
(X-Api-Key, Authorization, Cookie) before logging (GHSA-r5p8-cj3q-38cc).
* refactor: remove MongoDB log sink and add per-log-type retention
Logs are now written to and cleared from PostgreSQL only; MongoDB is no
longer used as a log sink or archive.
- Drop the MongoDB write/archival paths from the API request logger, the
webhook log writer, and the cleanup tasks; Postgres is the sole sink.
- Cleanup tasks now hard-delete expired rows in batches via `all_objects`
(rows are removed immediately, not soft-deleted).
- Add env-backed, per-log-type retention settings: API activity logs
(API_ACTIVITY_LOG_RETENTION_DAYS, default 14), webhook logs
(WEBHOOK_LOG_RETENTION_DAYS, default 14), email logs
(EMAIL_LOG_RETENTION_DAYS, default 7). HARD_DELETE_AFTER_DAYS no longer
drives any log cleanup.
- Delete settings/mongo.py, remove MONGO_DB_* settings and the plane.mongo
loggers, and drop the pymongo dependency.
* chore: gitignore local advisories.md notes file
* fix: use keyed HMAC-SHA256 for API token log identifier
Address CodeQL "weak hashing of sensitive data" by hashing the API key with
a SECRET_KEY-keyed HMAC instead of a bare SHA-256. The identifier is a
non-reversible tokenization of a high-entropy key (not password storage);
keying it also prevents precomputing the digest from a known key value.
* chore: address review feedback on log cleanup and request logging
- process_logs accepts extra kwargs so jobs enqueued by an older release
(with a mongo_log arg) don't fail during a rolling deploy.
- Log-cleanup batch delete failures are logged and skipped rather than
aborting the run, so a single bad batch can't block the rest.
- Extend logger middleware test to assert Authorization and Cookie headers
are redacted; add a test that a failing cleanup batch is swallowed.
* fix: fall back to default when a log retention env value is invalid
Negative (or unparseable) retention values would compute a future cutoff and
delete every log row. The retention settings now fall back to their defaults
in that case via a shared `_retention_days` helper.
2026-05-27 10:30:05 +00:00
|
|
|
The catch-all kwargs keep this task signature compatible with jobs enqueued
|
|
|
|
|
by an older release (which passed a `mongo_log` argument), so in-flight tasks
|
|
|
|
|
don't fail during a rolling deploy. It can be dropped once no such jobs remain.
|
|
|
|
|
"""
|
|
|
|
|
log_to_postgres(log_data)
|