Commit graph

7 commits

Author SHA1 Message Date
sriram veeraghanta
edf2475413
refactor: logging with retention + API token hardening (#9148)
* fix: harden API token handling against rate-limit tampering and plaintext logging

- Make `allowed_rate_limit` read-only on APITokenSerializer so users can no
  longer raise their own API token rate limit via PATCH (GHSA-xfgr-2x3f-g2cf).
- Stop persisting API keys in plaintext in APITokenLogMiddleware: store a
  SHA-256 hash as the token identifier and redact sensitive request headers
  (X-Api-Key, Authorization, Cookie) before logging (GHSA-r5p8-cj3q-38cc).

* refactor: remove MongoDB log sink and add per-log-type retention

Logs are now written to and cleared from PostgreSQL only; MongoDB is no
longer used as a log sink or archive.

- Drop the MongoDB write/archival paths from the API request logger, the
  webhook log writer, and the cleanup tasks; Postgres is the sole sink.
- Cleanup tasks now hard-delete expired rows in batches via `all_objects`
  (rows are removed immediately, not soft-deleted).
- Add env-backed, per-log-type retention settings: API activity logs
  (API_ACTIVITY_LOG_RETENTION_DAYS, default 14), webhook logs
  (WEBHOOK_LOG_RETENTION_DAYS, default 14), email logs
  (EMAIL_LOG_RETENTION_DAYS, default 7). HARD_DELETE_AFTER_DAYS no longer
  drives any log cleanup.
- Delete settings/mongo.py, remove MONGO_DB_* settings and the plane.mongo
  loggers, and drop the pymongo dependency.

* chore: gitignore local advisories.md notes file

* fix: use keyed HMAC-SHA256 for API token log identifier

Address CodeQL "weak hashing of sensitive data" by hashing the API key with
a SECRET_KEY-keyed HMAC instead of a bare SHA-256. The identifier is a
non-reversible tokenization of a high-entropy key (not password storage);
keying it also prevents precomputing the digest from a known key value.

* chore: address review feedback on log cleanup and request logging

- process_logs accepts extra kwargs so jobs enqueued by an older release
  (with a mongo_log arg) don't fail during a rolling deploy.
- Log-cleanup batch delete failures are logged and skipped rather than
  aborting the run, so a single bad batch can't block the rest.
- Extend logger middleware test to assert Authorization and Cookie headers
  are redacted; add a test that a failing cleanup batch is swallowed.

* fix: fall back to default when a log retention env value is invalid

Negative (or unparseable) retention values would compute a future cutoff and
delete every log row. The retention settings now fall back to their defaults
in that case via a shared `_retention_days` helper.
2026-05-27 16:00:05 +05:30
sriram veeraghanta
02d0ee3e0f
chore: add copyright (#8584)
* feat: adding new copyright info on all files

* chore: adding CI
2026-01-27 13:54:22 +05:30
Nikhil
491b5bc6cc
[WEB-5575]feat: enhance APITokenLogMiddleware to support logging to MongoDB (#8241)
* feat: enhance APITokenLogMiddleware to support logging to MongoDB

- Added functionality to log external API requests to MongoDB, with a fallback to PostgreSQL if MongoDB is unavailable.
- Implemented error handling for MongoDB connection and logging operations.
- Introduced additional fields for MongoDB logs, including timestamps and user identifiers.
- Refactored request logging logic to streamline the process and improve maintainability.

* fix: improve MongoDB availability checks in APITokenLogMiddleware

- Enhanced the logic for determining MongoDB availability by checking if the collection is not None.
- Added a check for MongoDB configuration before attempting to retrieve the collection.
- Updated error handling to ensure the middleware correctly reflects the state of MongoDB connectivity.

* feat: implement logging functionality in logger_task for API activity

- Added a new logger_task module to handle logging of API activity to MongoDB and PostgreSQL.
- Introduced functions for safely decoding request/response bodies and processing logs based on MongoDB availability.
- Refactored APITokenLogMiddleware to utilize the new logging functions, improving code organization and maintainability.

* refactor: simplify MongoDB logging in logger_task and middleware

- Removed direct dependency on MongoDB collection in log_to_mongo function, now retrieving it internally.
- Updated process_logs to check MongoDB configuration before logging, enhancing error handling.
- Cleaned up logger.py by removing unused imports related to MongoDB.

* feat: add Celery task decorator to process_logs function in logger_task

- Introduced the @shared_task decorator to the process_logs function, enabling asynchronous processing of log data.
- Updated function signature to include a return type of None for clarity.
2025-12-10 01:01:10 +05:30
Bavisetti Narayan
606e34ec81
[WIKI-730] chore: handle body too large error (#7963)
* chore: added middleware to handle body too large

* chore: added middleware to handle body too large

* chore: indentend the code

* chore: changed the response structure

* chore: changed the response structure

* chore: created a new file for middleware

* chore: added a standardized error key
2025-10-14 17:21:11 +05:30
sriram veeraghanta
9237f568dd
[WEB-5044] fix: ruff lint and format errors (#7868)
* fix: lint errors

* fix: file formatting

* fix: code refactor
2025-09-29 19:15:32 +05:30
Sangeetha
84879ee3bd
[WEB-4533] feat: read replica functionality (#7453)
* feat: read replica functionality

* fix: set use_read_replica to false

* chore: add use_read_replica to external APIs

* chore: remove use_read_replica on read endpoints

* chore: remove md files

* Updated all the necessary endpoints to use read replica

---------

Co-authored-by: Dheeraj Kumar Ketireddy <dheeru0198@gmail.com>
2025-07-28 17:41:02 +05:30
sriram veeraghanta
fdbe4c2ca6
chore: rename server to api (#7342) 2025-07-04 15:32:21 +05:30