Prevent tab, newline, and other ASCII control characters from appearing
in S3 object keys generated from user-provided upload filenames.
Fixes#9127
Co-authored-by: Cursor <cursoragent@cursor.com>
* chore(api): add docker compose test runner
Adds docker-compose-test.yml at the repo root that boots an isolated
postgres / valkey / rabbitmq / minio stack with health checks and tmpfs
data dirs, then runs pytest against it and exits. Includes a usage doc
under apps/api/tests/RUNNING_TESTS.md and a pointer in AGENTS.md.
Prereq: ./setup.sh (generates apps/api/.env).
Usage:
docker compose -f docker-compose-test.yml up --build \
--abort-on-container-exit --exit-code-from api-tests
docker compose -f docker-compose-test.yml down -v
* fix(api): correct bugs surfaced by the pytest suite
Five small bugs caught by enabling the pytest contract suite end-to-end.
Each is independently justifiable:
- api/serializers/cycle.py + api/views/cycle.py: CycleCreateSerializer.validate
required project_id in the request body, but the view only ever passes
it through the URL kwarg. Cycle create/update via the public API was
returning 400 "Project ID is required". Read project_id from
serializer context (passed by the view) in addition to body/instance.
- app/views/api.py: ApiTokenEndpoint.get(pk) and patch(pk) did not filter
out is_service=True tokens, so a user could read and modify service
tokens through the user token endpoint. The list mode and delete
already filter is_service=False; aligned the other two.
- bgtasks/work_item_link_task.py: validate_url_ip checked hostname before
scheme, so file:///etc/passwd raised "No hostname found" instead of
the documented "Only HTTP and HTTPS" error. Swapped the order so the
scheme guard matches the docstring intent.
- utils/path_validator.py: get_allowed_hosts used `WEB_URL or APP_BASE_URL`
so when both are configured to different hosts (the standard local
setup: WEB_URL=:8000, APP_BASE_URL=:3000), only one was added to the
allow-list. Redirects to APP_BASE_URL then had their next_path stripped
because the host wasn't allowed. Include every configured base URL.
* chore(api): align pytest tests with current behavior, clear warnings
Test-side fixes paired with the product fixes in the previous commit, plus
deprecation cleanup that drops the test run from 104 warnings to 0.
Tests:
- tests/contract/api/test_cycles.py: project fixture sets cycle_view=True;
the Project model defaults the flag to False, so cycle create/update
always tripped "Cycles are not enabled for this project".
- tests/contract/app/test_authentication.py: next_path uses "/workspaces"
(validate_next_path rejects values without a leading slash and returns
empty, which dropped the path from the redirect URL).
- tests/unit/bg_tasks/test_copy_s3_objects.py: mocked sync_with_external_service
now returns description_json; the task unconditionally writes the value
back to the Issue, and Issue.description_json is NOT NULL on UPDATE.
- tests/unit/utils/test_url.py: three length-limit tests placed the URL at
char 970+ on a single line, which contains_url truncates away as ReDoS
defense (500-char per-line cap). Restructured to keep test intent intact
while staying inside the per-line window.
Warning cleanup (104 → 0):
- settings/common.py: removed USE_L10N=True (deprecated in Django 4.0,
removed in 5.0; default is True).
- celery.py, settings/local.py, settings/production.py: pythonjsonlogger
moved jsonlogger → json; update the import / formatter path.
* fix: sanitize filenames in upload paths to prevent path traversal (GHSA-v57h-5999-w7xp)
Add server-side filename sanitization across all file upload endpoints
to prevent path traversal sequences (../) in user-supplied filenames
from being incorporated into S3 object keys. While S3 keys are flat
strings and not vulnerable to filesystem traversal, this adds
defense-in-depth and prevents S3 key pollution.
Changes:
- Add sanitize_filename() utility in path_validator.py
- Sanitize filenames in get_upload_path() for FileAsset and IssueAttachment models
- Sanitize name parameter in all upload view endpoints
* fix: address PR review feedback on filename sanitization
- Remove unused `import re`
- Normalize backslashes to forward slashes before os.path.basename()
so Windows-style paths (e.g. ..\..\..\evil.txt) are handled on POSIX
- Strip whitespace before removing leading dots so " .env" is caught
- Return None instead of "unnamed" for empty input so existing
`if not name` validation guards remain effective
- Add `or "unnamed"` fallback at call sites that lack a name guard
* fix: use random hex name as fallback in get_upload_path instead of "unnamed"
* fix: resolve ruff E501 line too long in DuplicateAssetEndpoint
* refactor: enhance URL validation and redirection logic in authentication views
* Updated authentication views (SignInAuthSpaceEndpoint, GitHubCallbackSpaceEndpoint, GitLabCallbackSpaceEndpoint, GoogleCallbackSpaceEndpoint, and MagicSignInSpaceEndpoint) to include url_has_allowed_host_and_scheme checks for safer redirection.
* Improved URL construction by ensuring proper formatting and fallback to base host when necessary.
* Added get_allowed_hosts function to path_validator.py for better host validation.
* refactor: improve comments and clean up code in path_validator.py
* Updated comments for clarity in the get_safe_redirect_url function.
* Removed unnecessary blank line to enhance
* feat: enhance path validation and URL safety in path_validator.py
* Added get_allowed_hosts function to retrieve allowed hosts from settings.
* Updated get_safe_redirect_url to validate URLs against allowed hosts.
* Improved URL construction logic for safer redirection handling.
* feat: enhance URL validation in authentication views
* Added url_has_allowed_host_and_scheme checks in SignUpAuthSpaceEndpoint and MagicSignInSpaceEndpoint for safer redirection.
* Updated redirect logic to fallback to base host if the constructed URL is not allowed.
* Improved overall URL safety and handling in authentication flows.
* fix: improve host extraction in get_allowed_hosts function
* Updated get_allowed_hosts to extract only the host from ADMIN_BASE_URL and SPACE_BASE_URL settings for better URL validation.
* Enhanced overall safety and clarity in allowed hosts retrieval.
* refactor: streamline URL construction in authentication views
* Updated MagicSignInSpaceEndpoint and MagicSignUpSpaceEndpoint to directly construct redirect URLs using formatted strings instead of the get_safe_redirect_url function.
* Enhanced get_safe_redirect_url to use quote for safer URL encoding of parameters.
* refactor: enhance URL validation and redirection in authentication views
* Added validate_next_path function to improve the safety of redirect URLs in MagicSignInSpaceEndpoint and MagicSignUpSpaceEndpoint.
* Updated URL construction to ensure proper handling of next_path and base_url.
* Streamlined the get_safe_redirect_url function for better parameter encoding.
* refactor: unify URL redirection logic across authentication views
* Introduced validate_next_path function to enhance URL safety in SignInAuthSpaceEndpoint, SignUpAuthSpaceEndpoint, GitHubCallbackSpaceEndpoint, GitLabCallbackSpaceEndpoint, and GoogleCallbackSpaceEndpoint.
* Updated URL construction to directly format the redirect URL, improving clarity and consistency across multiple authentication views.
* refactor: replace validate_next_path with get_safe_redirect_url for safer URL redirection across authentication views
* refactor: use get_safe_redirect_url for improved URL redirection in SignInAuthSpaceEndpoint and SignUpAuthSpaceEndpoint
* fix: redirect paths
---------
Co-authored-by: sriram veeraghanta <veeraghanta.sriram@gmail.com>
