vignesh/bottleneckcrm
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merge pull request #729 from Django-CRM/dev

Browse source 
Add documentation for self-hosting, production deployment, requiremen…
This commit is contained in:
Ashwin Kumar 2026-08-02 20:09:14 +05:30 committed by GitHub
commit 2d2d3e5c5b
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
65 changed files with 10173 additions and 1828 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

40
.github/workflows/tests.yml vendored
View file

@ -6,6 +6,9 @@ on:
paths:
- 'backend/**'
- 'frontend/**'
- 'docs/**'
- 'mkdocs.yml'
- '.readthedocs.yaml'
- '.coveragerc'
- '.github/workflows/tests.yml'
pull_request:
@ -13,6 +16,9 @@ on:
paths:
- 'backend/**'
- 'frontend/**'
- 'docs/**'
- 'mkdocs.yml'
- '.readthedocs.yaml'
- '.coveragerc'
- '.github/workflows/tests.yml'
@ -120,3 +126,37 @@ jobs:
- name: Build
working-directory: frontend
run: pnpm build
docs-build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install uv
uses: astral-sh/setup-uv@v6
with:
enable-cache: true
cache-dependency-glob: 'backend/uv.lock'
- name: Install docs dependencies
working-directory: backend
run: uv sync --frozen --group docs
# --strict fails on broken internal links and nav entries pointing at
# files that do not exist.
- name: Build docs
working-directory: backend
run: uv run mkdocs build --strict -f ../mkdocs.yml -d site
# These two tests skip when mkdocs is absent, so the backend-tests job
# (which installs no docs group) cannot run them. Without this step the
# "no internal artifacts published" guard would silently never execute
# in CI — a skip reads as a pass.
- name: Check docs build guards
working-directory: backend
env:
DJANGO_SETTINGS_MODULE: crm.test_settings
SECRET_KEY: test-secret-key-for-ci
ADMIN_EMAIL: admin@test.com
run: uv run pytest common/tests/test_docs_build_excludes_internal.py --no-cov -v

2
.gitignore vendored
View file

@ -52,6 +52,8 @@ coverage-badge.svg
# Documentation
docs/build/
# MkDocs build output
site/
# Node.js / Frontend
frontend/node_modules/

21
.readthedocs.yaml Normal file
View file

@ -0,0 +1,21 @@
# Read the Docs build configuration.
# The docs dependencies live in backend/pyproject.toml's `docs` group so the
# site builds with uv, exactly as CI does — one dependency source, one lockfile.
version: 2
build:
os: ubuntu-24.04
tools:
python: "3.12"
jobs:
install:
- asdf plugin add uv
- asdf install uv latest
- asdf global uv latest
- cd backend && uv sync --frozen --group docs
build:
html:
- cd backend && uv run mkdocs build --strict -f ../mkdocs.yml -d $READTHEDOCS_OUTPUT/html
mkdocs:
configuration: mkdocs.yml

91
CONTRIBUTING.md
View file

@ -8,97 +8,26 @@ Thank you for helping improve BottleCRM. Contributions of all sizes are welcome,
- For a substantial feature or architectural change, open an issue first so the approach can be discussed before implementation begins.
- Keep pull requests focused. Unrelated changes are easier to review when submitted separately.
## Prerequisites
## Development Setup
- Python 3.12+
- [`uv`](https://docs.astral.sh/uv/)
- Node.js 24
- pnpm 10
- PostgreSQL 14+
- Redis 7+
Full instructions — cloning, Docker, the backend (`uv`), the frontend (`pnpm`), running the mobile app, adding dependencies, and running tests — live in the documentation site under Contributing:
You can also use Docker Compose, which provides the application services and development dependencies.
## Set Up the Project
Clone your fork and enter the repository:
```bash
git clone https://github.com/<your-username>/Django-CRM.git
cd Django-CRM
```
### Docker Setup
To start the full development stack:
```bash
docker compose up --build
```
The frontend is available at `http://localhost:5173`, and the backend API is available at `http://localhost:8000`.
### Backend Setup
Start PostgreSQL and Redis, create the database named in `backend/.env.example`, and ensure its credentials match your local PostgreSQL configuration. Then run:
```bash
cd backend
uv sync
cp .env.example .env
uv run python manage.py migrate
uv run python manage.py runserver
```
To process background jobs, run the Celery worker in a separate terminal:
```bash
cd backend
uv run celery -A crm worker --loglevel=INFO
```
### Frontend Setup
```bash
cd frontend
pnpm install
pnpm run dev
```
## Development Checks
Run the backend test suite from `backend/`:
```bash
uv run pytest
```
When changing Django models, generate and commit the migration, then verify that no model changes are missing migrations:
```bash
uv run python manage.py makemigrations
uv run python manage.py makemigrations --check --dry-run
```
Run the frontend checks from `frontend/`:
```bash
pnpm run lint
pnpm run check
pnpm run build
```
Use `pnpm run format` to apply the configured frontend formatting rules.
- [Development setup](docs/contributing/development-setup.md) — get all three projects running locally.
- [Testing](docs/contributing/testing.md) — running the suite, markers, and what CI runs.
- [Code style](docs/contributing/code-style.md) — Python, frontend, and Flutter conventions, and what CI does and doesn't enforce.
- [Repository layout](docs/contributing/repo-layout.md) — where each part of the codebase lives.
- [Security rules](docs/contributing/security-rules.md) — the rules this codebase treats as non-negotiable.
- [Adding an org-scoped model](docs/contributing/adding-an-org-scoped-model.md) — what a new multi-tenant model needs, including its RLS policy.
## Submitting a Pull Request
1. Create a branch from `master` with a descriptive name.
2. Make focused changes and include tests where appropriate.
3. Run the relevant development checks.
3. Run the relevant development checks — see [Testing](docs/contributing/testing.md) and [Code style](docs/contributing/code-style.md).
4. Update documentation when behavior, setup, or public APIs change.
5. Push your branch and open a pull request against `master`.
In the pull request description, explain what changed, why it changed, how it was tested, and include screenshots for visible UI changes.
In the pull request description, explain what changed, why it changed, how it was tested, and include screenshots for visible UI changes. See [Pull requests](docs/contributing/pull-requests.md) for what CI runs and what review focuses on.
## Pull Request Checklist

387
README.md
View file

@ -7,7 +7,7 @@ A free, self-hosted, multi-tenant CRM built with Django REST Framework, SvelteKi
![Django](https://img.shields.io/badge/django-6.x-green.svg)
![SvelteKit](https://img.shields.io/badge/sveltekit-2.x-orange.svg)
![Svelte](https://img.shields.io/badge/svelte-5-orange.svg)
![Flutter](https://img.shields.io/badge/flutter-3.8+-blue.svg)
![Flutter](https://img.shields.io/badge/flutter-3.44+-blue.svg)
![Coverage](./coverage-badge.svg)
**BottleCRM is an open source CRM you run on your own infrastructure.** It covers the full
@ -19,7 +19,9 @@ the database layer, so a single deployment serves one startup or hundreds of ten
No per-seat pricing, no user caps, no feature paywall. MIT licensed — fork it, self-host it, own
your data.
**[Try it free →](https://bottlecrm.io/)** · [Features](https://bottlecrm.io/features) · [Docs](https://bottlecrm.io/docs) · [Pricing](https://bottlecrm.io/pricing) · [Migrate from Salesforce or HubSpot](https://bottlecrm.io/migration)
**[Try it free →](https://bottlecrm.io/)** · [Features](https://bottlecrm.io/features) · [Product docs](https://bottlecrm.io/docs) · [Pricing](https://bottlecrm.io/pricing) · [Migrate from Salesforce or HubSpot](https://bottlecrm.io/migration)
(That's the marketing site's product documentation. For the technical documentation — self-hosting, architecture, the API, contributing — see [Documentation](#documentation) below.)
## Why BottleCRM
@ -49,383 +51,28 @@ your data.
| **Stack** | Django REST + SvelteKit | Closed source | Varies |
| **License** | MIT | Proprietary | Varies, often GPL/AGPL |
## Features
### Core CRM Modules
- **[Leads](https://bottlecrm.io/features/lead-management)** - Capture, score and convert sales leads through your pipeline
- **[Accounts](https://bottlecrm.io/features/account-management)** - Manage company/organization records
- **[Contacts](https://bottlecrm.io/features/contact-management)** - Store and organize contact information
- **[Opportunities](https://bottlecrm.io/features/sales-pipeline)** - Visual sales pipeline for deals and forecasting
- **[Tickets & Cases](https://bottlecrm.io/features/ticket-management)** - Helpdesk with SLA tracking, approvals, escalations, macros and a knowledge base
- **[Tasks](https://bottlecrm.io/features/tasks)** - Task management with calendar and Kanban board views
- **[Invoices](https://bottlecrm.io/features/invoices)** - Estimates, recurring invoices and online payments
### AI & Integrations
- **AI Agents (MCP)** - Built-in [Model Context Protocol](https://modelcontextprotocol.io) server (`mcp_server/`) lets Claude, Cursor, Codex, Gemini and any MCP client search, create and update records via a personal access token — acting as you, with your role and permissions. See [`mcp_server/README.md`](mcp_server/README.md).
- **REST API** - Every feature is API-first and documented via OpenAPI/Swagger
- **Email Integration** - AWS SES integration for transactional emails
### Platform Features
- **[Native Mobile App](https://bottlecrm.io/features/mobile-app)** - Flutter app for iOS and Android covering leads, deals, tickets, tasks, goals and timesheets
- **Multi-Tenant Architecture** - PostgreSQL Row-Level Security keeps each organization's CRM data isolated
- **JWT Authentication** - Secure token-based authentication
- **Team Management** - Organize users into teams with role-based access
- **Activity Tracking** - Comprehensive audit logs and activity history
- **Comments & Attachments** - Collaborate with comments and file attachments on any record
- **Tags** - Flexible tagging system for organizing records
- **Background Tasks** - Celery + Redis for async task processing
### Built for your industry
Pre-configured setups for [professional services and agencies](https://bottlecrm.io/industries/professional-services-crm), [education and admissions](https://bottlecrm.io/industries/education-admissions-crm), and [real estate](https://bottlecrm.io/industries/real-estate-crm).
## Tech Stack
### Backend
- **Django 6.x** with Django REST Framework
- **PostgreSQL** for relational data storage
- **Redis** for caching and Celery broker
- **Celery** for background task processing
- **JWT** for authentication
- **AWS S3** for file storage
- **AWS SES** for email delivery
### Frontend
- **SvelteKit 2.x** with Svelte 5 (runes)
- **TailwindCSS 4** for styling
- **shadcn-svelte** UI components
- **Axios** for API communication
- **Lucide** icons
### Mobile
- **Flutter 3.8+** targeting iOS and Android
- **Material Design 3** with a flat, custom design system
- **Google Sign-In** + JWT against the same backend API
### AI
- **MCP server** (`bcrm-mcp`) built on FastMCP, stdio and HTTP transports
## Quick Start
### Prerequisites
- Python 3.12+
- Node.js 24 with pnpm 10
- PostgreSQL 14+
- Redis
- Flutter 3.8+ (only if you're building the mobile app)
### Backend Setup
The backend uses [`uv`](https://docs.astral.sh/uv/) for Python dependency management — it reads `pyproject.toml`, installs from `uv.lock`, and creates the virtual environment for you. Backend configuration starts from [`backend/.env.example`](backend/.env.example).
## Quick start
```bash
# Clone the repository
git clone https://github.com/django-crm/Django-CRM.git
cd Django-CRM/backend
# Install uv (one time, system-wide)
curl -LsSf https://astral.sh/uv/install.sh | sh
# Or on macOS via Homebrew: brew install uv
# Install Python (matches the version in .python-version) and all deps into .venv/
uv sync
# Set up environment variables
cp .env.example .env
# Edit .env with your database and other settings
# Run migrations
uv run python manage.py migrate
# Create a superuser
uv run python manage.py createsuperuser
# Start the development server
uv run python manage.py runserver
```
`uv run <cmd>` resolves binaries from `.venv/bin/` automatically — no need to `source .venv/bin/activate`. If you prefer the activate-then-run flow, that still works:
```bash
source .venv/bin/activate
python manage.py runserver # equivalent to `uv run python manage.py runserver`
```
Common dev commands (from `backend/`):
```bash
uv run pytest # run tests
uv run python manage.py makemigrations # create migrations
uv run celery -A crm worker --loglevel=INFO # background worker
uv add <package> # add a dependency (updates pyproject.toml + uv.lock)
uv add --group dev <package> # add a dev-only dependency
uv lock --upgrade # refresh the lockfile
```
### Frontend Setup
```bash
# In a new terminal, from the project root
cd frontend
# Install dependencies
pnpm install
# Start the development server
pnpm run dev
```
### Mobile Setup (optional)
```bash
# In a new terminal, from the project root
cd mobile
flutter pub get
flutter run # against a running backend on :8000
```
See [`mobile/README.md`](mobile/README.md) for emulator setup and release builds.
### Start Celery Worker
```bash
# In a new terminal
cd backend
uv run celery -A crm worker --loglevel=INFO
```
### Access the Application
- **Frontend**: http://localhost:5173
- **API Documentation**: http://localhost:8000/swagger-ui/
- **Admin Panel**: http://localhost:8000/admin/
### Connect your AI agent (MCP)
Let Claude, Cursor, Codex, Gemini, or any MCP client work in your CRM:
1. In the app, go to **Settings → API Tokens** and create a personal access token (shown once).
2. Register the `bcrm-mcp` server in your AI client, passing `BCRM_BASE_URL` (your API host, e.g. `http://localhost:8000`) and `BCRM_TOKEN` (the token). The token page shows ready-to-paste config for each client.
3. Restart the client and start asking.
The agent authenticates **as you** and inherits your role, org and RLS scope — it can't see or do anything you can't. Full setup, the tool list, and the security model are in [`mcp_server/README.md`](mcp_server/README.md).
## Docker Setup
Run the full stack (backend, frontend, PostgreSQL, Redis, Celery) with a single command:
```bash
# Start all services (first run will build images)
# An admin user (admin@localhost / admin) is created automatically
cd Django-CRM
docker compose up --build
# (Optional) Load sample data
docker compose exec backend python manage.py seed_data --email admin@example.com
```
Once running:
- **Frontend**: http://localhost:5173
- **API / Swagger**: http://localhost:8000/swagger-ui/
- **PostgreSQL**: localhost:5432
- **Redis**: localhost:6379
Frontend at http://localhost:5173, API and Swagger UI at http://localhost:8000/swagger-ui/. See
the [Docker quick start guide](docs/getting-started/docker-quick-start.md) for the default admin
login, loading demo data, and what each service does.
### Daily workflow
## Documentation
```bash
docker compose up # start all services (code changes auto-reload)
docker compose down # stop all services
docker compose down -v # stop and delete all data (full reset)
```
The full documentation — setup, self-hosting, architecture, the REST API, and contributing — is
built from this repository's [`docs/`](docs/) with MkDocs Material and publishes to Read the Docs
at <https://bottlecrm.readthedocs.io/>. Four starting points:
### Running commands inside containers
```bash
docker compose exec backend python manage.py migrate
docker compose exec backend python -m pytest
docker compose exec backend python manage.py manage_rls --status
```
### Custom environment overrides
The default env vars live in `.env.docker` (committed). To override locally without touching git:
1. Copy `.env.docker` to `.env.docker.local` (gitignored)
2. Edit values as needed — set your own `SECRET_KEY` here, not in `.env.docker`
3. Rebuild/restart: `docker compose up --build`
Every service loads `.env.docker` first and then `.env.docker.local` if it exists, so
your overrides win. No edits to `docker-compose.yml` are needed.
## Project Structure
```
Django-CRM/
├── backend/ # Django REST API
│ ├── accounts/ # Accounts module
│ ├── cases/ # Cases module
│ ├── common/ # Shared models, utilities, RLS
│ ├── contacts/ # Contacts module
│ ├── invoices/ # Invoices module
│ ├── leads/ # Leads module
│ ├── opportunity/ # Opportunities module
│ ├── tasks/ # Tasks module
│ └── crm/ # Django project settings
├── frontend/ # SvelteKit frontend
│ ├── src/
│ │ ├── lib/ # Components, stores, utilities
│ │ └── routes/ # SvelteKit routes
│ │ ├── (app)/ # Authenticated app routes
│ │ └── (no-layout)/ # Auth pages (login, etc.)
│ ├── static/ # Static assets
│ └── Dockerfile # Frontend dev container
├── mobile/ # Flutter app (iOS + Android)
│ ├── lib/
│ │ ├── config/ # API configuration
│ │ ├── services/ # API + auth services
│ │ ├── models/ # Typed API response models
│ │ └── screens/ # Leads, deals, tickets, tasks, settings
│ ├── android/ # Android build config
│ └── ios/ # iOS build config
├── mcp_server/ # MCP server (bcrm-mcp) for AI agents
│ └── src/bcrm_mcp/ # FastMCP tools over the REST API (stdio transport)
├── docker/ # Docker support files
│ ├── backend/
│ │ └── entrypoint.sh # DB wait + migrate + runserver
│ └── postgres/
│ └── init-rls-user.sql # Creates non-superuser for RLS
├── Dockerfile # Backend / Celery image
├── docker-compose.yml # Full-stack dev environment
└── .env.docker # Docker env vars (dev defaults)
```
## Multi-Tenancy & Security
BottleCRM isolates tenant-scoped data at both the application and the database layer. Every
org-scoped table is protected by a PostgreSQL Row-Level Security policy keyed on the
`app.current_org` session variable, which middleware sets from the authenticated user's JWT — so
a query that escapes the ORM's org filter still returns zero rows rather than another tenant's
data.
Self-hosters should follow the [Row-Level Security setup guide](RLS_SETUP.md) to configure and
verify the policies. The most important rule: **the application's database user must not be a
PostgreSQL superuser**, because superusers bypass RLS entirely. Verify with
`python manage.py manage_rls --status`.
## Development
### Testing
```bash
cd backend
# Run all tests with coverage
pytest
# Run tests without coverage (faster)
pytest --no-cov -x
# Run a specific module's tests
pytest accounts/tests/
pytest leads/tests/test_leads_kanban.py
# Run tests matching a keyword
pytest -k "test_login"
# View coverage report in browser
open htmlcov/index.html
```
### Dev login (skip the Google OAuth flow)
For local development you can mint a JWT for any user without going through Google sign-in. The command refuses to run unless `DEBUG=True`, and there's no web endpoint — it's only reachable through `manage.py`:
```bash
cd backend
# Mint tokens for an existing user (no org bound — same as the OAuth flow)
uv run python manage.py devlogin aswin.1231@gmail.com
# Bind to a specific org so you skip the orgswitch step on first load
uv run python manage.py devlogin aswin.1231@gmail.com --org "MicroPyramid"
# Create the user on the fly (random password) if they don't exist yet
uv run python manage.py devlogin newdev@example.com --create
```
The command prints the access token, the refresh token, and (with `--org`) the org's UUID, plus a
ready-to-paste `localStorage.setItem(...)` snippet.
**Note:** `localStorage` alone is not enough to sign you in. The API client reads it for direct
fetches, but SvelteKit's server-side guard (`frontend/src/hooks.server.js`) reads the
`jwt_access`, `jwt_refresh` and `org` **cookies** — without them you'll still be redirected to
`/login`. In the devtools console on `http://localhost:5173`, run the printed snippet *and*:
```js
const oneYear = 60 * 60 * 24 * 365;
document.cookie = `jwt_access=${ACCESS_TOKEN}; path=/; max-age=${oneYear}; samesite=lax`;
document.cookie = `jwt_refresh=${REFRESH_TOKEN}; path=/; max-age=${oneYear}; samesite=lax`;
document.cookie = `org=${ORG_UUID}; path=/; max-age=${oneYear}; samesite=lax`;
```
Then reload. The root route `/` is the dashboard — there is no `/dashboard` path.
### Frontend Commands
```bash
cd frontend
# Type checking
pnpm run check
# Linting
pnpm run lint
# Formatting
pnpm run format
```
### Mobile Commands
```bash
cd mobile
flutter test # run tests
flutter analyze --no-fatal-infos # static analysis
dart format . # formatting
flutter build apk --release # Android release build
```
## API Documentation
API routes vary by module and are documented from the generated OpenAPI schema. After starting the backend, open the [interactive Swagger UI](http://localhost:8000/swagger-ui/) to explore the available endpoints and request formats.
## FAQ
**Is BottleCRM really free?**
Yes. It's MIT licensed with unlimited users, unlimited records and no feature paywall. You only
pay if you want [managed hosting, setup or custom development](https://bottlecrm.io/pricing).
**Can I self-host it?**
That's the primary way to run it. `docker compose up --build` gives you the full stack; see
[Docker Setup](#docker-setup). Your database, your server, your data.
**Is it a good Salesforce or HubSpot alternative?**
For teams that want to own their data and avoid per-seat pricing, yes. It covers leads, pipeline,
contacts, accounts, tickets, tasks and invoicing. See the [migration guide](https://bottlecrm.io/migration)
for moving existing data across.
**Can I run it as a multi-tenant SaaS?**
Yes — multi-tenancy is built in at the database layer via PostgreSQL Row-Level Security, not
bolted on. See [Multi-Tenancy & Security](#multi-tenancy--security).
**Is there a mobile app?**
Yes, a native [Flutter app](https://bottlecrm.io/features/mobile-app) for iOS and Android sharing
the same backend API.
**Can AI agents use it?**
Yes. The bundled [MCP server](mcp_server/README.md) lets Claude, Cursor, Codex, Gemini or any MCP
client work in your CRM under your own role and permissions.
More at the [full FAQ](https://bottlecrm.io/faq).
- **[Getting started](docs/getting-started/docker-quick-start.md)** — Docker and manual setup, first sign-in, demo data.
- **[Self-hosting](docs/self-hosting/requirements.md)** — requirements, production deployment, PostgreSQL and Row-Level Security, backups, security hardening.
- **[API reference](docs/api/conventions.md)** — authentication, conventions, and every documented endpoint.
- **[Contributing](docs/contributing/development-setup.md)** — local development setup, testing, code style, and how to open a pull request.
## Contributing

318
RLS_SETUP.md
View file

@ -1,305 +1,17 @@
# PostgreSQL Row-Level Security (RLS) Setup Guide
This guide explains how to set up PostgreSQL with Row-Level Security for BottleCRM multi-tenancy.
## Overview
RLS provides database-level tenant isolation. Every query automatically filters by organization, ensuring data protection even if application code misses a filter.
**Key Points:**
- RLS uses PostgreSQL session variable `app.current_org` to identify the current tenant
- **Superusers bypass RLS** - the application must use a non-superuser database account
- 24 tables are RLS-protected (see list below)
---
## 1. Create PostgreSQL Database
```sql
-- Connect as postgres superuser
sudo -u postgres psql
-- Create the database
CREATE DATABASE bottlecrm;
```
---
## 2. Create Application User (Non-Superuser)
**CRITICAL:** The application user must NOT be a superuser, otherwise RLS is completely bypassed.
```sql
-- Connect as postgres superuser
sudo -u postgres psql
-- Create non-superuser for the application
CREATE USER crm_user WITH PASSWORD 'crm_password';
-- Verify user is NOT a superuser
SELECT usename, usesuper FROM pg_user WHERE usename = 'crm_user';
-- Should show: crm_user | f
```
---
## 3. Grant Database Permissions
```sql
-- Connect to bottlecrm database
\c bottlecrm
-- Grant connection rights
GRANT CONNECT ON DATABASE bottlecrm TO crm_user;
-- Grant schema privileges (required for migrations to create/alter tables)
GRANT ALL ON SCHEMA public TO crm_user;
-- Grant table permissions (SELECT, INSERT, UPDATE, DELETE)
GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO crm_user;
-- Grant sequence permissions (for auto-increment fields)
GRANT ALL PRIVILEGES ON ALL SEQUENCES IN SCHEMA public TO crm_user;
-- Grant default privileges for future tables
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON TABLES TO crm_user;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON SEQUENCES TO crm_user;
```
> **Note:** Granting schema/table privileges does NOT weaken RLS. These control DDL operations (CREATE/ALTER/DROP), while RLS controls row-level access. Only `BYPASSRLS` attribute or superuser status bypasses RLS.
---
## 4. Configure Django Environment
Set these environment variables for Django:
```bash
export DBNAME=bottlecrm
export DBUSER=crm_user
export DBPASSWORD=crm_password
export DBHOST=localhost
export DBPORT=5432
```
Or in your `.env` file:
```env
DBNAME=bottlecrm
DBUSER=crm_user
DBPASSWORD=crm_password
DBHOST=localhost
DBPORT=5432
```
---
## 5. Run Migrations
Migrations automatically enable RLS on all protected tables:
```bash
cd backend
uv run python manage.py migrate
```
---
## 6. Verify RLS Setup
### Check RLS Status
```bash
python manage.py manage_rls --status
```
Expected output:
```
RLS Status:
Database user "crm_user" is not a superuser - RLS will be enforced
lead: ENABLED (forced)
accounts: ENABLED (forced)
contacts: ENABLED (forced)
...
Enabled: 24, Disabled: 0
```
### Verify User is Not Superuser
```bash
python manage.py manage_rls --verify-user
```
Expected output:
```
Verifying database user...
Current user: crm_user
Is superuser: False
Can create DB: False
Database user is properly configured for RLS
```
### Test RLS Isolation
```bash
python manage.py manage_rls --test
```
This tests that:
- Data is visible with correct org context
- Data is isolated between organizations
- No data is returned without context (fail-safe)
---
## 7. RLS-Protected Tables (24)
| Category | Tables |
|----------|--------|
| Core Business | `lead`, `accounts`, `contacts`, `opportunity`, `case`, `task`, `invoice` |
| Supporting | `comment`, `commentFiles`, `attachments`, `document`, `teams`, `activity`, `tags`, `address`, `solution` |
| Boards (Kanban) | `board`, `board_column`, `board_task`, `board_member` |
| Settings/Email | `apiSettings`, `account_email`, `emailLogs`, `invoice_history` |
| Security | `security_audit_log` |
---
## 8. Production Checklist
- [ ] Application database user is NOT a superuser
- [ ] All 24 tables show RLS ENABLED (forced)
- [ ] `manage_rls --test` passes
- [ ] `manage_rls --verify-user` confirms non-superuser
- [ ] Environment variables are securely stored (not in version control)
---
## Troubleshooting
### "RLS will be bypassed" Warning
The database user is a superuser. Create a non-superuser as shown in Step 2.
### Tables Show "disabled"
Run migrations again or check if the table has an `org_id` column:
```sql
SELECT column_name FROM information_schema.columns
WHERE table_name = 'your_table' AND column_name = 'org_id';
```
### Data Visible Without Context
If `manage_rls --test` shows data without context, the RLS policies may need updating:
```bash
python manage.py migrate common
```
### Permission Denied Errors
Grant permissions again (Step 3), especially after creating new tables:
```sql
GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA public TO crm_user;
GRANT USAGE, SELECT ON ALL SEQUENCES IN SCHEMA public TO crm_user;
```
---
## How RLS Works
### Session Context
Each request sets the org context via middleware:
```sql
SELECT set_config('app.current_org', '<org_uuid>', true);
```
### Policy Definition
Each table has two policies:
1. **Isolation Policy** (SELECT/UPDATE/DELETE):
```sql
CREATE POLICY org_isolation ON "lead"
FOR ALL
USING (org_id::text = (select NULLIF(current_setting('app.current_org', true), '')));
```
2. **Insert Check Policy**:
```sql
CREATE POLICY org_insert_check ON "lead"
FOR INSERT
WITH CHECK (org_id::text = (select NULLIF(current_setting('app.current_org', true), '')));
```
The `NULLIF(..., '')` ensures **no rows are returned** when context is not set (fail-safe).
### Force RLS
```sql
ALTER TABLE "lead" FORCE ROW LEVEL SECURITY;
```
This ensures RLS applies even to the table owner.
---
## Adding RLS to New Tables
1. Inherit from `BaseOrgModel` in your model
2. Add table name to `ORG_SCOPED_TABLES` in `common/rls/__init__.py`
3. Create migration:
```python
from django.db import migrations
from common.rls import get_enable_policy_sql
def enable_rls(apps, schema_editor):
if schema_editor.connection.vendor != 'postgresql':
return
with schema_editor.connection.cursor() as cursor:
cursor.execute(get_enable_policy_sql('your_table_name'))
class Migration(migrations.Migration):
dependencies = [...]
operations = [
migrations.RunPython(enable_rls, migrations.RunPython.noop),
]
```
---
## Celery Tasks
Background tasks don't go through middleware, so you must set context manually:
```python
from common.tasks import set_rls_context
@app.task
def my_task(data_id, org_id):
set_rls_context(org_id) # REQUIRED before any query
obj = MyModel.objects.get(id=data_id)
```
---
## Quick Reference
```bash
# Check RLS status
python manage.py manage_rls --status
# Verify non-superuser
python manage.py manage_rls --verify-user
# Test isolation
python manage.py manage_rls --test
```
This file has moved into the documentation site. See
[PostgreSQL and RLS](docs/self-hosting/postgresql-and-rls.md) for the full guide: creating the
non-superuser application role, how RLS policies are enabled by migrations, verifying isolation
with `manage_rls`, Celery's `set_rls_context`, and the mistakes that most often leave a deployment
unprotected.
Two related pages worth knowing about too: [Security hardening](docs/self-hosting/security-hardening.md#database-role)
covers the database role as part of a production checklist, and
[Troubleshooting](docs/self-hosting/troubleshooting.md) covers the specific symptoms of a
misconfigured role (empty admin lists, data visible across organizations, and more).
This file is kept only so existing links to `RLS_SETUP.md` keep resolving; it is not maintained
as a setup guide in its own right. Notably, the number of RLS-protected tables cited in earlier
versions of this file (24) was out of date — the current, verified count is documented on the
linked page rather than repeated here, precisely so this file can't drift out of sync again.

169
backend/common/tests/test_docs_api_endpoints.py Normal file
View file

@ -0,0 +1,169 @@
"""Every endpoint named in docs/api/ must exist in the generated OpenAPI schema.
The API reference is hand-written by choice, so nothing else stops it drifting
from the code. The current bottlecrm.io docs document `POST /api/auth/refresh/`;
the real route is `POST /api/auth/refresh-token/`. This test is what catches
that class of error.
The check runs one way only: documented endpoints must exist. It does not
require every schema path to be documented, because the reference is curated.
"""
import re
from pathlib import Path
from django.conf import settings
from drf_spectacular.generators import SchemaGenerator
DOCS_API_DIR = Path(settings.BASE_DIR).parent / "docs" / "api"
ALLOWLIST_PATH = DOCS_API_DIR / ".endpoint-check-allowlist"
HTTP_METHODS = ("GET", "POST", "PUT", "PATCH", "DELETE")
# Matches "GET /api/leads/{id}/" in prose, tables, and fenced code blocks.
# Stops at whitespace and at the characters that delimit markdown table cells,
# inline code, and quoted strings.
ENDPOINT_RE = re.compile(
r"\b(" + "|".join(HTTP_METHODS) + r")\s+(/api/[^\s`'\"|)\]<>]*(?:<[^>]*>[^\s`'\"|)\]<>]*)*)"
)
# Path parameters appear as {id} in the schema and as {id} or <str:pk> in docs.
PARAM_RE = re.compile(r"\{[^}]*\}|<[^>]*>")
def _normalise_path(path: str) -> str:
"""Collapse parameter names and trailing punctuation so docs and schema compare."""
path = path.split("?", 1)[0].split("#", 1)[0]
path = path.rstrip(".,;:")
if not path.endswith("/"):
path += "/"
return PARAM_RE.sub("{}", path)
def _extract_endpoints(text: str) -> list[tuple[str, str]]:
return [
(method, _normalise_path(raw)) for method, raw in ENDPOINT_RE.findall(text)
]
def _schema_operations() -> dict[str, set[str]]:
schema = SchemaGenerator().get_schema(request=None, public=True)
operations: dict[str, set[str]] = {}
for path, methods in schema["paths"].items():
key = _normalise_path(path)
operations.setdefault(key, set()).update(
method.upper() for method in methods if method.upper() in HTTP_METHODS
)
return operations
def _load_allowlist() -> set[str]:
if not ALLOWLIST_PATH.exists():
return set()
entries = set()
for line in ALLOWLIST_PATH.read_text().splitlines():
entry = line.split("#", 1)[0].strip()
if entry:
entries.add(entry)
return entries
def _find_missing(
documented: list[tuple[str, str, str]],
operations: dict[str, set[str]],
allowlist: set[str],
) -> list[str]:
"""documented is a list of (source_filename, method, normalised_path)."""
missing = []
for filename, method, path in documented:
if f"{method} {path}" in allowlist:
continue
if path not in operations:
missing.append(f"{filename}: {method} {path} — no such path in the API")
elif method not in operations[path]:
missing.append(
f"{filename}: {method} {path} — path exists but does not accept "
f"{method} (accepts {', '.join(sorted(operations[path]))})"
)
return missing
def test_docs_api_directory_exists():
# Without this, every check below passes vacuously if the directory is
# renamed or moved: Path.glob() on a missing directory yields nothing
# rather than raising, so the whole guard would silently stop guarding.
assert DOCS_API_DIR.is_dir(), f"{DOCS_API_DIR} is missing"
def test_normalise_path_collapses_parameters_and_trailing_slash():
assert _normalise_path("/api/leads/{id}/") == "/api/leads/{}/"
assert _normalise_path("/api/leads/<str:pk>/") == "/api/leads/{}/"
assert _normalise_path("/api/leads") == "/api/leads/"
def test_normalise_path_strips_query_strings():
# Documented filter examples carry a query string; the schema path does not.
# Without stripping, a correct page fails the check.
assert _normalise_path("/api/leads/?status=open") == "/api/leads/"
assert _normalise_path("/api/leads/?status=open&page=2") == "/api/leads/"
assert _normalise_path("/api/leads/{id}/?expand=account") == "/api/leads/{}/"
def test_extract_endpoints_reads_prose_tables_and_code_blocks():
text = (
"Call `GET /api/leads/` to list them.\n"
"| POST /api/leads/ | create |\n"
"```http\nDELETE /api/leads/{id}/\n```\n"
)
assert _extract_endpoints(text) == [
("GET", "/api/leads/"),
("POST", "/api/leads/"),
("DELETE", "/api/leads/{}/"),
]
def test_find_missing_reports_a_path_that_does_not_exist():
# The real-world case: /api/auth/refresh/ does not exist; the route is
# /api/auth/refresh-token/.
operations = {"/api/auth/refresh-token/": {"POST"}}
missing = _find_missing(
[("authentication.md", "POST", "/api/auth/refresh/")], operations, set()
)
assert len(missing) == 1
assert "no such path" in missing[0]
def test_find_missing_reports_a_method_the_path_does_not_accept():
operations = {"/api/leads/": {"GET", "POST"}}
missing = _find_missing([("leads.md", "DELETE", "/api/leads/")], operations, set())
assert len(missing) == 1
assert "does not accept DELETE" in missing[0]
def test_find_missing_is_empty_when_everything_resolves():
operations = {"/api/leads/": {"GET", "POST"}}
documented = [("leads.md", "GET", "/api/leads/"), ("leads.md", "POST", "/api/leads/")]
assert _find_missing(documented, operations, set()) == []
def test_find_missing_honours_the_allowlist():
operations = {"/api/leads/": {"GET"}}
documented = [("leads.md", "POST", "/api/planned/")]
assert _find_missing(documented, operations, {"POST /api/planned/"}) == []
def test_every_documented_api_endpoint_exists():
operations = _schema_operations()
allowlist = _load_allowlist()
documented = [
(path.name, method, endpoint)
for path in sorted(DOCS_API_DIR.glob("*.md"))
for method, endpoint in _extract_endpoints(path.read_text())
]
missing = _find_missing(documented, operations, allowlist)
assert not missing, (
"docs/api/ documents endpoints that do not exist:\n "
+ "\n ".join(missing)
+ "\n\nFix the docs, or add an entry to docs/api/.endpoint-check-allowlist "
"with a comment explaining why."
)

62
backend/common/tests/test_docs_build_excludes_internal.py Normal file
View file

@ -0,0 +1,62 @@
"""The docs/ directory holds internal specs and plans alongside published pages.
MkDocs copies every file under docs_dir into the built site, including files no
nav entry references. Without an explicit exclusion, implementation plans would
be published to a public documentation site. This test is the guard.
"""
import shutil
import subprocess
import tempfile
from pathlib import Path
import pytest
from django.conf import settings
REPO_ROOT = Path(settings.BASE_DIR).parent
MKDOCS_CONFIG = REPO_ROOT / "mkdocs.yml"
INTERNAL_PREFIXES = ("superpowers", "design")
def _build_site(destination: Path) -> None:
subprocess.run(
[
"mkdocs",
"build",
"--strict",
"-f",
str(MKDOCS_CONFIG),
"-d",
str(destination),
],
cwd=REPO_ROOT,
check=True,
capture_output=True,
text=True,
)
@pytest.mark.skipif(shutil.which("mkdocs") is None, reason="mkdocs not installed")
def test_built_site_contains_no_internal_artifacts():
with tempfile.TemporaryDirectory() as tmp:
destination = Path(tmp) / "site"
_build_site(destination)
leaked = sorted(
str(p.relative_to(destination))
for p in destination.rglob("*")
if p.is_file()
and p.relative_to(destination).parts[0] in INTERNAL_PREFIXES
)
assert not leaked, (
"Internal artifacts were published to the docs site:\n"
+ "\n".join(leaked)
+ "\n\nAdd the directory to `exclude_docs` in mkdocs.yml."
)
@pytest.mark.skipif(shutil.which("mkdocs") is None, reason="mkdocs not installed")
def test_strict_build_succeeds():
# --strict turns broken internal links and missing nav targets into errors,
# so this doubles as the link check for every page in the nav.
with tempfile.TemporaryDirectory() as tmp:
_build_site(Path(tmp) / "site")

94
backend/common/tests/test_docs_environment_variables.py Normal file
View file

@ -0,0 +1,94 @@
"""The environment-variable reference must match what the code actually reads.
The current bottlecrm.io quick-start tells self-hosters to set JWT_SIGNING_KEY.
No such variable is read anywhere in this codebase. A reader following that page
sets a variable that does nothing and believes their install is configured.
"""
import re
from pathlib import Path
import pytest
from django.conf import settings
REPO_ROOT = Path(settings.BASE_DIR).parent
SETTINGS_PATH = Path(settings.BASE_DIR) / "crm" / "settings.py"
ENV_DOCKER_PATH = REPO_ROOT / ".env.docker"
COMPOSE_PATH = REPO_ROOT / "docker-compose.yml"
REFERENCE_PATH = REPO_ROOT / "docs" / "reference" / "environment-variables.md"
# Matches os.environ.get("NAME") and os.getenv("NAME"), including calls whose
# argument sits on the following line.
SETTINGS_ENV_RE = re.compile(
r"os\.(?:environ\.get|getenv)\(\s*[\"']([A-Z][A-Z0-9_]*)[\"']"
)
# Matches the leading `NAME` cell of a markdown table row.
DOC_TABLE_ROW_RE = re.compile(r"^\|\s*`([A-Z][A-Z0-9_]*)`", re.MULTILINE)
# Matches NAME= in .env files and NAME: in compose environment blocks.
ENV_FILE_RE = re.compile(r"^\s*-?\s*([A-Z][A-Z0-9_]*)\s*[=:]", re.MULTILINE)
# Read by settings.py but intentionally left out of the reference table.
UNDOCUMENTED_BY_DESIGN = {
"ENV_TYPE", # internal dev/prod switch; not something a self-hoster sets
}
def _documented_variables(text: str) -> set[str]:
return set(DOC_TABLE_ROW_RE.findall(text))
def _variables_read_by_settings(text: str) -> set[str]:
return set(SETTINGS_ENV_RE.findall(text))
def _variables_in_env_files() -> set[str]:
found: set[str] = set()
for path in (ENV_DOCKER_PATH, COMPOSE_PATH):
if path.exists():
found |= set(ENV_FILE_RE.findall(path.read_text()))
return found
def test_documented_variables_parses_a_reference_table():
text = "| Variable | Default |\n| --- | --- |\n| `SECRET_KEY` | none |\n"
assert _documented_variables(text) == {"SECRET_KEY"}
def test_variables_read_by_settings_handles_both_call_styles():
text = 'a = os.environ.get("DBNAME", "x")\nb = os.getenv("DEBUG")\n'
assert _variables_read_by_settings(text) == {"DBNAME", "DEBUG"}
def test_variables_read_by_settings_handles_a_wrapped_call():
text = 'EMAIL_BACKEND = os.environ.get(\n "EMAIL_BACKEND", "smtp"\n)\n'
assert _variables_read_by_settings(text) == {"EMAIL_BACKEND"}
@pytest.mark.skipif(
not REFERENCE_PATH.exists(), reason="reference page not written yet"
)
def test_no_documented_variable_is_read_nowhere():
documented = _documented_variables(REFERENCE_PATH.read_text())
known = _variables_read_by_settings(SETTINGS_PATH.read_text()) | _variables_in_env_files()
invented = sorted(documented - known)
assert not invented, (
"These variables are documented but read nowhere in the codebase:\n "
+ "\n ".join(invented)
+ "\n\nThis is the JWT_SIGNING_KEY failure. Remove them, or point the "
"reference at the code that reads them."
)
@pytest.mark.skipif(
not REFERENCE_PATH.exists(), reason="reference page not written yet"
)
def test_every_variable_settings_reads_is_documented():
documented = _documented_variables(REFERENCE_PATH.read_text())
read = _variables_read_by_settings(SETTINGS_PATH.read_text())
missing = sorted(read - documented - UNDOCUMENTED_BY_DESIGN)
assert not missing, (
"crm/settings.py reads these variables but the reference does not list them:\n "
+ "\n ".join(missing)
+ "\n\nAdd a row to docs/reference/environment-variables.md, or add the "
"name to UNDOCUMENTED_BY_DESIGN with a comment saying why."
)

3
backend/pyproject.toml
View file

@ -94,6 +94,9 @@ dev = [
"pytest-django>=4.12.0",
"pytest-cov>=7.1.0",
]
docs = [
"mkdocs-material>=9.5.0",
]
[tool.hatch.build.targets.wheel]
packages = [

59
backend/scripts/generate_endpoint_index.py Normal file
View file

@ -0,0 +1,59 @@
"""Render every route in the OpenAPI schema to docs/api/endpoint-index.md.
The curated API pages are written by hand. This index is not: a hand-maintained
list of 350 operations would be wrong within a week. Re-run this script when
routes change:
cd backend && uv run python scripts/generate_endpoint_index.py
"""
import os
import sys
from pathlib import Path
import django
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "crm.settings")
os.environ.setdefault("SECRET_KEY", "generate-endpoint-index-only")
os.environ.setdefault("ADMIN_EMAIL", "admin@localhost")
django.setup()
from drf_spectacular.generators import SchemaGenerator # noqa: E402
OUTPUT = Path(__file__).resolve().parent.parent.parent / "docs" / "api" / "endpoint-index.md"
METHODS = ("get", "post", "put", "patch", "delete")
HEADER = """# Endpoint index
Every route the API exposes, generated from the OpenAPI schema. The curated
pages in this section explain the endpoints you are most likely to need; this
table exists so nothing is invisible.
Interactive versions of the same schema are served by a running backend at
`/swagger-ui/` and `/api/schema/redoc/`, with the raw document at `/schema/`.
!!! note
This page is generated by `backend/scripts/generate_endpoint_index.py`.
Edit that script, not this file.
| Path | Methods |
| --- | --- |
"""
def main() -> None:
schema = SchemaGenerator().get_schema(request=None, public=True)
rows = []
for path in sorted(schema["paths"]):
methods = sorted(
m.upper() for m in schema["paths"][path] if m in METHODS
)
if methods:
rows.append(f"| `{path}` | {', '.join(methods)} |")
OUTPUT.write_text(HEADER + "\n".join(rows) + "\n")
print(f"wrote {OUTPUT} — {len(rows)} paths")
if __name__ == "__main__":
main()

2413
backend/uv.lock
View file

File diff suppressed because it is too large Load diff

4
docs/api/.endpoint-check-allowlist Normal file
View file

@ -0,0 +1,4 @@
# Endpoints documented in docs/api/ that deliberately do not appear in the
# generated OpenAPI schema. One "METHOD /path/" per line, each with a comment
# explaining why. An empty allowlist is the expected steady state — if you are
# adding a line here, first check the endpoint is not simply misspelled.

160
docs/api/accounts.md Normal file
View file

@ -0,0 +1,160 @@
# Accounts
Routes in `backend/accounts/urls.py`, all served by `backend/accounts/views.py`. See
[Conventions](conventions.md) for pagination, filtering and response-shape rules assumed rather than
repeated here, and [Errors](errors.md) for the 404-vs-403 principle.
## List accounts
`GET /api/accounts/` (`AccountsListView.get`, `backend/accounts/views.py:312-314`) splits the result
into active and inactive accounts rather than open/closed — `Account` has no status field, only
`is_active` (`accounts/views.py:237-238`):
```json
{
"per_page": 10,
"page_number": [1],
"active_accounts": {"offset": 10, "open_accounts": [ { "...": "AccountSerializer" } ], "open_accounts_count": 37},
"closed_accounts": {"offset": null, "close_accounts": [ { "...": "..." } ], "close_accounts_count": 4},
"contacts": [{"id": "...", "first_name": "..."}],
"teams": [{"...": "..."}],
"countries": [["GB", "United Kingdom"]],
"industries": [["SOFTWARE", "SOFTWARE"]],
"tags": [{"...": "..."}],
"users": [{"id": "...", "user__email": "..."}],
"leads": [{"...": "LeadSerializer"}],
"status": ["active", "inactive"]
}
```
A non-admin caller only sees accounts they created or are assigned to
(`accounts/views.py:195-199`). Filters: `name`, `city`, `industry` (all `icontains`); `tags`,
`assigned_to` (repeatable, `__id__in`); `search` (matches `name` only — unlike leads' `search`, which
ORs across four fields); `created_at__gte`/`created_at__lte`; `cf_<key>` (`accounts/views.py:201-233`).
Every account in the list (and the detail view below) carries a **`rollups`** object —
`won_amount`, `won_count`, `open_pipeline`, `open_deal_count`, `overdue_amount`, `open_tickets`,
`first_won_on` — computed as correlated subqueries over that account's `Opportunity`, `Invoice` and
`Case` rows, never stored (`annotate_rollups`, `accounts/views.py:135-182`). `rollups` is `null`,
not zero-filled, on any serialization that skips the annotation (`AccountSerializer.get_rollups`,
`accounts/serializer.py:41-54`) — both the list and detail endpoints here apply it, so in practice you
will always see it populated through this page's endpoints.
## Create an account
`POST /api/accounts/` (`AccountsListView.post`, `accounts/views.py:322-400`) is validated by
`AccountCreateSerializer` (`accounts/serializer.py:189-257`); see [Fields](#fields). On success
(`200`):
```json
{"error": false, "message": "Account Created Successfully", "id": "<uuid>"}
```
Two validators worth knowing: **`name` must be unique per org, case-insensitively**
(`validate_name`, `accounts/serializer.py:199-210`) — `{"error": true, "errors": {"name": ["Account already exists with this name"]}}`
on a duplicate; and **`annual_revenue` cannot be negative**
(`validate_annual_revenue`, `:212-224`), rejected as `400` rather than reaching the database's own
`account_revenue_non_negative` check constraint as an unhandled `500`.
`contacts`, `tags`, `teams` and `assigned_to` are not part of the serializer — the view resolves them
from the request body via the shared `handle_m2m_assignment` / `get_or_create_tags` helpers
(`common/utils.py:437-510`), each scoped to `org=request.profile.org`
(`accounts/views.py:347-368`). An `account_attachment` multipart file is accepted the same way.
## Retrieve, update, delete
`GET /api/accounts/{id}/` (`AccountDetailView.get`, `accounts/views.py:604-704`) scopes the lookup to
the caller's org and treats a malformed id the same as a missing one — both `404`
(`get_object`, `:407-421`, catching `UUIDField`'s `ValidationError` that `get_object_or_404` alone does
not). `assert_account_access` (`:423-445`) then requires the caller to be an admin, the account's
creator, or one of its assignees, or the request is refused with `403`. The response nests the record
under `account_obj`, and also carries `contacts` (`Account.contacts`, not the `Contact.account`
foreign key — see [Contacts: linked accounts](contacts.md#fields)), `opportunity_list`, `cases`,
`tasks`, `invoices`, `emails`, `comments`, `attachments`, `teams`, `users`, `users_mention`,
`comment_permission`, `leads`, `countries`, `currencies`, `case_types`, `case_priority`,
`case_status`, `custom_field_definitions`, a fixed, two-value `status: ["open", "close"]`
(`accounts/views.py:692` — a static list, unrelated to `Account.is_active`, which is what actually
drives [List accounts](#list-accounts)'s active/inactive split), and two keys worth flagging
explicitly:
```json
{
"stages": [["PROSPECTING", "Prospecting"], ["QUALIFICATION", "Qualification"], ["CLOSED_WON", "Closed Won"]],
"sources": [["NONE", "NONE"], ["CALL", "CALL"], ["WEBSITE", "WEBSITE"], ["OTHER", "OTHER"]]
}
```
(`accounts/views.py:673-674`.) These are `Opportunity`'s `STAGES` and `SOURCES` enums — surfaced here
because the account detail page is where an "add opportunity against this account" form lives, not
because an account has stages or sources of its own. `sources` on this endpoint is the *uppercase*
vocabulary that includes `WEBSITE`; it is not `Lead.source`'s `LEAD_SOURCE` — see
[Opportunities: Stages](opportunities.md#stages) for the distinction, and
[Leads](leads.md#list-leads) for why conflating the two is a real failure mode with this API.
`PUT /api/accounts/{id}/` (`:453-575`) and `PATCH /api/accounts/{id}/` (`:763-871`) both call
`assert_account_access` **before** validating the request body — deliberately, so an unauthorized
caller learns they're forbidden before learning whether their payload was well-formed
(`accounts/views.py:456-459`). Both accept the same `contacts`/`tags`/`teams`/`assigned_to` M2M
handling and `account_attachment` upload as create. Success:
`{"error": false, "message": "Account Updated Successfully"}`.
`DELETE /api/accounts/{id}/` (`:582-597`) allows an admin or the account's own creator
(`request.profile.user_id != self.object.created_by_id` is the rejection condition, `:584-591`) —
anyone else gets `403`.
## Comments and attachments
`POST /api/accounts/{id}/` (`AccountDetailView.post`, `accounts/views.py:712-755`) adds a comment
and/or an `account_attachment` file to an existing account, gated by the same
`assert_account_access` rule as `GET`/`PUT`/`PATCH` (`:719`). The response echoes `account_obj`,
`attachments` and `comments`.
`PUT` / `PATCH` / `DELETE /api/accounts/comment/{id}/` (`AccountCommentView`,
`accounts/views.py:874-968`) is restricted to an admin or the comment's own author
(`request.profile == obj.commented_by` — correct, since `commented_by` is a `Profile` foreign key).
`DELETE /api/accounts/attachment/{id}/` (`AccountAttachmentView.delete`, `:971-1010`) has its lookup
correctly scoped to the caller's org
(`get_object_or_404(self.model, pk=pk, org=self.request.profile.org)`, `:989-991`, with malformed ids
also mapped to `404`) and its ownership check compares the right types
(`request.profile.user_id == self.object.created_by_id`, `:997`). That makes accounts one of five apps
in this API whose attachment-delete lookup is org-scoped — the same is true of contacts
(`contacts/views.py:1046`), cases (`cases/views.py:1119-1121`), tasks
(`tasks/views/task_views.py:975-977`) and invoices (`invoices/api_views.py:1821`). Only **leads and
opportunity** still have the unscoped version — see
[Leads: Comments and attachments](leads.md#comments-and-attachments) and
[Architecture: Permissions and roles](../architecture/permissions-and-roles.md#object-level-checks).
(One stale comment in `cases/views.py` claims tasks is still broken — it isn't; the fix landed and the
comment didn't get updated. Verified against `tasks/views/task_views.py:975-987` directly rather than
taken on the comment's word.)
## Fields
`AccountCreateSerializer.Meta.fields` (`accounts/serializer.py:226-249`) is what
`POST /api/accounts/`, `PUT /api/accounts/{id}/` and `PATCH /api/accounts/{id}/` accept. Only `name`
is required at the model level (`accounts/models.py:26`, no `blank=True`); every other field on this
serializer has `blank=True, null=True` or a default.
| Field | Type | Required | Notes |
| --- | --- | --- | --- |
| `name` | string | **required** | Unique per org, case-insensitive |
| `email` | string | optional | |
| `phone` | string | optional | |
| `website` | string | optional | `URLField` — validated as a URL, unlike `Lead.website` (see [Leads](leads.md#fields)) |
| `industry` | one of `INDCHOICES` | optional | e.g. `SOFTWARE` |
| `number_of_employees` | integer | optional | Non-negative (`PositiveIntegerField`) |
| `annual_revenue` | decimal | optional | Non-negative — checked in the serializer, not just the DB |
| `currency` | one of `CURRENCY_CODES` | optional | Defaults from the org's `default_currency` if omitted and `annual_revenue` is set |
| `address_line`, `city`, `state`, `postcode` | string | optional | |
| `country` | one of `COUNTRIES` | optional | |
| `description` | text | optional | |
| `is_active` | boolean | optional | Defaults `true` |
Not part of the serializer, but accepted in the same request and resolved by the view (see
[Create an account](#create-an-account)): `contacts`, `tags`, `teams`, `assigned_to` (each a list of
ids, org-scoped), and `account_attachment` (a multipart file).
`GET /api/accounts/` and `GET /api/accounts/{id}/` additionally return, but never accept as input:
`id`, `created_by`, `created_at`, `org` (nested), `country_display`, `account_attachment`, `cases`,
`tasks`, `opportunities`, `rollups` (see [List accounts](#list-accounts)), and `custom_fields`
(validated separately against the org's `CustomFieldDefinition` rows).

256
docs/api/authentication.md Normal file
View file

@ -0,0 +1,256 @@
# Authentication
This page documents the calls a client makes to sign in and stay signed in: what to send, what
comes back, and what each failure mode means. For the design behind these endpoints — token
lifetimes, claims, why rotation is implemented by hand, why personal access tokens and organization
API keys exist alongside the JWT — see [Architecture: Authentication](../architecture/authentication.md),
which this page does not repeat or contradict. For a guided walkthrough of picking a sign-in path
locally, see [First sign-in](../getting-started/first-sign-in.md).
All three interactive sign-in paths below — Google OAuth, Google ID token, and both magic-link
verify endpoints — mint tokens through the same `OrgAwareRefreshToken.for_user_and_org()` helper and
are public: `permission_classes = []`, `authentication_classes = []`
(`backend/common/views/auth_views.py`). None of them require an existing token to call.
## Google OAuth (web)
`POST /api/auth/google/callback/` (`GoogleOAuthCallbackView`,
`backend/common/views/auth_views.py:52-198`) completes a PKCE authorization-code exchange for the
SvelteKit frontend.
Request body:
```json
{
"code": "<authorization code from Google's redirect>",
"code_verifier": "<PKCE verifier the frontend generated>",
"redirect_uri": "<the redirect_uri used in the initial authorize request>"
}
```
`redirect_uri` is taken from this field, not from any server-side setting — see
[Architecture: Authentication](../architecture/authentication.md#organization-api-keys) for the
related, dead `GOOGLE_REDIRECT_URI` config, and [Google OAuth](../self-hosting/google-oauth.md) for
the setting that actually matters.
The view exchanges these with Google, decodes the returned ID token, and rejects the sign-in
outright if Google's own `email_verified` claim is not `true` — an unverified Google address never
reaches the database. On success:
```json
{
"access_token": "<jwt>",
"refresh_token": "<jwt>",
"user": {"id": "<uuid>", "email": "user@example.com"}
}
```
The token is minted with no org claim (`OrgAwareRefreshToken.for_user_and_org(user, None)`,
`:190`) — both Google flows always hand back an org-less token (the magic-link flows differ; see
[Magic links](#magic-links) below), so the client's next call is normally
[switching organization](#switching-organization).
Failure responses: `400` for a missing `code`/`code_verifier`/`redirect_uri`, an invalid ID token,
a missing email, or an unverified email; `502` if Google's token endpoint could not be reached at
all (`requests.RequestException`, `:110-114`); `403` if the resulting user account is deactivated.
## Google ID token (mobile)
`POST /api/auth/google/` (`GoogleIdTokenView`, `backend/common/views/auth_views.py:201-319`) is the
mobile equivalent: the client has already obtained a Google ID token itself and hands it over
directly, rather than a code the server exchanges.
Request body:
```json
{"idToken": "<google id token>"}
```
The server verifies it against Google (`google.oauth2.id_token.verify_oauth2_token`, checked
against `GOOGLE_CLIENT_ID`) and applies the same `email_verified` check as the web flow. On success:
```json
{
"JWTtoken": "<jwt access token>",
"refresh_token": "<jwt>",
"user": {"id": "<uuid>", "email": "user@example.com", "name": "user", "profileImage": ""},
"organizations": [{"id": "<uuid>", "name": "Acme", "role": "ADMIN"}]
}
```
Note the access token field here is `JWTtoken`, not `access_token` — the two Google flows do not
share a response shape, so a client that supports both cannot read the access token the same way
from each. This response also includes `organizations` directly (the caller's existing memberships),
which the web callback response above does not; a mobile client can offer org selection without an
extra `GET /api/auth/me/` round trip. Failure responses mirror the web flow: `400` for a missing or
invalid `idToken`, a missing email, or an unverified email; `403` for a deactivated account.
## Magic links
Passwordless sign-in, in two deliveries, across three endpoints:
`POST /api/auth/magic-link/request/` (`MagicLinkRequestView`,
`backend/common/views/auth_views.py:586-655`):
```json
{"email": "user@example.com", "delivery": "link"}
```
`delivery` is optional and defaults to `"link"`; the only other value is `"code"`. This endpoint
**always returns `200`** with the same body, regardless of whether the address has an account,
whether it failed validation (a malformed or disposable-domain email), or whether it was rate
limited (five tokens per address per hour) — every one of those cases falls through to the same
generic response:
```json
{"message": "If this email is valid, you will receive a sign-in link."}
```
This is deliberate — see [First sign-in](../getting-started/first-sign-in.md#magic-links) — and
means a client cannot distinguish "sent" from "rejected" from this response; there is no `400` to
handle here at all. A generated token expires after 10 minutes.
`POST /api/auth/magic-link/verify/` (`MagicLinkVerifyView`,
`backend/common/views/auth_views.py:658-768`) exchanges the token from a link-delivery email:
```json
{"token": "<64-character hex token from the emailed link>"}
```
`POST /api/auth/magic-link/verify-code/` (`MagicLinkVerifyCodeView`,
`backend/common/views/auth_views.py:771-899`) exchanges a code-delivery OTP instead:
```json
{"email": "user@example.com", "code": "123456"}
```
A code is checked under `select_for_update` so two concurrent verify attempts can't both observe
the same attempt count, and is burned (marked used) after 5 failed attempts even if the correct
code is submitted later. Both verify endpoints return the same success shape:
```json
{
"access_token": "<jwt>",
"refresh_token": "<jwt>",
"user": {
"id": "<uuid>",
"email": "user@example.com",
"profile_pic": "",
"is_active": true,
"organizations": [{"id": "<uuid>", "name": "Acme", "role": "ADMIN", "is_organization_admin": true, "has_sales_access": true, "has_marketing_access": true}]
}
}
```
with one addition: if the user has **any** active org membership, the token is minted bound to
one of them and the response also includes:
```json
{"current_org": {"id": "<uuid>", "name": "Acme"}}
```
This is not gated on having exactly one membership — `Profile.objects.filter(user=user,
is_active=True).first()` (`backend/common/views/auth_views.py:740,745-746` for verify, `:874,878-879`
for verify-code) picks whichever active profile sorts first, and `Profile`'s default ordering is
`("-created_at",)` (`backend/common/models.py:238`), so for a user in more than one org this binds
to their **most recently created** membership, not necessarily the one they intend to use next. A
client for a user who might belong to multiple orgs cannot assume magic-link verification always
leaves org selection to a later step — it has to check whether `current_org` is present and let the
user switch away from it if it's the wrong one, via
[switching organization](#switching-organization). If there is no active org membership at all, the
token comes back org-less, same as the OAuth flows.
Failure responses: `400` with `{"error": "Invalid request"}` if the request body itself fails
validation (missing token, malformed email, code not exactly 6 digits); `400` with
`{"error": "Invalid or expired link"}` / `{"error": "Invalid or expired code"}` if the token or code
doesn't match an unused, unexpired row; `403` for a deactivated account. Verifying creates the user
if the email has never signed in before, exactly like the OAuth flows do.
## Using an access token
Send the access token as `Authorization: Bearer <access_token>`. It is valid for one hour — see
[Token model](../architecture/authentication.md#token-model) for why, and what happens after it
expires.
`GET /api/auth/me/` (`MeView`, `backend/common/views/auth_views.py:322-336`) returns the signed-in
user and every organization they have an active profile in:
```json
{
"id": "<uuid>",
"email": "user@example.com",
"profile_pic": "",
"is_active": true,
"organizations": [
{"id": "<uuid>", "name": "Acme", "role": "ADMIN", "is_organization_admin": true, "has_sales_access": true, "has_marketing_access": true}
]
}
```
Two things worth knowing about this specific endpoint: it sets `authentication_classes =
[JWTAuthentication]` explicitly (`:327`), so unlike most endpoints it does not also accept a
personal access token or an organization API key — only a JWT works here. And its exact path is one
of a short list `GetProfileAndOrg` skips org-context resolution for entirely
(`backend/common/middleware/get_company.py:29-38`) — the middleware returns immediately without
trying to set `request.profile`/`request.org`, so this endpoint never requires an `org_id` claim.
That is what makes it work with the org-less token an org-less sign-in above produces, and exactly
what makes it useful for letting a freshly-authenticated client list organizations to choose from.
## Refreshing
`POST /api/auth/refresh-token/` (`OrgAwareTokenRefreshView`,
`backend/common/views/auth_views.py:339-448`) exchanges a refresh token for a new access/refresh
pair:
```json
{"refresh": "<refresh token>"}
```
```json
{"access": "<new access token>", "refresh": "<new refresh token>"}
```
Note the field names here — `access` and `refresh` — do not match `access_token`/`refresh_token`
used by the sign-in endpoints above; this is the one response on this page that uses the shorter
pair. Failure responses: `400` if `refresh` is missing; `401` if the token is expired, malformed,
or already blacklisted (`TokenError`); `403` if the user account has been deactivated since the
token was issued; `403` if the token carries an `org_id` the user no longer has an active profile
in. See [Refresh and rotation](../architecture/authentication.md#refresh-and-rotation) for what
happens to the old token on a successful call, and why.
## Switching organization
`POST /api/auth/switch-org/` (`OrgSwitchView`, `backend/common/views/auth_views.py:451-583`) moves
an authenticated caller to a different org and re-mints both tokens bound to it. Unlike most
endpoints it only requires `IsAuthenticated`, not org context, because it has to work for a token
that has none yet:
```json
{
"org_id": "<uuid of an org the caller has an active profile in>",
"refresh": "<optional: the refresh token being replaced>"
}
```
```json
{
"access_token": "<jwt bound to org_id>",
"refresh_token": "<jwt bound to org_id>",
"current_org": {"id": "<uuid>", "name": "Acme"},
"profile": {"id": "<uuid>", "role": "ADMIN", "is_organization_admin": true}
}
```
`refresh` is optional; when it is sent and belongs to the caller, it is blacklisted as part of the
switch so the org-less (or previous-org) token stops working immediately rather than staying valid
for the rest of its 14-day life — see
[Organization switching](../architecture/authentication.md#organization-switching) for why the
ownership check on that field matters. Failure responses: `400` if `org_id` is missing or not a
valid UUID; `403` if the caller has no active profile in the requested org.
## Local development
For local development against a running backend with `DEBUG=True`, the fastest way to obtain a
token pair is the `devlogin` management command rather than any of the endpoints above — it needs
no OAuth provider and no outbound email. It refuses to run at all unless `DEBUG` is `True`. See
[First sign-in: Local development](../getting-started/first-sign-in.md) for its exact invocation,
flags, and org-resolution behavior; this page's endpoints are what `devlogin` is a shortcut around,
and the tokens it prints work identically against every endpoint described above.

439
docs/api/cases.md Normal file
View file

@ -0,0 +1,439 @@
# Cases
Routes in `backend/cases/urls.py`. CRUD and comments/attachments live in `backend/cases/views.py`;
CSV import in `backend/cases/import_views.py` / `backend/cases/services/csv_import.py`; approvals in
`backend/cases/approval_views.py` and `backend/cases/approvals.py`; the knowledge base in
`backend/cases/solution_views.py`. Object-level authorization for the case itself is centralized in
`backend/cases/access.py`; for knowledge-base articles, `backend/cases/kb_access.py`. See
[Conventions](conventions.md) for pagination, filtering and response-shape rules assumed rather than
repeated here, and [Errors](errors.md) for the 404-vs-403 principle referenced below.
Cases are also the largest module in this codebase — Kanban pipelines, merge/unmerge, parent/child
trees, time tracking, escalation, routing, inbound email and analytics all live under
`/api/cases/...` too. This page covers only the endpoints this documentation task scopes: list,
create, detail, comments/attachments, approvals, the knowledge base, and CSV import.
## A case's read, write and delete rules differ on purpose
Before anything else: **do not collapse a case's access rules into one check.** `cases/access.py`
states three explicit, different rules, and its own module docstring explains why — before this
module existed the three verbs had drifted into three different answers by accident, and a watcher's
ticket showed up in their list but answered `403` when they opened it:
```
read admin · creator · assignee · watcher
write admin · creator · assignee
delete admin · creator
```
(`cases/access.py:20-22`.) This is the same "read wide, write narrow" principle
[Architecture: Permissions and roles](../architecture/permissions-and-roles.md#read-wide-write-narrow)
documents in general — that page is the place to read the reasoning; this page just applies it.
Every endpoint under [Retrieve, update, delete](#retrieve-update-delete) and
[Comments and attachments](#comments-and-attachments) calls one of
`assert_case_read_access`/`assert_case_write_access`/`assert_case_delete_access`
(`cases/access.py:90-113`), and the admin check itself is the two-spellings-of-admin form —
`profile.role == "ADMIN" or bool(getattr(profile, "is_admin", False))` (`cases/access.py:38-40`) —
that recurs throughout this codebase. **The [Approvals](#approvals) endpoints are the exception**:
`cases/approval_views.py` never imports `cases.access` at all, and none of its case-touching views
call any of the three functions above — see [Approvals](#approvals) for exactly what that means in
practice.
## List cases
`GET /api/cases/` (`CaseListView.get`, `cases/views.py:285-287`) returns one paginated list, ordered
`-created_at, -id` (`:169-174`) — a comment on the queryset explains the tiebreak exists because `-id`
alone is a random UUID and was, in practice, no ordering at all. Soft-deleted cases
(`is_active=False`) are excluded by default; an org admin can pass `?include_deleted=true` to see them
(`:176-180`). Merged duplicates (`merged_into` set, or `status="Duplicate"`) are excluded by default
too; `?show_merged=true` includes them (`:181-186`). A non-admin caller sees only cases where they are
creator, assignee, or watcher — the same `visible_cases_qs` the detail view's read check uses
(`:190-198`), so list and detail agree by construction.
```json
{
"cases_count": 42,
"offset": 10,
"cases": [ { "...": "CaseSerializer" } ],
"status": [["New", "New"], ["Assigned", "Assigned"], ["Pending", "Pending"], ["Closed", "Closed"], ["Rejected", "Rejected"], ["Duplicate", "Duplicate"]],
"priority": [["Low", "Low"], ["Normal", "Normal"], ["High", "High"], ["Urgent", "Urgent"]],
"type_of_case": [["Question", "Question"], ["Incident", "Incident"], ["Problem", "Problem"]],
"open_count": 12,
"urgent_count": 2,
"awaiting_first_reply": 5,
"accounts_list": [ { "...": "AccountSerializer" } ],
"contacts_list": [ { "...": "ContactSerializer" } ],
"users": [{"id": "...", "user__email": "..."}]
}
```
(`cases/views.py:222-263`.) `open_count`, `urgent_count` and `awaiting_first_reply` are counted over
the whole filtered queryset, not the page. `awaiting_first_reply` counts open cases
(`status` in `New`/`Assigned`/`Pending`) with no `first_response_at` yet — it is not the same thing as
an SLA breach, which depends on the org's business calendar and is computed per-row rather than in
this aggregate. `accounts_list`/`contacts_list` feed the create form's pickers — org-wide for an admin,
but narrowed to accounts/contacts the caller created or is assigned to for a non-admin, the same split
the case queryset itself gets (`:187-206`); either way they can be heavy, and `?slim=true` omits both
(`:254-256`).
Filters read directly from `request.query_params` (`apply_case_list_filters`, `cases/views.py:85-155`,
shared with `GET /api/cases/watching/`): `name` (contains), `status` (repeatable — a list applies
`status__in`, a single value applies exact match), `priority` (exact), `account` (exact id),
`case_type` (exact), `assigned_to` (repeatable id list), `tags` (id list), `search` (`name` or
`description`, contains), `created_at__gte`/`created_at__lte` (date range), `sla_breached=true`
(a wall-clock approximation of the SLA badge, Postgres-only raw SQL), `cf_<key>` (custom field equals),
and `ordering` — whitelisted to `created_at`, `-created_at`, `priority`, `-priority`, `id`, `-id`,
`name`, `-name` (`:71-82,150-153`); anything else is silently ignored rather than erroring.
## Create a case
`POST /api/cases/` (`CaseListView.post`, `cases/views.py:306-416`) accepts `multipart/form-data` (for
an optional `case_attachment` file) or JSON, validated by `CaseCreateSerializer`
(`cases/serializer.py:139-281`; see [Fields](#fields)). On success (`200`):
```json
{
"error": false,
"message": "Case Created Successfully",
"id": "<uuid>",
"cases_obj": { "...": "CaseSerializer" }
}
```
A validation failure returns `400` with `{"error": true, "errors": {...}}`. Two validators worth
knowing:
- **`account`, if given, must belong to the caller's org** — `validate_account` rejects a
cross-tenant id with `"No such account."` (`cases/serializer.py:151-165`); the comment on it notes
this closes both a cross-tenant *write* and a cross-tenant *read*, since the create response nests
the account back through `CaseSerializer` → `AccountSerializer`.
- **`name` must be unique per org, case-insensitive** (`validate_name`, `:232-244`).
`contacts`, `teams`, `assigned_to` and `tags` are **not** part of `CaseCreateSerializer` — the view
reads each from the request body after the serializer succeeds, resolved against
`org=request.profile.org` (`:332-386`). Unlike [Leads](leads.md#fields), which documents a real
per-field, per-verb inconsistency in how these four are *parsed*, cases parses all four identically on
every verb: each is `json.loads`'d when it arrives as a string (which happens whenever the same
request also uploads `case_attachment` as multipart) before the id list is extracted. `case_attachment`
itself is read straight off `request.FILES` (`:388-395`) — on `POST` only; see
[Retrieve, update, delete](#retrieve-update-delete) for why `PATCH` can't set it at all, and for how
*writing* these four fields (not just parsing them) differs sharply across verbs.
## Retrieve, update, delete
`GET /api/cases/{id}/` (`CaseDetailView.get`, `cases/views.py:612-731`) looks the case up scoped to
the caller's org (`get_case_or_404`, `cases/access.py:57-71` — a malformed UUID or a case in another
org both answer `404`, never `500`, matching [Errors](errors.md#not-found-versus-forbidden)). If the
case has been merged into another one, the response is a redirect hint instead of the case itself —
`{"redirect_to": "<uuid>", "merged_into": "<uuid>", "source_case_id": "<uuid>", "source_case_name": "..."}`
— unless the caller passes `?show_merged=true` (`:614-630`). **This merged-case branch returns before
any read-access check runs at all** — `assert_case_read_access` isn't called until `:633`, several
lines after the merged branch's own `return` at `:630` — so `source_case_name` is disclosed to any org
member who can supply a valid case id, including one with no read access to the case whatsoever, as
long as it happens to be a merged duplicate. On the ordinary, non-merged path, read access genuinely is
checked before the rest of the response body is built (`assert_case_read_access`, `:633` — the comment
on this line notes the previous order built the payload for a case the requester was about to be
refused); it's specifically the merged-redirect shortcut that skips the check entirely. The response nests
the record under `cases_obj`, alongside `attachments`, `comments` (public only), `internal_notes`,
`contacts`, `solutions`, `activities` (last 20), `email_messages` (last 50, inbound-email threads),
`merged_from_cases`, `custom_field_definitions`, `status`, `priority`, `type_of_case`,
`comment_permission` and `users_mention` (`:707-730`). `comment_permission` is computed with the same
`has_case_write_access` the comment-post endpoint enforces (`:642`), so the button a client shows and
the answer the server gives agree by construction — the comment on this line notes that used to not be
true (`comment_permission` was creator-or-admin while the write endpoint below also allowed assignees).
`PUT /api/cases/{id}/` (`:448-570`) and `PATCH /api/cases/{id}/` (`:823-942`) both call
`assert_case_write_access` before touching anything. **`account` cannot be changed by either verb.**
`CaseCreateSerializer.__init__` sets `self.fields["account"].read_only = True` whenever `self.instance`
is set (`cases/serializer.py:146-149`) — true for both update calls, never for create — so on `PUT`/
`PATCH` an `account` in the body is silently dropped before validation ever runs; `validate_account`
never executes and no error is returned. `PUT` is a non-partial serializer instantiation, so it still
requires `name`/`status`/`priority` the same as `POST` does; `PATCH` passes `partial=True`
(`:832`), so none of the three are required there — a `PATCH` may touch just one field.
Closing a case is validated as a *transition*, not just a target value: `CaseCreateSerializer.validate()`
(`cases/serializer.py:167-230`) requires `closed_on` whenever `status` is being set to `"Closed"` from
anything other than `"Closed"`, and — when an active `pre_close` `ApprovalRule` matches the case's
priority/case_type/team — requires an `Approval` row in state `approved` for that case and rule before
the close is allowed. This exists because `Case.clean()` states the same two rules
(`cases/models.py:180-249`) but `Case.save()` (`:264-283`) never calls `full_clean()`, so without the
serializer-level check a matching rule could be armed and a `PATCH {"status": "Closed"}` would still
return `200` and record zero approvals — proven live per the comment on `validate()`. Success on either
verb, identical literal string on both (`views.py:564,936`):
`{"error": false, "message": "Case Updated Successfully"}`.
**Three of `Case.clean()`'s parent-tree guards are dead code through the API — and one field that
looks like a partial re-implementation of the model isn't a re-implementation at all.**
`Case.clean()` (`cases/models.py:180-249`) states four rules about `parent`: a case cannot be its own
parent (`:222-223`); linking cannot create a multi-level cycle, walking the whole parent chain to check
(`:226-238`); the tree is capped at `PARENT_MAX_DEPTH = 3` levels (`:178,240-243`); and a case that is
itself `status="Duplicate"` cannot be given a parent at all (`:247-250`, the self-side of the merge
guard — distinct from `parent.status == "Duplicate"`, checked separately at `:245-246`). None of the
four run through any API write, because `Case.save()` (`:264-283`) never calls `full_clean()`. Of the
four, `CaseCreateSerializer.validate_parent()` (`cases/serializer.py:246-263`) independently
re-implements exactly two: the self-parent check, and refusing a parent whose own `status` is
`Duplicate`. **The cycle walk and the depth limit are simply absent from the API — not partially
covered, not approximated — and so is the "a Duplicate case cannot receive a parent" direction.**
`validate_parent()` also rejects a cross-org `parent`, but that check has no model-side counterpart at
all — `Case.clean()` never compares `org_id`, so this is a serializer-only addition, not a partial port
of anything the model already states. Net effect: a same-org, non-self parent assignment that would
create a longer cycle (A's parent is B, then B's parent is set to A), exceed three levels, or give a
merged case a parent, is accepted by every write endpoint today — reported here, not fixed, per this
documentation task's scope.
`DELETE /api/cases/{id}/` (`:586-593`) calls `assert_case_delete_access` — admin or creator only, no
assignee exception (see [above](#a-cases-read-write-and-delete-rules-differ-on-purpose)).
## Comments and attachments
Adding a comment or an attachment is the same `POST /api/cases/{id}/` route the detail endpoint uses,
just with a body (`CaseDetailView.post`, `cases/views.py:749-806`), gated by
`assert_case_write_access` (`:755`) — the same rule `comment_permission` on `GET` reports. Send
`comment` (text, and optionally `is_internal` — a truthy string or boolean) and/or `case_attachment`
(multipart file), either or both. A comment created with `is_internal=true` is an internal note, not
visible to a customer-facing surface; the response splits both back out as `comments` (public) and
`internal_notes` (`:794-805`), and both are returned to any caller with read access — `is_internal` is
a display split, not an access-control split on this endpoint.
Editing or deleting an *existing* comment goes through `PUT`/`PATCH`/`DELETE /api/cases/comment/{id}/`
(`CaseCommentView`, `cases/views.py:945-1078`) — restricted to an admin, the `is_organization_admin`
flag (exposed as `.is_admin`, `common/models.py:243-246` — org-scoped, not the platform-level
superuser flag), or the comment's own author (`request.profile == obj.commented_by`, e.g. `:982`).
`Comment.commented_by` is a `Profile` foreign key (`common/models.py:275-277`), so this comparison is
correct — unlike some `created_by` comparisons elsewhere in this codebase; see
[Architecture: Permissions and roles](../architecture/permissions-and-roles.md#object-level-checks).
**`PUT` cannot, in practice, edit a comment.** `.put()` builds `CommentSerializer(obj, data=params)`
with no `partial=True` (`:984`), and `CommentSerializer.Meta.fields` includes `object_id` and `org`
(`common/serializer.py:239-250`) — plain model fields, not declared `read_only` the way `content_type`
and `commented_by` are on the same serializer. A non-partial serializer requires every field that isn't
read-only, so a body of just `{"comment": "..."}` — everything the documented
`CaseCommentEditSwaggerSerializer` contract asks for — fails validation on the missing `object_id`/
`org` and answers `400`. Only `.patch()` (`partial=True`, `:1028`) can actually change a comment's
text. Separately, because `object_id` and `org` are genuinely writable on `PATCH`, a comment's own
author can send both in a `PATCH` body to repoint the comment at a different record — including one in
another org, or one the author cannot otherwise read. `Comment.clean()`
(`common/models.py:299-314`) checks that the new `org` matches the target record's `org`, and
`Comment.save()` calls `self.full_clean()` (`:316-318`), so a mismatched pair raises Django's
`ValidationError`, uncaught here, as an unhandled `500`; a *matching* cross-org pair (new `org` set to
the target record's real org) saves successfully, leaving a comment authored by a profile from a
different org sitting on that org's record. Neither of these two behaviors is fixed here — reported
per this documentation task's scope.
`DELETE /api/cases/attachment/{id}/` (`CaseAttachmentView.delete`, `cases/views.py:1098-1142`) is
org-scoped (`.filter(pk=pk, org=request.profile.org)`, `:1119-1121`) and its ownership check compares
`request.profile.user_id` to `self.object.created_by_id` — both `User` ids, correctly typed
(`:1128-1130`). The view's own docstring documents two defects that were live in this exact endpoint
before the fix (unscoped lookup; a `Profile`-to-`User` comparison that made the endpoint silently
admin-only) — both are fixed here today. Whether the equivalent endpoint in another app still has
either defect is that app's own question; verify against that app's current source rather than this
docstring, which itself goes stale — see
[Architecture: Permissions and roles](../architecture/permissions-and-roles.md#object-level-checks)
for the current, cross-app answer.
## Approvals
Two kinds of endpoint: admin-configured rules, and per-case requests/decisions against them
(`cases/approval_views.py`). A rule (`ApprovalRule`, `cases/approvals.py:47-116`) matches a case on
`trigger_event` (only `"pre_close"` exists today), and optionally `priority`, `case_type` and `team` —
the most-specific active match wins ties by most recent (`find_matching_rule`, `:119-136`).
- `GET /api/cases/approval-rules/` (`ApprovalRuleListCreateView.get`, `approval_views.py:87-103`) —
any org member; each rule row carries a `pending_count` computed from the `Approval` log.
- `POST /api/cases/approval-rules/` (`:105-133`) — admin only. The approver list and match-team are
each validated to belong to the caller's org before save (`:117-129`).
- `GET/PUT/DELETE /api/cases/approval-rules/{id}/` (`ApprovalRuleDetailView`, `:136-194`) — `GET` is
open to any member; `PUT`/`DELETE` are admin only. Deleting a rule with request history soft-disables
it (`is_active=False`) instead of a hard delete (`:183-192`), since `Approval.rule` is
`on_delete=PROTECT`.
- `POST /api/cases/{id}/request-approval/` (`CaseRequestApprovalView.post`, `:201-279`) — **no case
access check of any kind.** The view fetches the case with a bare
`get_object_or_404(Case, id=pk, org=org)` (`:209`) — org-scoped, but nothing from `cases.access`;
`approval_views.py` doesn't import that module at all. Any org member, including one refused the case
itself with a `403` on `GET /api/cases/{id}/`, can file an approval request against it. The `201`
response is `ApprovalSerializer(approval).data`, whose `case_summary`
(`cases/serializer.py:1002-1013`) returns the case's `name`, `status`, `priority` and account name —
the same read-around `CaseSolutionLinkView` was fixed for (see [Solutions](#solutions)), reintroduced
here. Also note: because the lookup is `get_object_or_404` rather than the `get_case_or_404` helper
every other case endpoint on this page uses, a malformed (non-UUID) `id` here isn't a clean `404` —
`get_object_or_404` only catches `Model.DoesNotExist`, not the `ValidationError`/`ValueError` a bad
UUID raises against `Case.id`, so it reaches the client as an unhandled `500`, the exact failure mode
`get_case_or_404` (`cases/access.py:57-71`) exists to prevent elsewhere on this page. Beyond
existence-and-org-scope, the request is validated against an explicit `rule_id` (which must actively
match the case) or, when omitted, the best `find_matching_rule` result. A second request against the
same case+rule while one is already `pending` is refused with `409`, not a duplicate row (`:244-258`).
- `GET /api/cases/approvals/` (`ApprovalInboxView.get`, `:282-341`) — the inbox, also with no
case-level access check: the queryset is `Approval.objects.filter(org=org)` (`:296`), not filtered
through `visible_cases_qs` or any case-access helper, so a member's `mine=false` inbox view can
include approval rows — and their `case_summary` — for cases that member has no access to.
`?state=` filters (default `pending`; `all` drops the filter), `?case=<id>` scopes to one case, and
`?mine=true` restricts to rows the caller can currently act on (see below) — which deliberately
excludes the caller's own requests, "mine to decide" rather than "mine to have filed" (`:318-332`).
Neither gap is fixed here — reported per this documentation task's scope.
**Approving and rejecting both enforce that the requester cannot be the approver, with no admin
exception.** `ApprovalApproveView.post` (`:354-407`) and `ApprovalRejectView.post` (`:410-475`) both
check, in order: the approval is still `pending` (else `400`); the caller is in the rule's approver
pool via `can_be_acted_on_by` (else `403`); and — separately —
`approval.requested_by_id == request.profile.id` (else `403`, "You cannot approve/reject your own
request; another approver must decide it.", `:382-390` and `:443-451`). The comment on both is explicit
that this holds even for "an admin who defaults into every rule's pool" — being an approver and being
the requester are mutually exclusive for a single decision, regardless of role.
`Approval.can_be_acted_on_by` (`cases/approvals.py:188-196`) itself is: explicitly listed in
`rule.approvers`, **or** `profile.role == rule.approver_role` — note this second clause checks only
`role`, not the `is_organization_admin` flag `is_org_admin` elsewhere in this codebase also treats as
admin, so a profile that is only an admin via that flag is not automatically an approver unless
explicitly added to `rule.approvers`.
`ApprovalSerializer.can_act` and `.is_own_request` (`cases/serializer.py:1045-1062`) mirror the approve
endpoint's checks exactly, so a client rendering the inbox from the list response gets the same answer
the action endpoints would give — both need `context={"request": request}` to resolve; without it they
default to `False`, which the comment on the serializer notes is the safe default rather than a bug.
`POST /api/cases/approvals/{id}/cancel/` (`ApprovalCancelView.post`, `:478-518`) is a **different**
rule from approve/reject: only the original requester or an admin may cancel (`:498-505`) — there is no
"not the requester" restriction here, because cancelling your own request is the ordinary case.
## Solutions
The knowledge base (`cases/solution_views.py`, `cases/kb_access.py`). Read access is org-wide by
design — every member may read every article, "a knowledge base whose articles are hidden from the
agents answering the tickets is not a knowledge base" (`cases/kb_access.py` module docstring) — but
write and release are two different, narrower rules:
```
write author · admin title, body, and moving draft ↔ reviewed
release admin status → approved, and publish / unpublish
delete author · admin
```
`release` is deliberately narrower than `write`: the review workflow (`draft → reviewed → approved`,
plus a separate `is_published` flag) only means something if the person who approves an article is not
its own author. `assert_solution_release_access` takes no article argument at all — being the author is
the one thing that must not grant it (`cases/kb_access.py:72-81`).
`GET /api/cases/solutions/` (`SolutionListView.get`, `cases/solution_views.py:88-142`) is the one
endpoint in this API that calls DRF's `get_paginated_response()` and returns its standard
`count`/`next`/`previous`/`results` envelope — [Conventions](conventions.md#pagination) names this
exact endpoint as the sole example. It adds one extra top-level key, `totals`
(`count`/`published`/`draft`/`reviewed`/`approved_unpublished`), computed over the *whole* knowledge
base rather than the current filter, so the four status cards stay meaningful under a `?status=`
filter (`:122-134`). Filters: `?status=`, `?is_published=` (accepts `true`/`1`/`yes`/`on`, not just the
literal string `"true"`), `?search=` (title or description, contains).
`POST /api/cases/solutions/` (`:150-167`) is open to any member for an ordinary draft, but is gated by
`assert_solution_release_access` the moment the payload would create an already-`approved` or
already-`is_published` article in one request (`_wants_release`, `:28-46`) — without this, an author
could write and publish an article in a single `POST`, bypassing the review workflow entirely.
`GET/PUT/PATCH/DELETE /api/cases/solutions/{id}/` (`SolutionDetailView`, `:170-237`): `GET` is open to
any member; `PUT`/`PATCH` require `assert_solution_write_access` (author or admin) plus, again,
`assert_solution_release_access` if the request would move the article to `approved` or flip
`is_published`; `DELETE` requires the same write rule (not a separate, narrower one — deleting your own
draft is ordinary tidying).
`POST /api/cases/solutions/{id}/publish/` and `POST /api/cases/solutions/{id}/unpublish/`
(`SolutionPublishView`/`SolutionUnpublishView`, `:240-295`) both require
`assert_solution_release_access`. Publishing additionally requires `status == "approved"` (`400`
otherwise); the comment on the unpublish view is explicit that pulling a bad answer down is the *same*
switch as giving it to customers and therefore gated the same way, even though the urgency argument
runs the other direction.
Reading which solutions are linked to a case is not a separate endpoint — the linked set is returned
as part of `GET /api/cases/{id}/` (`solutions` key, [above](#retrieve-update-delete)). Linking is
`POST /api/cases/{id}/solutions/` and unlinking is `DELETE /api/cases/{id}/solutions/{solution_id}/`
(`CaseSolutionLinkView`, `cases/views.py:1145-1208`), gated by `assert_case_write_access` on the
*case* — the comment on `.post` notes this closes a read-around: before this check, a member refused a
case with `403` could still link an article to it and then read the case's name, description, account
and contacts back out through that article's own `linked_cases`. The read side of the same gap is
closed in `cases/solution_serializers.py`'s `SolutionDetailSerializer.get_linked_cases`, which filters
the cases it returns through the same `visible_cases_qs` the case list and detail views use, rather
than returning every case the article happens to be linked to.
## CSV import
Two endpoints, gated to org admins or members with `has_sales_access` (`_can_import`,
`cases/import_views.py:23-31`) — anyone else gets `403`.
`POST /api/cases/import/preview/` (`CaseImportPreviewView`, `import_views.py:56-88`) reads a
`multipart/form-data` upload in a field named `file` (`.csv` extension required). The 5 MB cap is
checked against `upload.size` — Django's reported size for the uploaded file, available before
`.read()` is called — **not** the bytes actually read, and the check is skipped entirely when
`upload.size` is falsy (`if upload.size and upload.size > MAX_UPLOAD_BYTES`, `import_views.py:48-51`).
It then validates every row without writing anything. Row numbers are 1-based over *data* rows, not
file lines — the header itself is never counted, so the first row of data is `"row": 1`
(`enumerate(data_rows, start=1)` over `rows[1:]`, `services/csv_import.py:216-217`):
```json
{
"header_error": null,
"valid": [
{
"row": 1, "name": "Cannot log in", "status": "New", "priority": "High",
"description": "", "case_type": "Incident", "closed_on": null,
"account_id": "<uuid or null>", "contact_ids": [], "assigned_ids": [],
"team_ids": [], "tag_names": []
}
],
"errors": [{"row": 2, "field": "status", "message": "Status must be one of: New, Assigned, Pending, Closed, Rejected, Duplicate"}],
"summary": {"total": 2, "valid": 1, "invalid": 1}
}
```
`POST /api/cases/import/commit/` (`CaseImportCommitView`, `:91-125`) re-validates the same file and
writes every valid row inside one transaction — **if any row fails validation, nothing is written**
(`services/csv_import.py` `commit_rows`, "Don't write anything if any row failed; users must fix the
file first"). On success (`200`): `{"error": false, "created": 12, "ids": ["<uuid>", "..."]}`.
Required headers: `name`, `status`, `priority`. Recognized optional headers: `description`,
`case_type`, `account_name`, `contact_emails`, `assigned_emails`, `team_names`, `tags`, `closed_on`
(`REQUIRED_HEADERS`/`OPTIONAL_HEADERS`, `services/csv_import.py:36-46`) — an unknown header, or a
missing required one, fails the whole file with `header_error` before any row is checked.
`contact_emails`, `assigned_emails` and `team_names` accept `;`-separated multiple values per cell and
must resolve to an existing contact/active member/team in the caller's org, or the row errors
(`:396-438`). `status`/`priority`/`case_type` are matched case-insensitively against the same choice
sets [Fields](#fields) documents for the API proper. A case name must be unique per org — checked
against both existing cases and other rows earlier in the same file (`:338-355`). The file is capped
at 5,000 data rows (`MAX_ROWS`, `:49`).
## Fields
`CaseCreateSerializer.Meta.fields` (`cases/serializer.py:265-281`) is what `POST /api/cases/`,
`PUT /api/cases/{id}/` and `PATCH /api/cases/{id}/` accept. "Required" below means required on
`POST` and on `PUT` (which is non-partial); `PATCH` passes `partial=True`, so nothing is required
there — see [Retrieve, update, delete](#retrieve-update-delete).
| Field | Type | Required | Notes |
| --- | --- | --- | --- |
| `name` | string, max 64 | **required** (POST/PUT) | Unique per org, case-insensitive |
| `status` | one of `STATUS_CHOICE` | **required** (POST/PUT) | `New`, `Assigned`, `Pending`, `Closed`, `Rejected`, `Duplicate` |
| `priority` | one of `PRIORITY_CHOICE` | **required** (POST/PUT) | `Low`, `Normal`, `High`, `Urgent` |
| `case_type` | one of `CASE_TYPE` | optional | `Question`, `Incident`, `Problem` |
| `closed_on` | date | **required when closing** | See [Retrieve, update, delete](#retrieve-update-delete); optional otherwise |
| `description` | text | optional | |
| `is_active` | boolean | optional | Defaults `true`; soft-delete flag, hidden from `GET /api/cases/` by default |
| `account` | uuid | optional on create; **write-once** | Must belong to the caller's org; silently `read_only` on `PUT`/`PATCH` — see [Retrieve, update, delete](#retrieve-update-delete) |
| `custom_fields` | object | optional | Validated against the org's `CustomFieldDefinition` rows |
| `parent` | uuid | optional | Same-org, non-self, parent not `Duplicate`; see the dead model-level guards [above](#retrieve-update-delete) |
| `is_problem` | boolean | optional | Defaults `false`; marks an ITIL "problem" (umbrella) ticket |
| `org` | — | **read-only** | Server-derived from `request.profile.org`; listed in `Meta.fields` but not writable |
Not part of the serializer, but accepted in the same request body and resolved by the view (each a
list of ids, org-scoped): `contacts`, `teams`, `assigned_to`, `tags`. Parsing is identical on every
verb (see [Create a case](#create-a-case)), but *writing* is not, and the difference is a real
data-loss trap: `POST` only ever adds. `PUT` calls `.clear()` on all four M2Ms **unconditionally**,
before checking whether the request even mentioned them (`views.py:485,501,515,531`) — so a `PUT`
that omits `assigned_to` from the body unassigns every assignee on the case, not just leaves them
unchanged. `PATCH` clears a given M2M only when its key is present in the body at all
(`if "assigned_to" in params:`, `:903`, and identically for the other three) — omitting a key on
`PATCH` genuinely leaves it untouched. `case_attachment` (a multipart file) is accepted on `POST` and
`PUT` (`views.py:388,545`) but **not on `PATCH`** — `CaseDetailView.patch` never reads
`request.FILES` at all, so a multipart `PATCH` carrying `case_attachment` silently drops the file with
no error.
`GET /api/cases/{id}/` and `GET /api/cases/` additionally return, but never accept as input: `id`,
`created_by`, `created_at`, `escalation_count`, `last_escalation_fired_at`, the SLA fields
(`sla_first_response_hours`, `sla_resolution_hours`, `first_response_at`, `resolved_at`,
`sla_paused_at`, `first_response_sla_deadline`, `resolution_sla_deadline`,
`is_sla_first_response_breached`, `is_sla_resolution_breached`), `parent_summary`, `child_count`, and
`time_summary` (`CaseSerializer.Meta.fields`, `cases/serializer.py:97-136`).

207
docs/api/contacts.md Normal file
View file

@ -0,0 +1,207 @@
# Contacts
Routes in `backend/contacts/urls.py`: `backend/contacts/views.py` for CRUD and comments/attachments,
`backend/contacts/import_views.py` and `backend/contacts/services/csv_import.py` for CSV import. See
[Conventions](conventions.md) for pagination, filtering and response-shape rules assumed rather than
repeated here.
## Two links to an account, not one
Before anything else: **a contact can be joined to an account two different ways**, and they are
independent. `Contact.account` is a foreign key the model itself calls "Primary account this contact
belongs to" (`contacts/models.py:71-78`); `Account.contacts` is a separate many-to-many
(`accounts/models.py:63-65`). Nothing in the database keeps them in sync — setting one does not set
the other by itself. The view layer bridges them on write: `link_primary_account(contact)`
(`contacts/views.py:48-65`) adds the contact to `contact.account.contacts` whenever `account` is set
via `POST /api/contacts/`, `PUT /api/contacts/{id}/` or `PATCH /api/contacts/{id}/` — but clearing the
FK later does **not** remove the M2M membership, on purpose (a person can be "no longer the primary
contact" without having "left the company" — `contacts/views.py:59-62`).
Both links are exposed on every `ContactSerializer` response, deliberately, because reading only one
tells the wrong story depending on the data:
```json
{
"account": "<uuid or null>",
"account_detail": {"id": "<uuid>", "name": "Acme Inc"},
"linked_accounts": [{"id": "<uuid>", "name": "Acme Inc"}]
}
```
`account_detail` resolves the FK (`contacts/serializer.py:28-39`); `linked_accounts` is the M2M
membership (`:41-56`). The account detail page's contact list is built from `Account.contacts` (the
M2M), not the FK — so a contact whose `account` FK is set but was never added to that account's
`contacts` (see [CSV import](#csv-import) below for exactly this case) will not appear there even
though `account_detail` on the contact itself looks correct.
## List contacts
`GET /api/contacts/` (`ContactsListView.get`, `contacts/views.py:199-201`) returns a single
paginated list — unlike leads and accounts, it is not split into two sections. It still doesn't call
DRF's `get_paginated_response()` though — [Conventions](conventions.md#pagination) names
`GET /api/cases/solutions/` as the only endpoint in this API that does. The view hand-builds `count`
and `results` itself (`contacts/views.py:166-167`), with no `next`/`previous`, alongside a legacy set
of keys kept for backwards compatibility (`:168-171`):
```json
{
"count": 41,
"results": [ { "...": "ContactSerializer" } ],
"per_page": 10,
"page_number": [1],
"contacts_count": 41,
"offset": 10,
"contact_obj_list": [ { "...": "same rows as results" } ],
"countries": [["GB", "United Kingdom"]],
"users": [{"id": "...", "user__email": "..."}]
}
```
`active_count` and `inactive_count` (`contacts/views.py:147-148`) are computed over the full filtered
queryset *before* any `is_active` filter is applied, so a client can show "12 active, 3 inactive" while
also asking for only one of the two via `?is_active=true`/`?is_active=false`
(`:149-150`) — the only exact-match boolean filter on this endpoint; every other filter here is a
substring match: `name` (`first_name` or `last_name`), `city`, `phone`, `email`, `search`
(`first_name`, `last_name`, `email` or `phone`), plus `assigned_to`/`tags` (repeatable id lists),
`created_at__gte/lte`, and `cf_<key>` (`contacts/views.py:90-139`). A non-admin caller only sees
contacts assigned to them or created by them (`:83-87`).
## Create a contact
`POST /api/contacts/` (`ContactsListView.post`, `contacts/views.py:218-311`) is validated by
`CreateContactSerializer` (`contacts/serializer.py:102-174`); see [Fields](#fields). On success
(`200`):
```json
{"error": false, "message": "Contact created Successfuly", "id": "<uuid>"}
```
(The message string is spelled exactly that way in the response — `Successfuly`, not `Successfully`;
document your integration tests against the real string, not the expected one.) Two validators worth
knowing: **`email` must be unique per org, case-insensitive**, checked only when supplied
(`validate_email`, `contacts/serializer.py:129-145`); and **`account`, if given, must belong to the
caller's org** — `validate_account` rejects a UUID from any other org's `Account` table with
`"No such account."` rather than silently attaching a cross-tenant record
(`contacts/serializer.py:114-127`). `teams`, `assigned_to` and `tags` are not part of the serializer
either — the view resolves each from the request body, tolerating either a JSON array or a
JSON-encoded string of ids (or of `{"id": "..."}` objects), consistently across all three fields and
across `POST` (`contacts/views.py:247-284`), `PUT` (`:478-518`) and `PATCH` (`:832-876`) alike —
contacts does not have the per-field, per-verb inconsistency [Leads](leads.md#fields) documents for
the equivalent fields there. `contact_attachment` is accepted as a multipart file.
## Retrieve, update, delete
`GET /api/contacts/{id}/` (`ContactDetailView.get`, `contacts/views.py:565-654`) org-scopes the
lookup and maps a malformed id to `404` the same way accounts does
(`get_object`, `:318-332`). `assert_contact_access` (`:334-361`) is the broadest access rule in this
group of four pages: an admin, the contact's creator, one of its assignees, **or anyone assigned to
an account the contact is linked to by either route** (FK or M2M — `account_ids`, `:363-369`) may
read, update or comment on the contact; anyone else gets `403`. The response nests the record under
`contact_obj` and also returns `address_obj` (the flat address fields, echoed as their own object),
`countries` (`contacts/views.py:608`), `comments`, `attachments`, `assigned_data`, `tasks`,
`users_mention`, `custom_field_definitions`, and three computed, capped lists: `opportunities`
(up to 10, via `Opportunity.contacts`),
`cases` (up to 10, via `Case.contacts`), and `colleagues` (up to 8 — other contacts sharing either
account link, matched on the account relationship rather than the free-text `organization` field,
which frequently names a different company — `contacts/views.py:371-424`).
`PUT /api/contacts/{id}/` (`:441-544`) and `PATCH /api/contacts/{id}/` (`:793-881`) both call
`link_primary_account` after saving (`:477`, `:830`), so changing `account` through either verb keeps
`Account.contacts` in step the same way create does. Success:
`{"error": false, "message": "Contact Updated Successfully"}`.
`DELETE /api/contacts/{id}/` (`:670-691`) is narrower than read/write access on purpose — no assignee
exception, only an admin or the contact's own creator may delete
(`self.request.profile.user_id != self.object.created_by_id`, `:676-679`).
## CSV import
Two endpoints, both gated to org admins or members with `has_sales_access`
(`_can_import`, `contacts/import_views.py:23-31`) — an ordinary member gets `403`:
`POST /api/contacts/import/preview/` (`ContactImportPreviewView`, `import_views.py:64-96`) reads a
`multipart/form-data` upload in a field named `file` (must end in `.csv`, capped at 5 MB, checked
against the actual bytes read rather than the client-supplied `Content-Length`,
`import_views.py:34-61`), validates every row without writing anything, and returns:
```json
{
"header_error": null,
"valid": [
{
"row": 2, "first_name": "Ada", "last_name": "Lovelace", "email": "ada@example.com",
"phone": "2025551234", "organization": null, "title": null, "department": null,
"do_not_call": false, "linkedin_url": null, "address_line": null, "city": null,
"state": null, "postcode": null, "country": null, "description": null,
"account_id": "<uuid or null>", "assigned_ids": [], "team_ids": [], "tag_names": []
}
],
"errors": [{"row": 3, "field": "email", "message": "'not-an-email' is not a valid email"}],
"summary": {"total": 2, "valid": 1, "invalid": 1}
}
```
`POST /api/contacts/import/commit/` (`ContactImportCommitView`, `:99-133`) re-validates the same file
and writes inside one transaction (`commit_rows` / `_commit_validated`,
`contacts/services/csv_import.py:634-714`) — if any row fails validation, **nothing is written**
(`:650-657`); a concurrent conflict (e.g. a duplicate email created by another request between preview
and commit) rolls the whole batch back and reports `400` rather than partially importing
(`:659-672`). On success (`200`):
```json
{"error": false, "created": 12, "ids": ["<uuid>", "..."]}
```
The CSV must have a header row with `first_name` and `last_name` at minimum; the full list of
recognized optional headers is `email`, `phone`, `organization`, `title`, `department`,
`do_not_call`, `linkedin_url`, `address_line`, `city`, `state`, `postcode`, `country`,
`description`, `account_name`, `assigned_emails`, `team_names`, `tags`
(`REQUIRED_HEADERS`/`OPTIONAL_HEADERS`, `csv_import.py:46-66`); an unrecognized header or a missing
required one fails the whole file with `header_error` before any row is checked. `assigned_emails`
and `team_names` accept `;`-separated multiple values per cell. Duplicates are rejected per org: email
and phone (normalized) are always hard errors; a matching first+last name is only a hard error when
the row supplies **neither** an email nor a phone to disambiguate it from an existing contact
(`csv_import.py:8-19`, `408-514`). The file is capped at 5,000 data rows (`MAX_ROWS`, `:68`).
**A row with `account_name` sets only the `account` foreign key, not the `Account.contacts`
membership.** `_commit_validated` creates each `Contact` with `account_id=vr.account_id` directly
(`csv_import.py:681-700`) and never calls `link_primary_account` — unlike `POST /api/contacts/` and
the `PUT`/`PATCH` update endpoints, which always do (see [Two links to an account](#two-links-to-an-account-not-one)).
A contact imported this way resolves correctly under `account_detail`, but will not show up on that
account's contact list (`Account.contacts`, the M2M) until something else adds it there. This looks
like an oversight rather than a documented design choice — it is reported here, not fixed, per this
documentation task's scope.
## Fields
`CreateContactSerializer.Meta.fields` (`contacts/serializer.py:147-174`) is what
`POST /api/contacts/`, `PUT /api/contacts/{id}/` and `PATCH /api/contacts/{id}/` accept. Unlike
`Lead.first_name`/`last_name`, `Contact.first_name` and `Contact.last_name` have neither
`blank=True` nor a serializer override (`contacts/models.py:19-20`) — **both are genuinely
required**.
| Field | Type | Required | Notes |
| --- | --- | --- | --- |
| `first_name` | string | **required** | |
| `last_name` | string | **required** | |
| `email` | string | optional | Unique per org, case-insensitive, when supplied |
| `phone` | string | optional | |
| `organization` | string | optional | Free text — may name a different company than `account` links to |
| `title` | string | optional | |
| `department` | string | optional | |
| `do_not_call` | boolean | optional | Defaults `false` |
| `linkedin_url` | string | optional | Validated as a URL |
| `address_line`, `city`, `state`, `postcode` | string | optional | |
| `country` | one of `COUNTRIES` | optional | |
| `description` | text | optional | |
| `account` | uuid | optional | Must belong to the caller's org; see [Two links to an account](#two-links-to-an-account-not-one) |
| `is_active` | boolean | optional | Defaults `true` |
Not part of the serializer, but accepted in the same request and resolved by the view (see
[Create a contact](#create-a-contact)): `teams`, `assigned_to`, `tags` (each a list of ids,
org-scoped), and `contact_attachment` (a multipart file).
`GET /api/contacts/` and `GET /api/contacts/{id}/` additionally return, but never accept as input:
`id`, `created_by`, `created_at`, `updated_at`, `org` (nested), `account_detail`, `linked_accounts`
(both described above), `contact_attachment`, and `custom_fields` (validated separately against the
org's `CustomFieldDefinition` rows).

202
docs/api/conventions.md Normal file
View file

@ -0,0 +1,202 @@
# Conventions
This page describes patterns that hold across most of the API — how the base URL is put together,
how a client authenticates, how list endpoints paginate and filter, and what a response body looks
like. It is deliberately drawn from the code that is actually there (a real pagination class, a
real list view, the real `REST_FRAMEWORK` settings) rather than from what a typical DRF API "should"
look like — several of these conventions are less uniform than that would suggest, and the
differences matter to anyone integrating against them.
## Base URL
Every endpoint in this section, and every endpoint in [Endpoint index](endpoint-index.md), is
mounted under `/api/` at the Django root (`backend/crm/urls.py`: `path("api/",
include("common.app_urls", ...))`), which in turn includes `backend/common/urls.py` — the file most
of this section's page references point at — at the same, empty prefix. Individual apps are mounted
under their own segment from there, for example `path("leads/", include("leads.urls", ...))`, so a
lead endpoint is `/api/leads/…`.
Locally this is `http://localhost:8000/api/` (the default in the Docker quick start's
`.env.docker`, and what `PUBLIC_DJANGO_API_URL` points the frontend at — see
[Docker quick start](../getting-started/docker-quick-start.md)). In a self-hosted deployment it is
whatever host you put `DOMAIN_NAME` / `ALLOWED_HOSTS` on, still under `/api/`.
## Authentication header
Three credential types authenticate a request, tried by `common.middleware.get_company.GetProfileAndOrg`
in this order and, independently, by the DRF authentication classes in `DEFAULT_AUTHENTICATION_CLASSES`
(`backend/crm/settings.py`, `common.pat_auth.PATAuthentication`,
`rest_framework_simplejwt.authentication.JWTAuthentication`, `common.external_auth.APIKeyAuthentication`):
```
Authorization: Bearer <jwt-access-token> # interactive sign-in (see authentication.md)
Authorization: Bearer bcrm_pat_... # personal access token
Token: bcrm_pat_... # personal access token, alternate header
Token: <org-api-key> # organization API key
```
A personal access token is recognized by its `bcrm_pat_` prefix and can be presented either as a
bearer token or in the `Token` header (`backend/common/pat_auth.py`, `_extract_raw`); an organization
API key is only ever read from `Token` (`backend/common/external_auth.py`,
`APIKeyAuthentication.authenticate`). See [Authentication](authentication.md) for how a JWT is
obtained and [Tokens and API keys](tokens-and-api-keys.md) for the other two. Which identity a
request authenticates as, and which org it is scoped to, is never something the request supplies
directly — it is derived from whichever of these three the server can validate.
One thing the generated OpenAPI schema does *not* reflect accurately: a large minority of
operations — 112 of the schema's 350, about a third — declare an `org` header parameter
(`organization_params_in_header`, defined in `backend/common/swagger_params.py` and copied into the
`swagger_params.py` of six other apps: `leads`, `cases`, `opportunity`, `contacts`, `accounts` and
`tasks`). Nothing in the
request-handling code reads a header by that name — org context comes only from the JWT's `org_id`
claim, the PAT's owning profile, or the API key's associated org, as above. Sending an `org` header
has no effect; do not rely on it.
## Pagination
List endpoints paginate with `rest_framework.pagination.LimitOffsetPagination`
(`DEFAULT_PAGINATION_CLASS` in `backend/crm/settings.py`), using the query parameters `limit` and
`offset`. The default page size is `PAGE_SIZE = 10` from the same settings block; there is no
`max_limit` set on the pagination class, so a caller can request a larger page and get it.
That default pagination class is not applied automatically, though — there is no
`generics.ListAPIView` anywhere in this codebase. Every list endpoint is a hand-written `APIView`
that mixes `LimitOffsetPagination` in directly and calls `self.paginate_queryset(...)` itself, for
example:
```python
class LeadListView(APIView, LimitOffsetPagination):
...
def get_context_data(self, **kwargs):
...
results_leads_open = self.paginate_queryset(
queryset_open.distinct(), self.request, view=self
)
```
(`backend/leads/views/lead_views.py:51`, `:167`.) The same shape repeats in `common/views/user_views.py`,
`common/views/team_views.py`, `common/views/document_views.py`, `common/views/tags_views.py`,
`cases/views.py`, `contacts/views.py`, `accounts/views.py`, `opportunity/views/opportunity_views.py`,
`tasks/views/task_views.py` and `invoices/api_views.py`, among others.
Because pagination is applied by hand, the response envelope is a per-endpoint choice, not a
guarantee. The common case in this codebase is that a view builds its own shape rather than calling
DRF's `get_paginated_response()`: `GET /api/leads/` reports pagination per section, since the list
is split into open and closed leads:
```python
context["open_leads"] = {
"leads_count": self.count, # LimitOffsetPagination's own running total
"open_leads": open_leads, # this page's serialized results
"offset": offset, # offset to request for the next page, or None on the last page
}
```
(`backend/leads/views/lead_views.py:180-184`.) But `get_paginated_response()` — and with it, DRF's
standard `count`/`next`/`previous`/`results` shape — does appear: `GET /api/cases/solutions/`
(`SolutionListView`, `backend/cases/solution_views.py:56`, routed at `backend/cases/urls.py:59`)
calls it directly and adds one extra key on top:
```python
response = self.get_paginated_response(serializer.data)
response.data["totals"] = counts
return response
```
(`backend/cases/solution_views.py:140-142`.) So both shapes are real. Treat the exact pagination
field names, and whether the standard DRF envelope or a hand-rolled one is used, as per-endpoint —
`limit`/`offset` as query parameters are consistent, but where the resulting count, next-offset and
results end up in the response body is not; check the endpoint you're calling.
## Filtering and search
There is no shared filter backend — `DEFAULT_FILTER_BACKENDS` is not set in `REST_FRAMEWORK`, and
no view uses `django_filters`. Each list view reads `request.query_params` directly inside its own
`get`/`get_context_data` and applies whatever filters it defines. `GET /api/leads/`
(`backend/leads/views/lead_views.py:105-160`) is representative of how rich these can get:
| Query parameter | Matches |
| --- | --- |
| `name` | `first_name` or `last_name`, case-insensitive contains |
| `salutation` | case-insensitive contains |
| `source` | exact |
| `assigned_to` | repeatable; `assigned_to__id__in` |
| `status` | exact |
| `tags` | repeatable; `tags__id__in` |
| `city` | case-insensitive contains |
| `email` | case-insensitive contains |
| `rating` | exact |
| `search` | `first_name`, `last_name`, `company_name` or `email`, case-insensitive contains, OR'd together |
| `created_at__gte`, `created_at__lte` | date range |
| `close_date__gte`, `close_date__lte` | date range |
| `cf_<key>` | custom field `<key>` equals the given value (`custom_fields__contains`) |
For example, `GET /api/leads/?status=assigned&source=partner&search=acme` — `status` and `source`
are exact matches against `LEAD_STATUS` and `LEAD_SOURCE` (`backend/common/utils.py:50-67`:
statuses are `assigned`, `in process`, `converted`, `recycled`, `closed`; sources are `call`,
`email`, `existing customer`, `partner`, `public relations`, `compaign`, `other`), so a filter value
outside those exact strings matches nothing rather than erroring. Other list endpoints define
their own, generally smaller, filter sets in the same style — there is no single reference for
"every filter the API supports"; read the view (or watch for a dedicated section on that resource's
page, where one exists) rather than assuming this table applies elsewhere.
## Response shape
There is no single response envelope used across the whole API. Three shapes recur, and which one
a given endpoint uses is part of that endpoint's own contract:
**Mutations wrapped in an `error` flag.** The large majority of create/update/delete endpoints
return `{"error": bool, ...}`, checked explicitly rather than relying on the HTTP status code alone,
for example a successful lead create (`backend/leads/views/lead_views.py:402-404`):
```json
{"error": false, "message": "Lead Created Successfully"}
```
and a failed one (`backend/leads/views/lead_views.py:406-408`):
```json
{"error": true, "errors": {"email": ["Another lead in this organisation already uses that email address."]}}
```
The same `{"error": false, "tokens": [...]}` shape is used by the personal-access-token list in
[Tokens and API keys](tokens-and-api-keys.md#personal-access-tokens).
**Bare serializer data.** Some newer endpoints skip the `error` wrapper entirely and return
`serializer.data` (or the created/updated object's serialization) directly, relying on
`serializer.is_valid(raise_exception=True)` and the HTTP status code to signal failure — for
example `MacroListCreateView.post` returns `Response(MacroSerializer(macro).data, status=201)`
(`backend/macros/views.py:136`).
**Single-object detail responses nest the record, not return it bare.** `GET /api/leads/{id}/`
does not return a `Lead` object at the top level; it returns the record under `lead_obj` alongside
related data fetched in the same call:
```json
{
"lead_obj": {"id": "...", "first_name": "...", "...": "..."},
"attachments": [...],
"comments": [...],
"users_mention": [...],
"assigned_data": [...]
}
```
(`backend/leads/views/lead_views.py:412-517`.) Don't assume a detail endpoint returns its resource
at the top level of the response; check the endpoint.
## Content types
`REST_FRAMEWORK` sets no `DEFAULT_PARSER_CLASSES`, so DRF's own defaults apply: a request body may
be `application/json`, `application/x-www-form-urlencoded`, or `multipart/form-data`
(`JSONParser`, `FormParser`, `MultiPartParser`, in that order). `multipart/form-data` is what every
endpoint that accepts a file upload alongside other fields needs — for example `POST /api/leads/`
reads an optional `lead_attachment` file straight off `request.FILES` in the same call that creates
the lead (`backend/leads/views/lead_views.py:330`), so a lead with an initial attachment is created
with one multipart request rather than a create-then-upload round trip.
`REST_FRAMEWORK` sets no `DEFAULT_RENDERER_CLASSES` either, so both of DRF's defaults are live:
`JSONRenderer` for `application/json` (what every example on this page assumes, and what every
client integration should send `Accept: application/json` for) and `BrowsableAPIRenderer` for
`text/html` — request the same endpoint with a browser's default `Accept` header and DRF's
interactive HTML form is what comes back instead of JSON.

221
docs/api/endpoint-index.md Normal file
View file

@ -0,0 +1,221 @@
# Endpoint index
Every route the API exposes, generated from the OpenAPI schema. The curated
pages in this section explain the endpoints you are most likely to need; this
table exists so nothing is invisible.
Interactive versions of the same schema are served by a running backend at
`/swagger-ui/` and `/api/schema/redoc/`, with the raw document at `/schema/`.
!!! note
This page is generated by `backend/scripts/generate_endpoint_index.py`.
Edit that script, not this file.
| Path | Methods |
| --- | --- |
| `/api/accounts/` | GET, POST |
| `/api/accounts/attachment/{id}/` | DELETE |
| `/api/accounts/comment/{id}/` | DELETE, PATCH, PUT |
| `/api/accounts/{id}/` | DELETE, GET, PATCH, POST, PUT |
| `/api/accounts/{id}/create_mail/` | POST |
| `/api/activities/` | GET |
| `/api/api-settings/` | GET, POST |
| `/api/api-settings/{id}/` | DELETE, GET, PATCH, PUT |
| `/api/auth/google/` | POST |
| `/api/auth/google/callback/` | POST |
| `/api/auth/magic-link/request/` | POST |
| `/api/auth/magic-link/verify-code/` | POST |
| `/api/auth/magic-link/verify/` | POST |
| `/api/auth/me/` | GET |
| `/api/auth/profile/` | GET |
| `/api/auth/refresh-token/` | POST |
| `/api/auth/switch-org/` | POST |
| `/api/boards/` | GET, POST |
| `/api/boards/columns/{column_pk}/tasks/` | GET, POST |
| `/api/boards/tasks/{id}/` | DELETE, PUT |
| `/api/boards/{board_pk}/columns/` | GET, POST |
| `/api/boards/{id}/` | DELETE, GET, PATCH, PUT |
| `/api/business-hours/calendar/` | GET, PUT |
| `/api/business-hours/calendar/{id}/` | GET, PUT |
| `/api/business-hours/calendar/{id}/holidays/` | POST |
| `/api/business-hours/calendar/{id}/holidays/{hid}/` | DELETE |
| `/api/cases/` | GET, POST |
| `/api/cases/analytics/agents/` | GET |
| `/api/cases/analytics/backlog/` | GET |
| `/api/cases/analytics/drilldown/` | GET |
| `/api/cases/analytics/export/` | GET |
| `/api/cases/analytics/frt/` | GET |
| `/api/cases/analytics/mttr/` | GET |
| `/api/cases/analytics/service/` | GET |
| `/api/cases/analytics/sla/` | GET |
| `/api/cases/approval-rules/` | GET, POST |
| `/api/cases/approval-rules/{id}/` | DELETE, GET, PUT |
| `/api/cases/approvals/` | GET |
| `/api/cases/approvals/{id}/approve/` | POST |
| `/api/cases/approvals/{id}/cancel/` | POST |
| `/api/cases/approvals/{id}/reject/` | POST |
| `/api/cases/attachment/{id}/` | DELETE |
| `/api/cases/bulk/delete/` | POST |
| `/api/cases/bulk/update/` | POST |
| `/api/cases/comment/{id}/` | DELETE, PATCH, PUT |
| `/api/cases/csat/aggregate/` | GET |
| `/api/cases/escalation-policies/` | GET, POST |
| `/api/cases/escalation-policies/{id}/` | DELETE, GET, PUT |
| `/api/cases/import/commit/` | POST |
| `/api/cases/import/preview/` | POST |
| `/api/cases/inbound/{mailbox_id}/` | POST |
| `/api/cases/kanban/` | GET |
| `/api/cases/mailboxes/` | GET, POST |
| `/api/cases/mailboxes/{id}/` | DELETE, GET, PUT |
| `/api/cases/pipelines/` | GET, POST |
| `/api/cases/pipelines/{id}/` | DELETE, GET, PUT |
| `/api/cases/pipelines/{pipeline_pk}/stages/` | POST |
| `/api/cases/pipelines/{pipeline_pk}/stages/reorder/` | POST |
| `/api/cases/reopen-policy/` | GET, PUT |
| `/api/cases/routing-rules/` | GET, POST |
| `/api/cases/routing-rules/{id}/` | DELETE, GET, PUT |
| `/api/cases/routing-rules/{id}/test/` | POST |
| `/api/cases/solutions/` | GET, POST |
| `/api/cases/solutions/{id}/` | DELETE, GET, PATCH, PUT |
| `/api/cases/solutions/{id}/publish/` | POST |
| `/api/cases/solutions/{id}/unpublish/` | POST |
| `/api/cases/stages/{id}/` | DELETE, PUT |
| `/api/cases/watching/` | GET |
| `/api/cases/{id}/` | DELETE, GET, PATCH, POST, PUT |
| `/api/cases/{id}/activities/` | GET |
| `/api/cases/{id}/close-with-children/` | POST |
| `/api/cases/{id}/link/` | POST |
| `/api/cases/{id}/merge/{into_id}/` | POST |
| `/api/cases/{id}/move/` | PATCH |
| `/api/cases/{id}/request-approval/` | POST |
| `/api/cases/{id}/solution-suggestions/` | GET |
| `/api/cases/{id}/solutions/` | DELETE, POST |
| `/api/cases/{id}/solutions/{solution_pk}/` | DELETE, POST |
| `/api/cases/{id}/time-entries/` | GET, POST |
| `/api/cases/{id}/time-entries/start/` | POST |
| `/api/cases/{id}/time-summary/` | GET |
| `/api/cases/{id}/tree/` | GET |
| `/api/cases/{id}/unmerge/` | POST |
| `/api/cases/{id}/watch/` | DELETE, POST |
| `/api/cases/{id}/watchers/` | GET |
| `/api/contacts/` | GET, POST |
| `/api/contacts/attachment/{id}/` | DELETE |
| `/api/contacts/comment/{id}/` | DELETE, PATCH, PUT |
| `/api/contacts/import/commit/` | POST |
| `/api/contacts/import/preview/` | POST |
| `/api/contacts/{id}/` | DELETE, GET, PATCH, POST, PUT |
| `/api/custom-fields/` | GET, POST |
| `/api/custom-fields/{id}/` | DELETE, GET, PUT |
| `/api/dashboard/` | GET |
| `/api/dashboard/today/` | GET |
| `/api/documents/` | GET, POST |
| `/api/documents/{id}/` | DELETE, GET, PATCH, PUT |
| `/api/invoices/` | GET, POST |
| `/api/invoices/attachments/{id}/` | DELETE |
| `/api/invoices/comments/{id}/` | DELETE, PUT |
| `/api/invoices/estimates/` | GET, POST |
| `/api/invoices/estimates/{id}/` | DELETE, GET, PUT |
| `/api/invoices/estimates/{id}/convert/` | POST |
| `/api/invoices/estimates/{id}/pdf/` | GET |
| `/api/invoices/estimates/{id}/send/` | POST |
| `/api/invoices/from-opportunity/{opportunity_id}/` | POST |
| `/api/invoices/from-time-entries/` | POST |
| `/api/invoices/products/` | GET, POST |
| `/api/invoices/products/{id}/` | DELETE, GET, PUT |
| `/api/invoices/recurring/` | GET, POST |
| `/api/invoices/recurring/{id}/` | DELETE, GET, PUT |
| `/api/invoices/recurring/{id}/toggle/` | POST |
| `/api/invoices/reports/aging/` | GET |
| `/api/invoices/reports/dashboard/` | GET |
| `/api/invoices/reports/revenue/` | GET |
| `/api/invoices/templates/` | GET, POST |
| `/api/invoices/templates/{id}/` | DELETE, GET, PUT |
| `/api/invoices/{id}/` | GET, PUT |
| `/api/invoices/{id}/cancel/` | POST |
| `/api/invoices/{id}/duplicate/` | POST |
| `/api/invoices/{id}/mark-paid/` | POST |
| `/api/invoices/{id}/pdf/` | GET |
| `/api/invoices/{id}/send/` | POST |
| `/api/invoices/{invoice_id}/attachments/` | POST |
| `/api/invoices/{invoice_id}/comments/` | POST |
| `/api/invoices/{invoice_id}/line-items/` | GET, POST |
| `/api/invoices/{invoice_id}/line-items/{id}/` | DELETE, PUT |
| `/api/invoices/{invoice_id}/payments/` | GET, POST |
| `/api/invoices/{invoice_id}/payments/{id}/` | DELETE |
| `/api/leads/` | GET, POST |
| `/api/leads/attachment/{id}/` | DELETE |
| `/api/leads/comment/{id}/` | DELETE, PATCH, PUT |
| `/api/leads/create-from-site/` | POST |
| `/api/leads/kanban/` | GET |
| `/api/leads/pipelines/` | GET, POST |
| `/api/leads/pipelines/{id}/` | DELETE, GET, PUT |
| `/api/leads/pipelines/{pipeline_pk}/stages/` | POST |
| `/api/leads/pipelines/{pipeline_pk}/stages/reorder/` | POST |
| `/api/leads/stages/{id}/` | DELETE, PUT |
| `/api/leads/upload/` | POST |
| `/api/leads/{id}/` | DELETE, GET, PATCH, POST, PUT |
| `/api/leads/{id}/move/` | PATCH |
| `/api/macros/` | GET, POST |
| `/api/macros/{id}/` | DELETE, GET, PATCH, PUT |
| `/api/macros/{id}/render/` | POST |
| `/api/notifications/` | GET |
| `/api/notifications/read-all/` | POST |
| `/api/notifications/stream/` | GET |
| `/api/notifications/{id}/` | DELETE |
| `/api/notifications/{id}/read/` | POST |
| `/api/opportunities/` | GET, POST |
| `/api/opportunities/aging-config/` | GET, PUT |
| `/api/opportunities/attachment/{id}/` | DELETE |
| `/api/opportunities/comment/{id}/` | DELETE, PATCH, PUT |
| `/api/opportunities/goals/` | GET, POST |
| `/api/opportunities/goals/leaderboard/` | GET |
| `/api/opportunities/goals/{id}/` | DELETE, GET, PUT |
| `/api/opportunities/kanban/` | GET |
| `/api/opportunities/{id}/` | DELETE, GET, PATCH, POST, PUT |
| `/api/opportunities/{id}/move/` | PATCH |
| `/api/opportunities/{opportunity_id}/line-items/` | GET, POST |
| `/api/opportunities/{opportunity_id}/line-items/{line_item_id}/` | DELETE, GET, PUT |
| `/api/org/` | GET, POST |
| `/api/org/api-key/` | GET, POST |
| `/api/org/settings/` | GET, PATCH |
| `/api/org/tokens/` | GET |
| `/api/org/tokens/{id}/` | DELETE |
| `/api/org/{id}/` | GET, PATCH, PUT |
| `/api/packs/` | GET |
| `/api/packs/sample-data/` | DELETE |
| `/api/packs/{pack_id}/apply/` | POST |
| `/api/profile/` | GET, PATCH |
| `/api/profile/tokens/` | GET, POST |
| `/api/profile/tokens/{id}/` | DELETE |
| `/api/public/csat/{token}/` | GET, POST |
| `/api/public/estimate/{token}/` | GET |
| `/api/public/estimate/{token}/accept/` | POST |
| `/api/public/estimate/{token}/decline/` | POST |
| `/api/public/estimate/{token}/pdf/` | GET |
| `/api/public/invoice/{token}/` | GET |
| `/api/public/invoice/{token}/pdf/` | GET |
| `/api/search/` | GET |
| `/api/tags/` | GET, POST |
| `/api/tags/{id}/` | DELETE, GET, PUT |
| `/api/tags/{id}/restore/` | POST |
| `/api/tasks/` | GET, POST |
| `/api/tasks/attachment/{id}/` | DELETE |
| `/api/tasks/comment/{id}/` | DELETE, PATCH, PUT |
| `/api/tasks/kanban/` | GET |
| `/api/tasks/pipelines/` | GET, POST |
| `/api/tasks/pipelines/{id}/` | DELETE, GET, PUT |
| `/api/tasks/pipelines/{pipeline_pk}/stages/` | POST |
| `/api/tasks/pipelines/{pipeline_pk}/stages/reorder/` | POST |
| `/api/tasks/stages/{id}/` | DELETE, PUT |
| `/api/tasks/{id}/` | DELETE, GET, PATCH, POST, PUT |
| `/api/tasks/{id}/move/` | PATCH |
| `/api/teams/` | GET, POST |
| `/api/teams/{id}/` | DELETE, GET, PATCH, PUT |
| `/api/time-entries/timesheet/` | GET |
| `/api/time-entries/unbilled/` | GET |
| `/api/time-entries/{id}/` | DELETE, PUT |
| `/api/time-entries/{id}/stop/` | POST |
| `/api/user/{id}/` | DELETE, GET, PATCH, PUT |
| `/api/user/{id}/status/` | POST |
| `/api/users/` | GET, POST |
| `/api/users/get-teams-and-users/` | GET |

143
docs/api/errors.md Normal file
View file

@ -0,0 +1,143 @@
# Errors
How to tell a failed request from a successful one, and how to read the body of a failed one.
`REST_FRAMEWORK["EXCEPTION_HANDLER"]` in `backend/crm/settings.py` is set to DRF's own
`rest_framework.views.exception_handler` — unmodified, no custom subclass — so an exception DRF
itself raises (a bad permission check, a missing object, an unhandled `ValidationError`) comes back
in DRF's own default shape. A view that catches its own errors and builds a `Response` by hand,
which most views in this codebase do, can and often does return something else. Both are documented
below with real examples, because a client has to handle both.
## Status codes
`200` for a successful read or a successful write — many mutation endpoints return `200` rather
than `201` even on creation, wrapping the result in the `{"error": false, ...}` shape described in
[Conventions](conventions.md#response-shape) rather than relying on the status code alone (see
`POST /api/leads/`, `backend/leads/views/lead_views.py:402-404`). Some do use `201`: personal access
token creation, for instance, returns `201` (`backend/common/views/pat_views.py:69`). Neither is
"more correct" than the other here — check the endpoint.
Beyond that: `400` for a request the server understood but rejected (validation, or a
request-shaped error like a missing required field); `401` for a request with no valid credentials
at all; `403` for a request that authenticated fine but isn't allowed to do this; `404` for a
resource that doesn't exist, or that exists but the caller must not be able to distinguish from not
existing — see [Not found versus forbidden](#not-found-versus-forbidden); `502` where a view depends
on an external service and that service is unreachable, for example the Google token exchange in
[Google OAuth](authentication.md#google-oauth-web) (`backend/common/views/auth_views.py:110-114`).
There is no application-level `429` — `REST_FRAMEWORK` sets no `DEFAULT_THROTTLE_CLASSES`, so DRF's
built-in throttling is not in play anywhere. The one place this codebase does rate-limit
(`POST /api/auth/magic-link/request/`, five tokens per address per hour) enforces it by silently
returning its normal `200` rather than a `429`, specifically so the response can't be used to
distinguish "rate limited" from "sent" from "invalid address" — see
[Magic links](authentication.md#magic-links).
## Validation errors
The dominant pattern in this codebase is not DRF's automatic one. Most views call
`serializer.is_valid()` without `raise_exception=True`, check the boolean themselves, and build the
error response by hand as `{"error": true, "errors": serializer.errors}` — this exact shape appears
well over 200 times across the backend. A `validate_<field>` failure lands under that field's key,
DRF's normal behavior:
```python
def validate_email(self, value):
...
if duplicates.exists():
raise serializers.ValidationError(
"Another lead in this organisation already uses that email address."
)
return value
```
(`backend/leads/serializer.py:107-133`, `LeadCreateSerializer.validate_email` — the comment above it
explains this exists because a duplicate email used to reach the database as an unhandled
`IntegrityError` and come back as an opaque `500`.) A `POST /api/leads/` with a duplicate email
returns:
```json
{"error": true, "errors": {"email": ["Another lead in this organisation already uses that email address."]}}
```
A `validate()` failure that isn't tied to one field lands under DRF's default
`non_field_errors` key (`NON_FIELD_ERRORS_KEY` is not overridden anywhere in this codebase), still
nested the same way: `{"error": true, "errors": {"non_field_errors": ["..."]}}`.
A smaller, newer set of views instead call `serializer.is_valid(raise_exception=True)` — for
example `MacroListCreateView.post` (`backend/macros/views.py:123`) — and let DRF's exception handler
build the response itself, with no `error`/`errors` wrapper at all:
```json
{"email": ["Another lead in this organisation already uses that email address."]}
```
Both shapes exist in the API today. Check the specific endpoint's page (or the view, if it isn't
documented yet) rather than assuming one; do not assume the presence or absence of an `error` key
tells you whether the request succeeded — the status code is what does that.
## Authorization errors
Authentication and authorization failures go through DRF's permission machinery unmodified, so they
always come back as `{"detail": "<message>"}`, whether the message is DRF's own default or one this
codebase set:
```json
{"detail": "Authentication credentials were not provided."}
```
```json
{"detail": "Organization context is required. Please login again."}
```
The second is `HasOrgContext.message` (`backend/common/permissions.py:11-41`), one of the two
permission classes almost every org-scoped view lists — `permission_classes = (IsAuthenticated,
HasOrgContext)`, and often a role check on top such as `IsOrgAdmin`
(`backend/common/permissions.py:44-59`, `"You must be an organization administrator to perform this
action."`). Whether a failure comes back as `401` or `403` is not decided by which permission class
failed — DRF's `APIView.permission_denied()` raises `NotAuthenticated` (`401`) whenever no
authenticator on the request actually succeeded, and `PermissionDenied` (`403`, carrying the failing
permission class's `message`) whenever one did but the permission check itself failed. In practice:
no credentials, or invalid ones → `401`; a valid, authenticated session that isn't allowed to do
this → `403`.
## Not found versus forbidden
The rule that holds everywhere an object is looked up by id: a record outside the caller's org
returns `404`, the same as a record that never existed at all, never `403`. Every such lookup
filters on `org=request.profile.org` (or the RLS-protected queryset equivalent) before matching the
id, so a request for another org's record simply finds nothing —
`get_object_or_404(Macro, pk=pk, org=request.profile.org)` (`backend/macros/views.py:148`, repeated
at `:168` for the plain `GET`) is one of dozens of examples of the same shape.
`invoices/permissions.py` states the reasoning explicitly in its own module docstring:
```
1. Does this record exist *in the caller's org*? -> 404 if not, so that a
caller in another org cannot probe for record IDs.
2. May this caller act on it? -> 403 if not.
```
(`backend/invoices/permissions.py:5-9`.) A `403` response inherently confirms the record exists —
it's not possible to be forbidden from acting on nothing — so reserving `403` for "exists, but you
may not do this" and `404` for everything else is what stops an id becoming an oracle a caller in
another org can probe with.
Within a single org, whether a permission failure still returns `403` or drops all the way to `404`
depends on what the resource is. Invoices and estimates keep step 2 above as a real `403`: an org
member who is neither the record's creator, an assignee, nor an admin is refused with
`{"error": true, "message": "Permission denied"}` (`backend/invoices/permissions.py:80-84`) — a
colleague can tell the invoice exists, just not read or change it. A personal (as opposed to
org-scope) macro goes further and hides itself completely from everyone except its owner and org
admins, `404` even for a same-org caller, precisely to avoid the invoice case's disclosure:
```python
if macro.scope == Macro.SCOPE_PERSONAL and macro.owner_id != request.profile.id:
# A personal macro you don't own is invisible to you — the list and
# GET hide it (404). The write verbs must not confirm it exists via
# a 403 either, or the id space leaks which rows are somebody else's
# personal macros. Mirror the GET: 404.
return Response({"detail": "Not found."}, status=status.HTTP_404_NOT_FOUND)
```
(`backend/macros/views.py:156-164`.) Both are deliberate, documented choices for the resource in
question, not an inconsistency to paper over — when you add an authorization check to a new
endpoint, decide which of the two your resource needs (does an org-mate being aware this record
exists leak anything?) rather than defaulting to whichever is easier to write.

315
docs/api/invoices.md Normal file
View file

@ -0,0 +1,315 @@
# Invoices and estimates
Authenticated routes in `backend/invoices/api_urls.py`, all mounted under `/api/invoices/...`. **The
DRF views live in `backend/invoices/api_views.py` — not `views.py`**, which is a legacy,
non-API module in this app. Anonymous client-portal routes are in
`backend/invoices/public_urls.py`, served by `backend/invoices/public_views.py`, mounted at
`/api/public/...`. Authorization for every authenticated invoice/estimate/recurring-invoice endpoint
is centralized in `backend/invoices/permissions.py`. See [Conventions](conventions.md) for pagination,
filtering and response-shape rules assumed rather than repeated here, and [Errors](errors.md) for the
404-vs-403 principle this page's own module docstring states almost verbatim.
This page covers invoices, estimates, line items and the public portal — the endpoints this
documentation task scopes. Payments, products, templates, recurring invoices, dashboards/reports, and
invoice-from-opportunity/from-time-entries are also part of this module (`api_urls.py`) but are out of
scope for this page.
## One access rule for invoices, estimates and recurring invoices
`invoices/permissions.py` states a single rule, reused by all three record types via
`get_invoice_or_error`/`get_estimate_or_error`/`get_recurring_or_error`
(`invoices/permissions.py:101-122`):
```
1. Does this record exist in the caller's org? -> 404 if not
2. May this caller act on it? -> 403 if not
```
— the exact two-step [Errors: Not found versus forbidden](errors.md#not-found-versus-forbidden) quotes
as this module's own docstring. Step 2, `has_object_access` (`:32-58`), grants access to an org admin,
a Django superuser, the record's creator, or any profile in its `assigned_to` set — there is no
separate, narrower rule for write vs. delete the way [Cases](cases.md#a-cases-read-write-and-delete-rules-differ-on-purpose)
has; one rule covers read, write and delete alike for invoices and estimates.
**The two comparisons inside that rule are against deliberately different types.** `created_by` is a
`User` FK (from `AuditModel`), so it is compared against `request.user`; `assigned_to` holds `Profile`
rows, so it is compared against `request.profile` (`:42-58`). The module docstring calls out that
getting this backwards — comparing `request.profile` to `created_by`, as the estimate views used to —
is not merely always-`False` the way the same mistake is elsewhere in this codebase: because it reaches
the ORM as `Q(created_by=<Profile instance>)`, Django raises `ValueError` on the type mismatch, and the
non-admin estimate list answered `500` rather than an empty or wrong list. That specific defect is
fixed in the current source (`_get_or_error`/`has_object_access` as shown), but it is a concrete example
of why this codebase keeps insisting a permission check be provably tested in both directions.
## Invoices
`GET /api/invoices/` (`InvoiceListView.get`, `invoices/api_views.py:176-194`) hand-builds a response
with the same `count`/`next`/`previous`/`results` key names DRF's `get_paginated_response()` would
produce, plus a `totals` block computed over every invoice the caller can see (not just the page or the
current filter). This is the hand-rolled shape, not the DRF-generated one —
[Conventions](conventions.md#pagination) and [Cases](cases.md#solutions) both name
`GET /api/cases/solutions/` as the only endpoint in this API that actually calls
`get_paginated_response()`; this view reaches the same four key names by building the dict itself:
```json
{
"count": 41,
"next": null,
"previous": null,
"results": [ { "...": "InvoiceListSerializer" } ],
"totals": {
"count": 41, "outstanding": "12500.00", "overdue": "3200.00",
"due_this_month": "4100.00", "paid_this_quarter": "9800.00",
"draft": "600.00", "action_needed": 5
}
}
```
`totals` is computed over `Invoice.objects.filter(pk__in=queryset.values("pk"))` rather than the
visibility-scoped queryset directly (`_totals`, `:196-235`) — the comment explains why: a non-admin's
queryset joins `assigned_to` (a many-to-many), so summing across that join would multiply an invoice's
amount by its number of assignees; re-querying by `pk__in` drops the join before the `Sum`.
`action_needed` is a **count**, not an amount — invoices still `Draft`, plus anything unpaid and past
`due_date` (`:209`). A non-admin caller sees only invoices they created or are assigned to (`:96-103`,
the same rule [above](#one-access-rule-for-invoices-estimates-and-recurring-invoices) applies at list
scope).
Filters (`filter_queryset`, `:105-174`): `search` (matches `invoice_title`, `invoice_number`,
`client_name` or `client_email`, contains, capped to 100 characters), `status` (exact),
`account`/`contact`/`opportunity` (exact id), `assigned_to` (exact id), `created_by` (exact id),
`issue_date_gte`/`issue_date_lte`, `due_date_gte`/`due_date_lte`, `cf_<key>`, and `sort` — whitelisted
to `created_at`/`due_date`/`issue_date`/`total_amount`/`status`, each optionally `-`-prefixed
(`:163-172`); an unrecognized `sort` value is silently ignored, leaving the default `-created_at`.
`POST /api/invoices/` (`:237-292`) is validated by `InvoiceCreateSerializer`
(`invoices/serializer.py:542-618`; see [Fields](#fields)). On success (`201`):
```json
{"error": false, "message": "Invoice created successfully", "invoice": { "...": "InvoiceSerializer" }}
```
Creating an invoice queues `create_invoice_history` (an audit-trail Celery task) and, if the invoice
already has assignees, `send_email` to notify them (`:261-278`).
`GET /api/invoices/{id}/` (`InvoiceDetailView.get`, `:295-341`) returns
`{"invoice": ..., "attachments": [...], "comments": [...], "history": [...], "custom_field_definitions": [...]}`.
`PUT /api/invoices/{id}/` (`:343-400`) accepts a **partial** body (`partial=True` is passed even though
this is `PUT`, `:376`) through the same `InvoiceCreateSerializer`. **`InvoiceDetailView` defines only
`get` and `put` — no delete method of any kind exists for an invoice**; an invoice is retired via
`POST /api/invoices/{id}/cancel/`, never removed. (Contrast [Estimates](#estimates), which does support
a hard delete.)
Three lifecycle actions, each `get_invoice_or_error`-gated and each a `POST` with no request body:
- `POST /api/invoices/{id}/send/` (`InvoiceSendView.post`, `:403-440`) refuses to re-send a `Paid` or
`Cancelled` invoice (`400`); moves `Draft` → `Sent`, stamps `sent_at`/`is_email_sent`, and queues
`send_invoice_to_client`.
- `POST /api/invoices/{id}/mark-paid/` (`InvoiceMarkPaidView.post`, `:443-481`) records a `Payment` —
it defaults `amount` to the current `amount_due`, `payment_method` to `OTHER`, and `payment_date` to
today when the caller omits them, but routes the whole thing through
`PaymentCreateSerializer` (`invoices/serializer.py:365-396`) either way, so a payment amount of `0`
or one exceeding the outstanding balance is still rejected (`400`).
- `POST /api/invoices/{id}/cancel/` (`InvoiceCancelView.post`, `:563-599`) refuses an already-`Cancelled`
invoice and refuses a `Paid` one (`400` either way) — cancelling is for money that will never be
collected, not for reversing a completed payment.
`POST /api/invoices/{id}/duplicate/` (`InvoiceDuplicateView.post`, `:484-560`) copies the invoice's
client/billing/discount/tax/template fields and every line item into a new `Draft` invoice with a fresh
`invoice_number`, `issue_date` reset to today, and no payments — `201` with the new
`InvoiceSerializer` payload.
`GET /api/invoices/{id}/pdf/` (`InvoicePDFView.get`, `:602-631`) streams
`application/pdf`; a missing PDF backend answers `503`, any other generation failure `500`.
## Estimates
The same list/detail/lifecycle shape as invoices, with one structural difference: **an estimate can be
hard-deleted; an invoice cannot** (see [above](#invoices)).
`GET /api/invoices/estimates/` (`EstimateListView.get`, `:1027-1064`) — same hand-built
`count`/`next`/`previous`/`results` shape as the invoice list above, without a `totals` block. Filters:
`status`, `account`, `search` (matches
`estimate_number`, `title` or `client_name`), `cf_<key>`. Visibility follows the same rule as invoices
— non-admin sees only estimates they created or are assigned to.
`POST /api/invoices/estimates/` (`:1066-1100`) is validated by `EstimateCreateSerializer`
(`invoices/serializer.py:892-1018`; see [Fields](#fields)). **Unlike `InvoiceCreateSerializer`,
`EstimateCreateSerializer` does not mark `status` read-only** — nothing stops an authenticated,
same-org caller with write access from `POST`ing or `PUT`ing an estimate directly into
`status: "Accepted"`. Doing so does **not** run any of the checks or side effects the public accept
flow enforces (see [Public portal endpoints](#public-portal-endpoints)): no expiry check, and none of
`accepted_by_name`/`accepted_by_email`/`accepted_ip`/`accepted_user_agent` get populated, even though
the model comment above those fields calls acceptance "an authorisation" that the app records who gave.
This is reported here, not fixed, per this documentation task's scope.
`GET/PUT/DELETE /api/invoices/estimates/{id}/` (`EstimateDetailView`, `:1103-1187`) — `DELETE` is a
real hard delete (`estimate.delete()`, `:1182`), gated by the same access rule as read/write.
`POST /api/invoices/estimates/{id}/convert/` (`EstimateConvertView.post`, `:1189-1266`) creates a new
`Draft` invoice from the estimate's client/billing/discount/tax fields and line items, then sets
`estimate.status = "Accepted"`, `accepted_at = now()`, and `estimate.converted_to_invoice = invoice`
(`:1253-1257`). **This is the field that stops a second conversion from duplicating the invoice**: the
view's very first check is `if estimate.converted_to_invoice: return 400` (`:1200-1204`), before
anything else runs. Any client rendering an estimate list needs to read this field, not just `status`,
to decide whether to offer "raise an invoice" —
`EstimateListSerializer.converted_to_invoice` (`invoices/serializer.py:832,846,866-870`) is on the list
row specifically so the worklist can tell "accepted, not yet billed" apart from "already billed" without
a second request per row; the class docstring is explicit that omitting it "would offer to raise a
second invoice for an estimate that already has one."
`POST /api/invoices/estimates/{id}/send/` (`EstimateSendView.post`, `:1269-1291`) moves `Draft` → `Sent`
and queues `send_estimate_to_client`. `GET /api/invoices/estimates/{id}/pdf/`
(`EstimatePDFView.get`, `:1294-1334`) streams the PDF the same way the invoice one does.
## Line items
Two ways line items reach the database, with different validation coverage.
**Nested in an invoice create/update.** `InvoiceCreateSerializer`'s `line_items`
(`InvoiceLineItemCreateSerializer`, many, `invoices/serializer.py:288-304`) is written inside the same
request as the invoice itself; `validate()` additionally rejects any line item whose `product` belongs
to a different org (`:689-704` — the comment notes the nested serializer's own `product` field is an
unscoped `PrimaryKeyRelatedField` over every org's products, so this cross-org check has to happen at
the parent serializer, where org context exists). On `PUT`, sending `line_items` **replaces** the set —
existing rows are deleted and recreated (`InvoiceCreateSerializer.update`, `:731-753`); omitting the key
entirely leaves existing line items untouched.
**Standalone, via `POST /api/invoices/{invoice_id}/line-items/`**
(`InvoiceLineItemListView.post`, `invoices/api_views.py:653-679`) and
**`PUT`/`DELETE /api/invoices/{invoice_id}/line-items/{id}/`**
(`InvoiceLineItemDetailView`, `:682-751`). Both route through `InvoiceLineItemCreateSerializer`
directly, with no wrapping parent-serializer `validate()` — and **neither has an application-code check
that `product` belongs to the caller's org**: `InvoiceLineItemCreateSerializer` declares no `product`
field of its own (`invoices/serializer.py:288-304`), so DRF derives a plain
`PrimaryKeyRelatedField(queryset=Product.objects.all())` — every org's products — and both views use it
raw, unlike the *nested* path, which does check (`InvoiceCreateSerializer.validate()`, `:689-704`; grep
confirms this is the only `product`-org check anywhere in `invoices/serializer.py`). This is a real gap
against [the ORM-filter contract](../architecture/multi-tenancy-and-rls.md#the-two-layer-contract) —
the explicit application-level filter that page calls non-optional is missing here — but it is not an
unconditional cross-tenant read. `product` (the table, along with
`invoice_line_item`) is itself RLS-protected: both are in `ORG_SCOPED_TABLES`
(`common/rls/__init__.py:97`), with policy enabled by
`common/migrations/0008_enable_rls_product_invoice_line_item.py`. Under the non-superuser database role
[PostgreSQL and RLS](../self-hosting/postgresql-and-rls.md) mandates, the unscoped
`Product.objects.all()` queryset sees zero rows for a foreign
org's id under RLS's fail-safe default, and the request answers `400` ("invalid pk") rather than
attaching the row — the missing ORM filter is caught by the safety net. Under the *default*
configuration — `DBUSER` unset, which falls back to the Postgres superuser, and therefore bypasses RLS
entirely, per [Multi-tenancy and RLS](../architecture/multi-tenancy-and-rls.md#the-two-layer-contract)
— nothing stops it: the row attaches, and `InvoiceLineItemSerializer.product_name`
(`source="product.name"`, `:254,263`) discloses that other org's product name on every subsequent read.
Whether this is exploitable in a given deployment is exactly the question "is `DBUSER` a superuser?"
answers — which is also why it's worth fixing regardless of how any one deployment is configured: the
ORM filter is the contract, RLS is the net, and this endpoint is missing the contract. Reported here,
not fixed, per this documentation task's scope.
Every write to a line item (either path) recalculates and saves the parent invoice's totals
immediately afterward (`invoice.recalculate_totals(); invoice.save()`) — a client does not need a
separate call to refresh `total_amount`/`amount_due` after adding, editing or removing a line item.
`InvoiceLineItemCreateSerializer.Meta.fields` (`invoices/serializer.py:291-303`):
`product`, `name`, `description`, `quantity`, `unit_price`, `discount_type`, `discount_value`,
`tax_rate`, `order`. Every one of these is optional — `product` is nullable, `name` is `blank=True,
null=True`, `description` is `blank=True`, `quantity` defaults `1`, `unit_price`/`discount_value`/
`tax_rate`/`order` all default `0`, and `discount_type` is `blank=True` over `DISCOUNT_TYPES`
(`PERCENTAGE`, `FIXED`) (`invoices/models.py:512-565,71-74`). Posting `{}` is valid and creates a
line item with `quantity=1`, `unit_price=0`. **Estimate line items work identically** —
`EstimateLineItemCreateSerializer` has the same field list (`invoices/serializer.py:306-321`) — except
there is no standalone endpoint for them at all: estimate line items are only ever written nested
inside `POST`/`PUT /api/invoices/estimates/{id}/`, and `EstimateCreateSerializer.validate()`
(`:969-980`) has **no application-level product-org check whatsoever** — not even the
parent-serializer-level check `InvoiceCreateSerializer` has. The same RLS caveat applies here as above:
`product` is RLS-protected, so this is a missing-ORM-filter gap caught by the safety net under a
correctly-configured (non-superuser) database role, and a real cross-tenant read under the default,
superuser one.
## Public portal endpoints
**Mounted at `/api/public/...`, not under `/api/invoices/...`** — `backend/crm/urls.py:23`:
`path("api/public/", include("invoices.public_urls", ...))`. Every view here sets
`authentication_classes = []` and `permission_classes = []` (`invoices/public_views.py`) — these
endpoints are genuinely anonymous, reachable by anyone holding the URL, by design (an emailed invoice
or estimate link).
**Resolving the org happens before the RLS context is set, from an unscoped token lookup — this is the
whole reason the endpoint works at all under RLS.** `_resolve_org_context(token, resource_type)`
(`invoices/public_views.py:30-46`) calls `common.portal_tokens.resolve_portal_org`, which hashes the
raw URL token (`sha256`) and looks it up in `common.models.PortalAccessToken` — a small table carrying
no RLS policy — to get an `org_id`, then calls `set_rls_context(org_id)`. Only *after* that does the
view run its own `Invoice`/`Estimate` query. An unknown or malformed token leaves the RLS context
empty; the subsequent scoped query then returns nothing under RLS's fail-safe default, and the caller
gets the same `404` a disabled link would give — a stranger probing token values learns nothing about
whether one is real. [Architecture: Multi-tenancy and RLS](../architecture/multi-tenancy-and-rls.md#portal-tokens)
documents this mechanism in full, including why the lookup table has to be unscoped in the first place
(you cannot set `app.current_org` until you know the org, and you cannot look up the row that would
tell you the org until the context is set) — read that page for the general pattern; this page is the
concrete instance of it for invoices and estimates. Every endpoint below additionally requires
`public_link_enabled=True` on the record, independent of the token resolving at all — an admin can
revoke a previously-shared link without rotating the token.
- `GET /api/public/invoice/{token}/` (`PublicInvoiceView.get`, `:61-163`) — returns the invoice's
client-facing fields, billing address, line items and payment history. First view stamps `viewed_at`
and flips `Sent` → `Viewed`.
- `GET /api/public/invoice/{token}/pdf/` (`PublicInvoicePDFView.get`, `:166-206`) — same PDF the
authenticated endpoint generates, `include_payments=True`.
- `GET /api/public/estimate/{token}/` (`PublicEstimateView.get`, `:209-301`) — same shape, for
estimates; also stamps `viewed_at` on first view.
- `GET /api/public/estimate/{token}/pdf/` (`PublicEstimatePDFView.get`, `:304-344`).
- `POST /api/public/estimate/{token}/accept/` (`PublicEstimateAcceptView.post`, `:347-430`) — the
authenticated `status` write does not exist for the anonymous caller, so acceptance is its own
endpoint with its own rules, all enforced server-side because "the frontend is not a trust boundary"
(comment at `:359-363`): the estimate must currently be `Sent` or `Viewed` (`400` otherwise); it must
not be expired (`estimate.is_expired`, checked against `expiry_date`, `400` with the expiry date in
the message if it is — enforced here for the first time; the model comment notes `expiry_date` was
previously "stored and printed on the estimate but enforced nowhere"); and the request body must
supply `name` and a syntactically valid `email` (`400` otherwise). On success, the view records who
accepted: `accepted_by_name`, `accepted_by_email` (truncated to the column's max length), `accepted_ip`
(best-effort, from `X-Forwarded-For` or `REMOTE_ADDR` — spoofable, treated as evidence, not proof),
and `accepted_user_agent`, alongside `status = "Accepted"` and `accepted_at`.
- `POST /api/public/estimate/{token}/decline/` (`PublicEstimateDeclineView.post`, `:433-468`) — same
`Sent`/`Viewed` precondition, no identity capture, sets `status = "Declined"` and `declined_at`.
## Fields
`InvoiceCreateSerializer.Meta.fields` (`invoices/serializer.py:558-618`) is what `POST /api/invoices/`
and `PUT /api/invoices/{id}/` accept.
| Field | Type | Required | Notes |
| --- | --- | --- | --- |
| `invoice_title` | string, max 100 | **required** | |
| `account_id` | uuid | **required** | Must belong to the caller's org; the model itself allows null, but the serializer makes it mandatory |
| `contact_id` | uuid | **required** | Must belong to the caller's org; must belong to `account_id` if both given |
| `opportunity_id` | uuid | optional | Must belong to the caller's org |
| `template_id` | uuid | optional | Must belong to the caller's org — without this check a cross-org template's branding/markup would render on this invoice's PDF |
| `client_name`, `client_email`, `client_phone` | string | optional | Denormalized for the PDF and portal; both `client_name`/`client_email` default `""` |
| `billing_*` / `client_*` address fields | string | optional | `billing_country`/`client_country` are one of `COUNTRIES` |
| `discount_type` | one of `PERCENTAGE`, `FIXED` | optional | |
| `discount_value`, `tax_rate`, `shipping_amount` | decimal | optional | Default `0` |
| `currency` | one of `CURRENCY_CODES` | optional | Defaults `USD` |
| `issue_date` | date | optional | Defaults today |
| `due_date` | date | optional | |
| `payment_terms` | one of `PAYMENT_TERMS` | optional | Defaults `NET_30` |
| `reminder_enabled`, `reminder_days_before`, `reminder_days_after`, `reminder_frequency` | — | optional | |
| `public_link_enabled` | boolean | optional | Gates the portal endpoints independently of the token; defaults `true` |
| `notes`, `terms` | text | optional | |
| `billing_period`, `po_number` | string | optional | |
| `line_items` | list | optional | See [Line items](#line-items) |
| `status` | one of `INVOICE_STATUS` | **read-only** | Always created `Draft`; changed only via `send`/`mark-paid`/`cancel` |
`invoice_number` is not part of the serializer at all — `Invoice.save()` generates and assigns it
server-side the first time the row is saved (`invoices/models.py:384-411`).
`EstimateCreateSerializer.Meta.fields` (`invoices/serializer.py:904-928`) mirrors the invoice shape
closely: `title` (**required**), `account_id`/`contact_id` (**required**, org-validated),
`opportunity_id` (optional, org-validated), the same client/address/discount/tax/currency/notes/terms
fields, `expiry_date` (optional — this is what the public accept endpoint checks), `public_link_enabled`,
and `line_items`. **`status` is writable here** — see the caveat in [Estimates](#estimates). Like
invoices, `estimate_number` is server-generated on first save, not part of the serializer.
`GET` responses on both (`InvoiceSerializer`/`EstimateSerializer`, `invoices/serializer.py:444-539,
873-889`) additionally return, but never accept as input: `id`, `subtotal`, `discount_amount`,
`tax_amount`, `total_amount`, `amount_paid`, `amount_due` (all computed by
`recalculate_totals()`), `sent_at`/`viewed_at`/`paid_at`/`cancelled_at` (invoice) or
`sent_at`/`viewed_at`/`accepted_at`/`declined_at`/`accepted_by_*` (estimate), `public_token`,
`public_url`, `created_by`, `created_at`, `updated_at`, and — on the estimate only —
`converted_to_invoice` (see [Estimates](#estimates)).

246
docs/api/leads.md Normal file
View file

@ -0,0 +1,246 @@
# Leads
Routes in `backend/leads/urls.py`, views split across `leads/views/lead_views.py` (list and
detail) and `leads/views/lead_interactions.py` (comment and attachment editing). See
[Conventions](conventions.md) for pagination, filtering and response-shape rules this page assumes
rather than repeats, and [Errors](errors.md) for the 404-vs-403 principle referenced below.
## List leads
`GET /api/leads/` (`LeadListView.get`, `backend/leads/views/lead_views.py:246-248`) returns two
separately paginated sections rather than one list — open leads and closed leads — because the view
excludes converted leads from the queryset entirely and then splits what's left on `status`:
```python
queryset = (
self.model.objects.filter(org=self.request.profile.org)
.exclude(status="converted")
...
)
...
queryset_open = queryset.exclude(status="closed")
...
queryset_close = queryset.filter(status="closed")
```
(`lead_views.py:90-98,166,187`.) A non-admin caller only sees leads assigned to them or created by
them (`lead_views.py:99-103`); an admin or superuser sees every lead in the org. The response follows
the hand-rolled envelope [Conventions](conventions.md#pagination) describes for this exact endpoint:
```json
{
"per_page": 10,
"page_number": [1],
"open_leads": {"leads_count": 42, "open_leads": [ { "...": "LeadSerializer" } ], "offset": 10},
"totals": {"count": 42, "unworked_over_a_week": 5},
"close_leads": {"leads_count": 8, "close_leads": [ { "...": "..." } ], "offset": null},
"contacts": [{"id": "...", "first_name": "..."}],
"status": [["assigned", "Assigned"], ["in process", "In Process"], ["converted", "Converted"], ["recycled", "Recycled"], ["closed", "Closed"]],
"source": [["call", "Call"], ["email", "Email"], ["existing customer", "Existing Customer"], ["partner", "Partner"], ["public relations", "Public Relations"], ["compaign", "Campaign"], ["other", "Other"]],
"tags": [{"...": "..."}],
"users": [{"id": "...", "user__email": "..."}],
"countries": [["GB", "United Kingdom"], ["AF", "Afghanistan"]],
"industries": [["ADVERTISING", "ADVERTISING"]]
}
```
`page_number` is worth calling out: the view builds it as `(int(self.offset / 10) + 1,)`
(`lead_views.py:178`) — a one-element Python tuple, trailing comma and all — which the JSON renderer
turns into a one-element array like `[1]`, not a bare integer. `totals.unworked_over_a_week` counts
leads in the filtered, org-scoped `open` queryset whose `last_contacted` (or, absent that, creation
date) is more than 7 days old (`UNWORKED_AFTER_DAYS`, `lead_views.py:58,60-86`) — not a stored status,
just a computed staleness signal.
**`source` accepts only the values in `LEAD_SOURCE`** (`backend/common/utils.py:59-67`): `call`,
`email`, `existing customer`, `partner`, `public relations`, `compaign`, `other`. There is no
`website` value in this enum — that spelling belongs to a completely different, uppercase vocabulary
(`Opportunity.lead_source`, using `SOURCES`; see [Opportunities](opportunities.md#stages)). Note also
that `compaign` is not a typo in this documentation — it is the literal stored value for "Campaign"
leads. A client that sends `source=campaign` (the correctly-spelled word) gets zero rows back, not an
error, because `?source=` is an exact match (`lead_views.py:115-116`), and a client that tries to
*create* a lead with `source: "campaign"` gets a validation error, because `campaign` is not one of
the choices `Lead.source` accepts (`leads/models.py:75-77`).
Filters beyond `source` and `status` — `name`, `salutation`, `assigned_to`, `tags`, `city`, `email`,
`rating`, `search`, `created_at__gte/lte`, `close_date__gte/lte`, `cf_<key>` — are documented in full,
with exact match-vs-contains semantics for each, in
[Conventions: Filtering and search](conventions.md#filtering-and-search), which uses this exact
endpoint as its example.
## Create a lead
`POST /api/leads/` (`LeadListView.post`, `lead_views.py:273-409`) accepts `multipart/form-data` (for
an optional `lead_attachment` file, read straight off `request.FILES`, `lead_views.py:331`) or JSON.
The body is validated by `LeadCreateSerializer` (`leads/serializer.py:88-224`); see
[Fields](#fields) below for exactly which keys it accepts. On success (`200`, not `201` — see
[Errors: Status codes](errors.md#status-codes)):
```json
{"error": false, "message": "Lead Created Successfully"}
```
Sending `"status": "converted"` on create runs the lead straight through
`leads.services.convert_lead_to_account` and returns a different shape instead:
```json
{
"error": false,
"message": "Lead Converted Successfully",
"account_id": "<uuid>",
"contact_id": "<uuid or null>",
"opportunity_id": "<uuid or null>"
}
```
A validation failure returns `400` with `{"error": true, "errors": {...}}`
(`lead_views.py:406-409`), the shape [Errors](errors.md#validation-errors) documents. Two validators
worth knowing about before you integrate:
- **Email uniqueness is case-insensitive per org**, and only checked when an email is actually
supplied (`validate_email`, `leads/serializer.py:107-134`):
`{"error": true, "errors": {"email": ["Another lead in this organisation already uses that email address."]}}`.
- **Email becomes required, but only when `status` is `"converted"`** — `LeadCreateSerializer.__init__`
flips `self.fields["email"].required = True` exactly in that case, having otherwise made
`first_name`, `last_name` and `salutation` optional regardless of what the `Lead` model itself
requires (`leads/serializer.py:100-104`). Every other field on this serializer is optional; see
[Fields](#fields).
`tags`, `contacts`, `teams` and `assigned_to` are **not** part of `LeadCreateSerializer` — the view
reads each straight off the request body after the serializer succeeds, resolved against
`org=request.profile.org` once parsed. They are **not** handled uniformly, and the gap matters because
this same endpoint accepts `multipart/form-data` (above) — see [Fields](#fields) for the exact
per-verb, per-field table and the concrete failure a multipart request hits.
## Retrieve, update, delete
`GET /api/leads/{id}/` (`LeadDetailView.get`, `lead_views.py:568-571`) looks the lead up scoped to the
caller's org (`get_object`, `:416-417` — a lead in another org is a `404`, same as one that never
existed, matching the cross-org rule [Errors](errors.md#not-found-versus-forbidden) states generally).
It then calls `assert_lead_access` (`:419-443`): **an admin or superuser sees any lead; anyone else
must be the lead's creator or one of its assignees, or the request is refused with a `403`** — not a
`404` — even though `LeadListView` would have hidden that same lead from that same user entirely.
[Errors: Not found versus forbidden](errors.md#not-found-versus-forbidden) doesn't name leads at
all — its own text says the within-org choice between `403` and `404` "depends on what the resource
is," naming invoices (`403`) and personal macros (`404`) as its two worked examples — so treat this
paragraph, not that page, as the source for what leads actually does: there is no comment or test in
`leads/` recording this as a deliberate disclosure decision the way there is for invoices or macros;
it's simply the behavior as written. The response nests the record under `lead_obj`, matching the
shape [Conventions](conventions.md#response-shape) documents for this exact endpoint, alongside
`attachments`, `comments`, `users_mention`, `assigned_data`, `users`, `users_excluding_team`, `source`,
`status`, `teams`, `countries` and `custom_field_definitions` (`lead_views.py:513-540`).
`PUT /api/leads/{id}/` (`:674-823`) and `PATCH /api/leads/{id}/` (`:847-1064`) both enforce the same
creator-or-assignee-or-admin rule before touching anything, and both can convert the lead when
`status` (or, for `PATCH` only, `is_converted`) is set. `leads/workflow.py` is explicit that this
guard covers less than "any already-closed-off status": it deliberately separates two ideas that used
to be conflated under one name (`leads/workflow.py:11-30`). **`IRREVERSIBLE_STATUSES` is `{"converted"}`
alone** (`:34`) — converting an already-`converted` lead a second time is refused with `400` rather
than silently creating a second `Opportunity` against the same `Account`. `CLOSING_STATUSES` is the
separate, deliberately-*reversible* set `{"closed"}` (`:37`; "reopening a closed lead is ordinary
work," per the module docstring at `:24`), and `recycled` is in neither set. **Converting a `closed`
or `recycled` lead succeeds** and
runs the full conversion, same as converting an `assigned` or `in process` one — there is no guard
against it. `PATCH`'s conversion branch returns before `LeadCreateSerializer` (and its
`validate_status`, which enforces the same `IRREVERSIBLE_STATUSES`-only rule) ever runs, so it repeats
the check inline instead (`lead_views.py:875-894`). Both verbs re-attach `tags`/`contacts`/`teams`/
`assigned_to` from the request body when present, but — like `POST` — not identically across the four
fields; see [Fields](#fields) for the exact per-verb table. Success returns
`{"error": false, "message": "Lead updated Successfully"}` (or the conversion shape above); failure
returns `{"error": true, "errors": {...}}`.
`DELETE /api/leads/{id}/` (`:1080-1095`) allows an admin, a superuser, or the lead's own creator
(`request.profile.user == self.object.created_by`) — anyone else gets `403` — and additionally
requires the lead's org to match the caller's, even though `get_object` already guarantees that by
construction.
## Comments and attachments
Adding a comment or an attachment to a lead is not a separate endpoint — it's the same
`POST /api/leads/{id}/` route the create endpoint uses, just with an existing `pk`
(`LeadDetailView.post`, `lead_views.py:589-651`). The caller must be an admin/superuser, the lead's
creator, or one of its assignees (`:599-610`). Send `comment` (text) and/or `lead_attachment` (a
multipart file) — either or both — and the response echoes the same `lead_obj`/`attachments`/`comments`
shape as `GET`.
Editing or removing an *existing* comment goes through a dedicated route:
`PUT` / `PATCH` / `DELETE /api/leads/comment/{id}/` (`LeadCommentView`,
`backend/leads/views/lead_interactions.py:60-183`) — restricted to an admin, a superuser, or the
comment's own author (`request.profile == obj.commented_by`; `commented_by` is a `Profile` foreign
key, so this comparison is correct, unlike some `created_by` comparisons elsewhere in this codebase —
see [Architecture: Permissions and roles](../architecture/permissions-and-roles.md#object-level-checks)).
`DELETE /api/leads/attachment/{id}/` (`LeadAttachmentView.delete`, `lead_interactions.py:203-221`) has
a real gap, documented in
[Architecture: Permissions and roles](../architecture/permissions-and-roles.md#object-level-checks):
the lookup is `self.model.objects.get(pk=pk)` with **no `org=` filter at all**
(`lead_interactions.py:204`), so the id space for attachment deletion is not scoped to the caller's
org by the ORM query — only PostgreSQL RLS, if correctly configured for the deployment, stands between
this endpoint and a cross-tenant delete. The ownership check on the same endpoint
(`request.profile.user == self.object.created_by`, `:208`) is correct — it compares a `User` to a
`User` — so once an attachment is found, only its uploader, an admin, or a superuser can delete it;
the defect is specifically the missing org scope in the lookup, not the permission check.
## Fields
`LeadCreateSerializer.Meta.fields` (`leads/serializer.py:178-213`) is what `POST /api/leads/`,
`PUT /api/leads/{id}/` and `PATCH /api/leads/{id}/` accept. Every field the `Lead` model itself
declares is optional (`blank=True, null=True`, or has a default) except `first_name` and `last_name`,
which have `null=True` but not `blank=True` at the model level and would default to required under
DRF's normal derivation — the serializer's `__init__` explicitly overrides both to `required=False`
(`leads/serializer.py:102-103`, `leads/models.py:49-50`). The practical result: **a lead can be
created with no name at all**, as long as `status` isn't `"converted"` (which requires `email`
instead — see [Create a lead](#create-a-lead)).
| Field | Type | Required | Notes |
| --- | --- | --- | --- |
| `title` | string | optional | Free text, e.g. "Enterprise Deal" |
| `salutation` | string | optional | |
| `first_name` | string | optional | Forced optional by the serializer; see above |
| `last_name` | string | optional | Forced optional by the serializer; see above |
| `email` | string | **required only when `status: "converted"`** | Must be unique per org, case-insensitive |
| `phone` | string | optional | |
| `job_title` | string | optional | |
| `website` | string | optional | Free text, not validated as a URL |
| `linkedin_url` | string | optional | Validated as a URL |
| `status` | one of `LEAD_STATUS` | optional | `assigned`, `in process`, `converted`, `recycled`, `closed` |
| `source` | one of `LEAD_SOURCE` | optional | `call`, `email`, `existing customer`, `partner`, `public relations`, `compaign`, `other` — no `website` |
| `industry` | one of `INDCHOICES` | optional | e.g. `SOFTWARE` |
| `rating` | `HOT` \| `WARM` \| `COLD` | optional | |
| `opportunity_amount` | decimal | optional | Non-negative (DB constraint) |
| `currency` | one of `CURRENCY_CODES` | optional | Defaults from the org's `default_currency` if omitted and `opportunity_amount` is set |
| `probability` | integer, 0–100 | optional | |
| `close_date` | date | optional | |
| `address_line`, `city`, `state`, `postcode` | string | optional | |
| `country` | one of `COUNTRIES` | optional | |
| `last_contacted`, `next_follow_up` | date | optional | |
| `description` | text | optional | |
| `company_name` | string | optional | |
| `is_active` | boolean | optional | Defaults `true` |
Not part of the serializer, but accepted in the same request body and resolved by the view on
`POST`/`PUT`/`PATCH` (each a list of ids, org-scoped): `tags`, `contacts`, `teams`, `assigned_to`; plus
`lead_attachment` (a multipart file). These four id-list fields are **not parsed the same way on every
verb**, and the difference is a real trap for a client that also uploads a file (multipart fields
arrive as strings, never as JSON arrays):
| Field | `POST /api/leads/` | `PUT /api/leads/{id}/` | `PATCH /api/leads/{id}/` |
| --- | --- | --- | --- |
| `tags` | plain `id__in=data.get("tags")` — no string handling (`lead_views.py:318-323`) | same gap (`:715-721`) | `json.loads`'d if a string, `{"id": …}` unwrapped (`:981-995`) |
| `contacts` | same gap (`:325-329`) | `json.loads`'d if a string (`:732-745`) | `json.loads`'d if a string (`:997-1011`) |
| `teams` | `json.loads`'d if a string (`:340-350`) | same (`:747-758`) | same (`:1013-1027`) |
| `assigned_to` | `json.loads`'d if a string (`:352-364`) | same (`:760-773`) | same (`:1029-1043`) |
Sending `tags` or `contacts` as a JSON-encoded string in the same request as a `multipart/form-data`
`lead_attachment` upload — on `POST`, or on `PUT` for `tags` specifically — hands Django's `id__in`
lookup a raw string. Django iterates it character by character; every character fails `UUIDField`
parsing, and the request dies as an unhandled `500` rather than a clean `400` — the same failure mode
already documented for a different field at `contacts/views.py:105-108`. Sending those two fields as a
real JSON body (no file) works fine, because DRF parses a JSON request body into an actual list before
the view ever reads it.
`GET /api/leads/{id}/` and `GET /api/leads/` additionally return, but never accept as input: `id`,
`created_by`, `created_at`, `updated_at`, `is_sample` (server-set only — see
`leads/serializer.py:80-85`), `lead_attachment` and `lead_comments` (nested read-only serializations of
this lead's attachments and comments — `leads/serializer.py:64-65`), `stage` and `kanban_order`
(managed by the Kanban endpoints, not this page), and `custom_fields` (validated separately against the
org's `CustomFieldDefinition` rows).

186
docs/api/opportunities.md Normal file
View file

@ -0,0 +1,186 @@
# Opportunities
Routes in `backend/opportunity/urls.py`, views split across `opportunity/views/opportunity_views.py`
(list and detail) and `opportunity/views/kanban_views.py` (stage board and move). See
[Conventions](conventions.md) for pagination, filtering and response-shape rules assumed rather than
repeated here.
## List opportunities
`GET /api/opportunities/` (`OpportunityListView.get`, `backend/opportunity/views/opportunity_views.py:272-274`)
returns a single paginated list plus org-wide `totals` computed over the whole filtered queryset, not
just the current page (`get_totals`, `:73-110`):
```json
{
"opportunities_count": 23,
"totals": {"count": 23, "amount_sum": "184500.00", "weighted_sum": "97250.00", "stalled_count": 2},
"offset": 10,
"per_page": 10,
"page_number": [1],
"opportunities": [ { "...": "OpportunitySerializer" } ],
"accounts_list": [ { "...": "AccountSerializer" } ],
"contacts_list": [ { "...": "ContactSerializer" } ],
"tags": [ { "...": "..." } ],
"stage": [["PROSPECTING", "Prospecting"], ["QUALIFICATION", "Qualification"]],
"lead_source": [["NONE", "NONE"], ["CALL", "CALL"], ["WEBSITE", "WEBSITE"]],
"currency": [["USD", "USD, Dollar"]]
}
```
`weighted_sum` is the forecast — `SUM(amount * probability / 100)` — as opposed to `amount_sum`,
which is what every open and closed deal in the filtered set is worth at face value
(`:90-98`). `stalled_count` uses the same per-stage aging thresholds as `?rotten=true` below and as
`Opportunity.get_aging_status()`; see [Stages](#stages). A non-admin caller only sees deals they
created or are assigned to (`:119-131`).
Filters: `name`, `search` (both `icontains` against `name` only — `search` here does **not** OR
across multiple fields the way it does on [Leads](leads.md#list-leads)); `account` (exact); `stage`,
`lead_source` (**`__contains`, not exact** — see [Stages](#stages) for why this matters); `tags`,
`assigned_to` (repeatable, id lists); `created_at__gte/lte`, `closed_on__gte/lte`,
`amount__gte/lte`; `cf_<key>`; and two boolean flags, `open=true` (excludes closed stages) and
`rotten=true` (stalled deals only) — see [Stages](#stages) (`opportunity_views.py:133-192`).
## Create an opportunity
`POST /api/opportunities/` (`OpportunityListView.post`, `opportunity_views.py:292-413`) is validated
by `OpportunityCreateSerializer` (`opportunity/serializer.py:199-297`); see [Fields](#fields). On
success (`200`):
```json
{"error": false, "message": "Opportunity Created Successfully", "id": "<uuid>"}
```
**`name` must be unique per org, case-insensitive** (`validate_name`,
`opportunity/serializer.py:213-228`). The serializer also enforces, at the API layer, the two rules
`Opportunity.clean()` states at the model layer — closed deals need a close date, and `CLOSED_WON`
deals need an amount (`validate()`, `:242-268`; see [Stages](#stages)) — because DRF's
`ModelSerializer` never calls `Model.clean()`, so without this the model's own rules would only be
enforced by unit tests against the model, never through the API.
`contacts`, `tags`, `teams` and `assigned_to` are resolved from the request body the same
list-of-ids way as [Leads](leads.md#create-a-lead) and [Accounts](accounts.md#create-an-account)
(`opportunity_views.py:317-376`); `opportunity_attachment` is accepted as a multipart file.
## Retrieve, update, delete
`GET /api/opportunities/{id}/` (`OpportunityDetailView.get`, `opportunity_views.py:661-736`) looks the
deal up scoped to the caller's org — unlike leads/accounts/contacts, this lookup returns `None` rather
than raising, so a missing or cross-org id comes back as an explicit `404` response
(`"error": true, "errors": "Opportunity not found."`, `:663-667`) instead of Django's
`get_object_or_404`. `assert_deal_access` (`:423-447`) then requires the caller to be an admin, the
deal's creator, or one of its assignees, or the request is refused with `403`. The response nests the
record under `opportunity_obj`, and also returns `comments`, `attachments`, `contacts`,
`users`, `stage`, `lead_source`, `currency`, `comment_permission`, `users_mention` and
`custom_field_definitions`.
`PUT /api/opportunities/{id}/` (`:464-596`) and `PATCH /api/opportunities/{id}/` (`:831-953`) both
enforce `assert_deal_access`, and both stamp `closed_by = request.profile` and persist it with an
explicit `.save()` whenever the incoming `stage` is `CLOSED_WON` or `CLOSED_LOST`
(`:537-539`, `:942-944`) — a deal moved into a closed stage always records who closed it. Both clear
and re-add `contacts`/`tags`/`teams`/`assigned_to` from the request body the same way create does.
Success: `{"error": false, "message": "Opportunity Updated Successfully"}`.
`DELETE /api/opportunities/{id}/` (`:612-637`) allows an admin, a superuser, or the deal's own creator
(`request.profile.user != self.object.created_by`, a `User`-to-`User` comparison, `:624-632`) —
anyone else gets `403`.
## Stages
**`stage` is a fixed, org-wide enum, not a configurable pipeline.** `Opportunity.stage` is a
`CharField` with `choices=STAGES` and `default="PROSPECTING"`
(`opportunity/models.py:54-56`; `STAGES` in `backend/common/utils.py:87-94`):
`PROSPECTING`, `QUALIFICATION`, `PROPOSAL`, `NEGOTIATION`, `CLOSED_WON`, `CLOSED_LOST`. There is no
`Pipeline`/`Stage` model behind it the way there is for the other three record types that have a
Kanban board in this API — `LeadPipeline`/`LeadStage` (`leads/models.py:229-330`),
`CasePipeline`/`CaseStage` (`cases/models.py:490,527`), and `Board`/`BoardColumn` for tasks
(`tasks/models.py:16,83`) are all per-org, admin-configurable. Opportunity's Kanban view says this
about itself in its own module docstring: "Status-based only (Opportunity has no Pipeline/Stage model
— it groups by the flat `stage` CharField)" (`opportunity/views/kanban_views.py:1-6`). Every org gets
the same six stages; there is no endpoint to add, rename or reorder them.
Two consequences of moving a deal between stages, both enforced server-side (see
[Create an opportunity](#create-an-opportunity)): entering `CLOSED_WON` or `CLOSED_LOST` requires
`closed_on`; entering `CLOSED_WON` specifically also requires `amount`
(`CLOSED_STAGES`, `AMOUNT_REQUIRED_STAGES`, `opportunity/workflow.py:17-21`). `probability` is
auto-filled from a fixed per-stage default (`STAGE_PROBABILITIES`, `workflow.py:8-15`) whenever the
value on the record is `0` or unset (`Opportunity.save()`, `opportunity/models.py:262-264`) — sending
your own `probability` still works, this only fills the gap when you don't.
**Filtering by stage is a substring match, not an exact one** — `?stage=` and `?lead_source=` both
compile to `__contains` (`opportunity_views.py:138-143`), unlike `Lead.status`/`Lead.source`, which
are exact matches (see [Leads](leads.md#list-leads)). `?stage=CLOSED` therefore matches both
`CLOSED_WON` and `CLOSED_LOST`; `?stage=closed` (lowercase) matches neither, because `__contains` on a
`CharField` is case-sensitive.
`?rotten=true` on `GET /api/opportunities/` and the `stalled_count` in its `totals` both use the same
definition as `Opportunity.get_aging_status()`: a deal is stalled once it has sat in its current,
non-closed stage for at least `expected_days * 1.5` (`ROTTEN_MULTIPLIER`, `workflow.py:32`), where
`expected_days` comes from the org's `StageAgingConfig` for that stage if one exists, else a built-in
default (`DEFAULT_STAGE_EXPECTED_DAYS`, `workflow.py:24-29`; `stalled_filter`,
`opportunity_views.py:43-66`).
`GET /api/opportunities/kanban/` (`OpportunityKanbanView.get`, `kanban_views.py:39-120`) returns one
column per `STAGES` value, applying the same org/role scoping and a subset of the list filters
(`search`, `account`, `assigned_to`, `tags`, `closed_on__gte/lte`):
```json
{
"mode": "status",
"pipeline": null,
"columns": [
{"id": "PROSPECTING", "name": "Prospecting", "order": 1, "color": "#3B82F6", "stage_type": "open", "is_status_column": true, "wip_limit": null, "item_count": 4, "items": [ { "...": "OpportunityKanbanCardSerializer" } ]}
],
"total_items": 23
}
```
`"pipeline": null` is not a placeholder for missing data — it is always `null` here, precisely because
there is no pipeline object behind this board.
`PATCH /api/opportunities/{id}/move/` (`OpportunityMoveView.patch`, `kanban_views.py:140-204`)
**requires `stage`** — `OpportunityMoveSerializer.stage` is a `ChoiceField` with no `required=False`
(`opportunity/serializer.py:362-367`, whose own docstring says so explicitly), and the view reads
`data["stage"]` unconditionally (`kanban_views.py:178`) — a reorder-only body with no `stage` key gets
a `400`, not a silent no-op reorder. When `stage` is sent, `kanban_order` can optionally come with it
to reorder within the (possibly new) column at the same time (fractional indexing, same convention as
tasks — averaging two neighbors' `kanban_order`, or offsetting by ±1000 with only one neighbor given).
Its own access check
intends to mirror `assert_deal_access` (admin, creator, or assignee) but the creator half is broken:
`is_owner = request.profile == opportunity.created_by` compares a `Profile` to `Opportunity.created_by`,
which is a `User` foreign key — the two are never equal, so a non-admin who created a deal but did not
also assign it to themselves cannot move it and gets `403` (`kanban_views.py:161-168`). This is one of
the two open `created_by` type-mismatch call sites in this codebase's `opportunity` app; see
[Architecture: Permissions and roles](../architecture/permissions-and-roles.md#object-level-checks)
for the other and for why it hasn't been fixed as part of this documentation task.
## Fields
`OpportunityCreateSerializer.Meta.fields` (`opportunity/serializer.py:270-289`) is what
`POST /api/opportunities/`, `PUT /api/opportunities/{id}/` and `PATCH /api/opportunities/{id}/`
accept. Only `name` is required at the model level (`opportunity/models.py:46`, no `blank=True`).
| Field | Type | Required | Notes |
| --- | --- | --- | --- |
| `name` | string | **required** | Unique per org, case-insensitive |
| `account` | uuid | optional | |
| `stage` | one of `STAGES` | optional | Defaults `PROSPECTING`; see [Stages](#stages) |
| `opportunity_type` | one of `OPPORTUNITY_TYPES` | optional | `NEW_BUSINESS`, `EXISTING_BUSINESS`, `RENEWAL`, `UPSELL`, `CROSS_SELL` |
| `currency` | one of `CURRENCY_CODES` | optional | Defaults from the org's `default_currency` if omitted |
| `amount` | decimal | **required if `stage: "CLOSED_WON"`** | Otherwise optional; non-negative (DB constraint) |
| `probability` | integer, 0–100 | optional | Auto-filled from `stage` if `0`/unset — see [Stages](#stages) |
| `closed_on` | date | **required if `stage` is `CLOSED_WON` or `CLOSED_LOST`** | Otherwise optional |
| `lead_source` | one of `SOURCES` | optional | Uppercase — `NONE`, `CALL`, `EMAIL`, `EXISTING CUSTOMER`, `PARTNER`, `PUBLIC RELATIONS`, `CAMPAIGN`, `WEBSITE`, `OTHER`. **Not** `Lead.source`'s `LEAD_SOURCE` — see [Leads](leads.md#list-leads) |
| `description` | text | optional | |
| `is_active` | boolean | optional | Defaults `true` |
Not part of the serializer, but accepted in the same request and resolved by the view (see
[Create an opportunity](#create-an-opportunity)): `contacts`, `tags`, `teams`, `assigned_to` (each a
list of ids, org-scoped), and `opportunity_attachment` (a multipart file).
`GET /api/opportunities/` and `GET /api/opportunities/{id}/` additionally return, but never accept as
input: `id`, `closed_by`, `created_by`, `created_at`, `org` (nested), `created_on_arrow`,
`amount_source` (`MANUAL` or `CALCULATED` — server-derived by `recalculate_amount()` from whether the
deal has line items, `opportunity/models.py:68-73,185-202`; not writable through this serializer),
`stage_changed_at`, `days_in_stage`, `aging_status` (`green`/`yellow`/`red`, from
`Opportunity.get_aging_status()`), `line_items`, `line_items_total`, and `custom_fields` (validated
separately against the org's `CustomFieldDefinition` rows).

292
docs/api/tasks.md Normal file
View file

@ -0,0 +1,292 @@
# Tasks
Routes in `backend/tasks/urls.py`. CRUD, comments and attachments are in
`backend/tasks/views/task_views.py`; Kanban pipelines/stages and the move endpoint in
`backend/tasks/views/kanban_views.py`; Kanban **boards** — a separate feature — in
`backend/tasks/views/board_views.py`. Object-level authorization is centralized in
`backend/tasks/access.py`. See [Conventions](conventions.md) for pagination, filtering and
response-shape rules assumed rather than repeated here, and [Errors](errors.md) for the
404-vs-403 principle referenced below.
## `Task.save()` calls `full_clean()`
Before anything else, one fact that changes how you should read every write endpoint below:
**`Task.save()` explicitly calls `self.full_clean()`** (`tasks/models.py:402-404`). This is not the
default — [Architecture: Data model](../architecture/data-model.md#conventions) documents that most
models here (`Lead`, `Opportunity`, `Case`) define real validation in `clean()` that is never invoked
through the API, because their `save()` methods don't call `full_clean()`. `Task` is one of a small
handful that do (alongside `Comment.save()` and `Attachments.save()`, which enforce that a comment or
attachment's `org` matches its parent record's org — same page). The consequence is the opposite
problem: when `full_clean()` *does* run and finds a violation, it raises Django's own
`django.core.exceptions.ValidationError` — not DRF's — and [Errors](errors.md) states plainly, in its
own opening paragraph, that this codebase's `EXCEPTION_HANDLER` is DRF's default, unmodified, which
does not know how to render that exception. Unless the specific view code path catches it, a
model-level rule violation on a `Task` reaches the client as an unhandled **`500`**, not a clean `400`.
The one rule `Task.clean()` currently enforces is "at most one parent entity" — a task may link to
at most one of `account`, `opportunity`, `case`, `lead` (`tasks/models.py:384-400`). The main
create/update surface guards against this twice: `TaskCreateSerializer.validate()`
(`tasks/serializer.py:331-365`) re-implements the same rule and returns a clean `400` with the field
named, *and* `TaskListView.post` / `TaskDetailView.put` / `TaskDetailView.patch` additionally wrap
`serializer.save()` in `try/except django.core.exceptions.ValidationError` as a backstop
(`_model_errors`, `tasks/views/task_views.py:56-66`, used at `:277-281,577-581,702-706`) — so for
`POST /api/tasks/`, `PUT /api/tasks/{id}/` and `PATCH /api/tasks/{id}/` specifically, this rule
reaches the client as `{"error": true, "errors": {"account": "..."}}` with `400` today, not a `500`.
**Not every write path shares that backstop.** `PATCH /api/tasks/{id}/move/`
(`TaskMoveView.patch`, `tasks/views/kanban_views.py:253-318`) — the Kanban drag-and-drop endpoint —
sets `task.stage`, `task.status` and `task.kanban_order` directly and calls `task.save()` with no
`try/except` around it (`:310`) and without going through `TaskCreateSerializer` at all. `full_clean()`
validates every field on the instance, not only the ones a given endpoint touched, so any model-level
violation it finds here — on this task's existing `account`/`opportunity`/`case`/`lead` combination, or
any other `clean_fields()` rule — surfaces as the unhandled `500` described above. This isn't only a
theoretical exposure through the parent-entity fields: `TaskStage.maps_to_status`
(`tasks/models.py:257-263`) is a plain `CharField`, `blank=True, null=True`, with no `choices=`
constraint — free text an admin sets when configuring a pipeline stage, not validated against
`Task.STATUS_CHOICES` at all. When a task moves into a stage that has one set,
`if stage.maps_to_status: task.status = stage.maps_to_status` (`kanban_views.py:297-298`) assigns it
straight onto `task.status` — a field that *does* have `choices=` at the model level. A single typo'd
`maps_to_status` on one admin-configured stage turns every future move into that stage into an
unhandled `500` for every user, until an admin notices and fixes the stage. This endpoint has no
equivalent of the two-layer guard the main CRUD endpoints have.
## List tasks
`GET /api/tasks/` (`TaskListView.get`, `tasks/views/task_views.py:233-235`) returns one paginated
list. Visibility is `visible_tasks_qs` (`tasks/access.py:43-48`) — admin sees every task in the org;
anyone else sees only tasks they created or are assigned to. This is the same rule the detail view
enforces (`tasks/access.py` module docstring: "the list has to agree with the detail view or the queue
lies").
```json
{
"tasks_count": 24,
"offset": 10,
"tasks": [ { "...": "TaskListSerializer" } ],
"totals": {"count": 24, "open": 18, "overdue": 3, "due_today": 1, "due_this_week": 5, "no_due_date": 6, "unassigned": 2},
"status": [["New", "New"], ["In Progress", "In Progress"], ["Completed", "Completed"]],
"priority": [["Low", "Low"], ["Medium", "Medium"], ["High", "High"]],
"users": [{"id": "...", "user__email": "..."}],
"accounts_list": [ { "...": "AccountSerializer" } ],
"contacts_list": [ { "...": "ContactSerializer" } ]
}
```
(`tasks/views/task_views.py:130-209`.) `totals` is one aggregate query over the whole filtered,
visibility-scoped queryset, not the page — `overdue`/`due_today`/`due_this_week`/`no_due_date`/
`unassigned` all additionally require the task still be open (`status != "Completed"`), so a task that
finished late does not count as overdue (`:145-167`). `accounts_list`/`contacts_list` are the full org
catalogues for the parent-entity picker; `?slim=true` omits both (`:203-209`).
Filters, read directly from `request.query_params`
(`tasks/views/task_views.py:86-129`): `title` (contains), `status` (exact), `priority` (exact),
`assigned_to` (repeatable id list), `tags` (id list), `search` (matches `title`, contains — not a
combined search across other fields the way [Leads](leads.md#list-leads) or
[Contacts](contacts.md#list-contacts) define one), `due_date__gte`/`due_date__lte`,
`created_at__gte`/`created_at__lte`, `account`, `opportunity`, `case`, `lead` (each an exact parent-id
match), and `cf_<key>` (custom field equals).
## Create a task
`POST /api/tasks/` (`TaskListView.post`, `tasks/views/task_views.py:252-349`) accepts JSON or
`multipart/form-data`, validated by `TaskCreateSerializer` (`tasks/serializer.py:315-389`; see
[Fields](#fields)). On success (`200`): `{"error": false, "message": "Task Created Successfully"}`. A
validation failure returns `400` with `{"error": true, "errors": {...}}` — including the "at most one
parent" rule and the model's own `full_clean()` backstop, both described [above](#tasksave-calls-full_clean).
`account`, `opportunity`, `case` and `lead` are each scoped to the caller's org in
`TaskCreateSerializer.__init__` (`tasks/serializer.py:328-329`) — the comment there notes a plain
`ModelSerializer` would otherwise build each as a `PrimaryKeyRelatedField` over *every* row in the
table, and that a task in one org was proven to accept another org's account, with the list endpoint
then rendering that org's account name back to the requester. `created_by` is listed in
`Meta.fields` but is `read_only` (`:389`) — the comment explains that leaving it writable let a client
name any user in any org as a task's creator, which becomes a privilege-escalation vector the moment
the creator clause in `tasks/access.py` is consulted for anything (which it is — see
[Fields](#fields)).
`contacts`, `teams`, `assigned_to` and `tags` are **not** part of `TaskCreateSerializer` — the view
resolves each from the request body after the serializer succeeds, org-scoped, `json.loads`'d when
sent as a string (the same shape [Cases](cases.md#create-a-case) uses). *Parsing* is consistent across
all four fields on every verb — tasks does not have the per-field inconsistency
[Leads](leads.md#fields) documents. *Writing* is a different story, and the same data-loss trap
[Cases](cases.md#fields) documents applies here verbatim: `POST` only adds
(`task_views.py:282,296,308,322`); `PUT` calls `.clear()` on all four **unconditionally**, before
checking whether the body mentioned them at all (`:582,597,610,625`), so a `PUT` that omits
`assigned_to` unassigns the task from everyone — including, via `visible_tasks_qs`
(`tasks/access.py:43-48`), removing it from those people's task lists entirely; `PATCH` clears a given
M2M only when its key is present in the body (`:709-758`).
## Retrieve, update, delete
`GET /api/tasks/{id}/` (`TaskDetailView.get`, `tasks/views/task_views.py:460-463`) org-scopes the
lookup (`get_task_or_404`, `tasks/access.py:51-65` — a malformed id or a cross-org task both answer
`404`) and enforces `assert_task_access` before building the response (`:365`) — raised, not returned,
specifically because a `Response` object handed back through this method used to be wrapped in a
second `Response` and crash with a `500` instead of answering `403` (comment at `:361-364`). The
response nests the record under `task_obj`, alongside `attachments`, `comments`, `users_mention`,
`assigned_data`, `custom_field_definitions`, `users` (admins see every active member; non-admins see
only other admins — the same shape [Cases](cases.md#list-cases) uses for its assignee picker),
`users_excluding_team` and `teams`.
`PUT /api/tasks/{id}/` (`:545-650`) and `PATCH /api/tasks/{id}/` (`:667-781`) both enforce
`assert_task_access` — access here has only two tiers, not three the way a case does: **`access`
(read/write together) is admin · creator · assignee; `delete` is the narrower admin · creator**
(`tasks/access.py:1-10`). The module docstring is explicit about why a task collapses read and write
into one rule where a case does not: a case can have a *watcher*, someone following a ticket without
being handed it, and that extra read-only tier has no equivalent on a task — "everyone who may open one
may also work it." `contacts`/`teams`/`assigned_to`/`tags` are re-attached from the request body the
same way `POST` does. Success: `{"error": false, "message": "Task updated Successfully"}`.
`DELETE /api/tasks/{id}/` (`:797-805`) calls `assert_task_delete_access` — admin or creator only, no
assignee exception, same reasoning as [Cases](cases.md#retrieve-update-delete): being assigned a task
is a reason to work it, not to erase it.
## Comments and attachments
The same shape as [Cases](cases.md#comments-and-attachments): `POST /api/tasks/{id}/`
(`TaskDetailView.post`, `tasks/views/task_views.py:481-528`) adds a `comment` and/or a
`task_attachment` file to an existing task, gated by `assert_task_access`. The comment on this method
notes a real trap: validating the comment through `CommentSerializer` fails *silently* here, because
that serializer requires `object_id` and `org`, neither of which the client sends — `is_valid()`
returns `False` and the intended write is simply skipped, so the view constructs the `Comment` directly
with Django's ORM instead of going through the serializer (`:488-500`).
`PUT`/`PATCH`/`DELETE /api/tasks/comment/{id}/` (`TaskCommentView`, `:808-946`) edit or remove an
existing comment — admin, or the comment's own author (`request.profile == obj.commented_by`, a
`Profile`-to-`Profile` comparison, correct). **`PUT` cannot, in practice, edit a comment.** It builds
`CommentSerializer(obj, data=params)` with no `partial=True` (`:851`), and `CommentSerializer.Meta.fields`
includes the plain (not `read_only`) model fields `object_id` and `org`
(`common/serializer.py:239-250`) — a body of just `{"comment": "..."}`, everything the documented
`TaskCommentEditSwaggerSerializer` contract asks for, fails validation on the missing `object_id`/`org`
and answers `400`. Only `.patch()` (`partial=True`, `:896`) can actually change a comment's text.
Separately, because `object_id`/`org` genuinely are writable on `PATCH`, a comment's own author can
supply both there to repoint the comment at a different record, including one in another org or one
they cannot otherwise read — `Comment.clean()` (`common/models.py:299-314`) only catches the mismatched
case (`org` doesn't match the new target's `org`), and only because `Comment.save()` calls
`self.full_clean()` (`:316-318`), which turns that mismatch into an unhandled `500` rather than a `400`
(same failure mode as [above](#tasksave-calls-full_clean), on a different model); a *matching* cross-org
pair saves without error. Neither behavior is fixed here — reported per this documentation task's
scope. The identical `PUT`/`PATCH` pattern exists on [Cases](cases.md#comments-and-attachments)' own
comment-edit endpoint too.
`DELETE /api/tasks/attachment/{id}/` (`TaskAttachmentView.delete`, `:949-1000`) is org-scoped
(`.filter(pk=pk, org=request.profile.org)`, `:975-977`) and its ownership check compares
`request.profile.user_id` to `self.object.created_by_id` (`:987`) — both fixed here. The comment on
this method documents that both defects (an unscoped lookup across every org's attachments, and a
`Profile`-to-`User` comparison that silently made the endpoint admin-only) were proven live before the
fix — see [Architecture: Permissions and roles](../architecture/permissions-and-roles.md#object-level-checks)
for the current, cross-app state of the same defect pattern in other modules.
## Boards and columns
**Kanban *boards* are a separate feature from a task's own `stage`/`pipeline` fields, with their own
model family (`Board`, `BoardColumn`, `BoardTask`, `BoardMember` — `tasks/models.py:16-194`) and their
own endpoints mounted at `/api/boards/...`, not under `/api/tasks/...`.** Confirmed in
`backend/common/app_urls/__init__.py:26`: `path("boards/", include((board_urlpatterns, "api_boards")))`,
where `board_urlpatterns` is imported from `tasks/urls.py:58-75`. A `BoardTask` card is not a `Task`
row — it has its own title/description/priority/assignees and only optionally references an
`account`/`contact`/`opportunity` for context (`tasks/models.py:116-195`); the `/api/tasks/kanban/`,
`/api/tasks/pipelines/...`, `/api/tasks/pipelines/{pipeline_id}/stages/` and `/api/tasks/stages/{id}/`
endpoints documented implicitly above (via `stage` in [Fields](#fields)) operate on real `Task` rows and
are a different Kanban system entirely.
Access to a board is per-board membership, not org-wide: `Board.owner` plus `BoardMember` rows carry a
`role` of `owner`/`admin`/`member` (`tasks/models.py:43-81`). Every endpoint that acts on an *existing*
board checks membership before doing anything (the one exception is creating a board in the first
place — `POST /api/boards/` requires nothing beyond org membership, since there's no board to be a
member of yet); some further gate by role:
| Endpoint | Who |
| --- | --- |
| `GET /api/boards/` | Any authenticated member of the org; results are owner-or-member boards only (`board_views.py:82-89`) |
| `POST /api/boards/` | Any org member; creates the board and, by default, three starter columns (`:126-157`) |
| `GET /api/boards/{id}/` | Board owner or member (`404` otherwise — not `403`, `:165-192`) |
| `PUT/PATCH /api/boards/{id}/` | Board `owner` or `admin` role (`:206-271`) |
| `DELETE /api/boards/{id}/` | Board owner only (`:282-299`) |
| `GET /api/boards/{board_id}/columns/` | Board owner or member (`:316-329`) |
| `POST /api/boards/{board_id}/columns/` | Board `owner` or `admin` role (`:355-367`) |
| `GET/POST /api/boards/columns/{column_id}/tasks/` | Any board member (no role check, `:413-455`) |
| `PUT /api/boards/tasks/{id}/` | Any board member (no role check, `:490-503`) |
| `DELETE /api/boards/tasks/{id}/` | Any board member — the role check lists `owner`/`admin`/`member`, which is every role that can exist (`:570-583`) |
**A malformed board/column/card id is a `500`, not a `404`, on every endpoint in this table.** Every
lookup in `board_views.py` uses Django's `get_object_or_404` against a UUID primary key
(`:167,319,358,416,445,493,573`) — and `get_object_or_404` only catches the model's `DoesNotExist`, not
the `ValidationError`/`ValueError` a non-UUID string raises against a UUID field. This is the exact bug
class `get_task_or_404`/`get_case_or_404` (`tasks/access.py:51-65`, `cases/access.py:57-71`) exist to
prevent on the rest of this API; boards never adopted that helper. Not fixed here — reported per this
documentation task's scope.
**There is no endpoint to update or delete a single column once created.** `board_urlpatterns`
(`tasks/urls.py:58-75`) defines only `GET`/`POST /api/boards/{board_id}/columns/` — a list-and-create
route — with no `columns/{id}/` detail path at all. A column's name, color, order or WIP `limit` cannot
be changed, and a column cannot be removed, through this API today.
**`GET /api/boards/{id}/` does not return each column's cards.** `BoardSerializer.columns` uses
`BoardColumnListSerializer` (`tasks/serializer.py:109-121,123-143`), which reports only a `task_count`
per column, not the cards themselves. To read the actual cards, call
`GET /api/boards/{board_id}/columns/` separately, which uses the full `BoardColumnSerializer` and
nests nested `BoardTaskSerializer` cards under each column (`board_views.py:334-343`).
`PUT /api/boards/tasks/{id}/` (`BoardTaskDetailView.put`, `board_views.py:490-559`) is also how a card
moves between columns: send `column` (validated to be a column of the same board — a column id from
another board, or from another org's board by construction, is a `400`, not a silent no-op,
`:512-529`) and/or `order`; the view renumbers both the source and target columns to a dense
`0..n-1` ordering after the move so drag positions survive a reload (`_resequence_column`, `:22-48`).
`BoardTaskSerializer`'s write surface is deliberately narrow (`tasks/serializer.py:36-90`): a card
write may set `title`, `description`, `order`, `priority`, `due_date`, and assignees (via the
write-only `assigned_to_ids`, applied by the view after save and org-filtered). Everything else is
locked down, by two different mechanisms: `id`, `created_at`, `updated_at`, `completed_at`, `org`,
`column`, `created_by`, `updated_by`, `contact` and `opportunity` are all in `read_only_fields`
(`:67-78`) — `column` because moving a card is the dedicated `PUT` flow above, and the CRM-link FKs
because, unfiltered, they exposed the same cross-org `PrimaryKeyRelatedField` gap
`TaskCreateSerializer` was hardened against. `account` reaches the same outcome a different way: it's
overridden as a plain `SerializerMethodField()` (`:60`), not listed in `read_only_fields` at all —
overriding a model FK with a method field removes it from the write surface just as effectively,
per the field's own comment (`:82-84`).
## Fields
`TaskCreateSerializer.Meta.fields` (`tasks/serializer.py:367-389`) is what `POST /api/tasks/`,
`PUT /api/tasks/{id}/` and `PATCH /api/tasks/{id}/` accept.
| Field | Type | Required | Notes |
| --- | --- | --- | --- |
| `title` | string, max 200 | **required** | Explicitly forced `required=True` in `__init__` (`:321`), matching the model |
| `status` | one of `Task.STATUS_CHOICES` | **required** | `New`, `In Progress`, `Completed` — a different vocabulary from `Case.status`, see [Cases: Fields](cases.md#fields) |
| `priority` | one of `Task.PRIORITY_CHOICES` | **required** | `Low`, `Medium`, `High` — a different vocabulary from `Case.priority` |
| `due_date` | date | optional | |
| `description` | text | optional | |
| `account` | uuid | optional | Org-scoped; at most one of `account`/`opportunity`/`case`/`lead` may be set — see [above](#tasksave-calls-full_clean) |
| `opportunity` | uuid | optional | Same one-parent rule |
| `case` | uuid | optional | Same one-parent rule |
| `lead` | uuid | optional | Same one-parent rule |
| `created_by` | — | **read-only** | Listed in `Meta.fields`, but server-derived; see [Create a task](#create-a-task) |
Not part of `TaskCreateSerializer`, but accepted in the same request body: `contacts`, `teams`,
`assigned_to`, `tags` (each a list of ids, org-scoped — see [above](#retrieve-update-delete) for how
*writing* them diverges across verbs); plus `task_attachment` (a multipart file) and `custom_fields`
(an object). Neither of the last two goes through the serializer either — the view reads
`custom_fields` straight off `request.data`, validates it against the org's
`CustomFieldDefinition` rows, and passes the cleaned result to `serializer.save(custom_fields=...)`
on all three write verbs (`task_views.py:256-276,556-574,681-699`); `task_attachment` is read off
`request.FILES` the same way it is on [Cases](cases.md#create-a-case).
`GET /api/tasks/{id}/` returns the record via `TaskSerializer`; `GET /api/tasks/` returns each row via
the slimmer `TaskListSerializer`, which additionally drops `description`, `custom_fields`, `contacts`
and `teams` (compare `TaskSerializer.Meta.fields`, `tasks/serializer.py:287-309`, against
`TaskListSerializer.Meta.fields`, `:256-271`). **Two fields declared on `TaskSerializer` never actually
reach the client.** `task_attachment` and `task_comments` are declared as nested, read-only serializer
fields (`:280-281`, listed in `Meta.fields` at `:306-307`) — but `Task` has no attribute by either name.
There is no `GenericRelation` defined anywhere in this codebase (`grep -rn "GenericRelation"` outside
the virtualenv returns nothing), so DRF's attempt to read `task_attachment`/`task_comments` off the
instance raises `AttributeError`, which a read-only field with no explicit default converts into
`SkipField` — the keys are silently omitted from every response, not emitted empty. The comments and
attachments a client actually needs come from the separate top-level `attachments`/`comments` keys
`TaskDetailView.get` builds independently (`:421-430`, [above](#retrieve-update-delete)) — a field
being present in `Meta.fields` is not proof it is ever emitted; check what the model can actually
resolve. **`stage` and `kanban_order` are not returned by either `GET`, and are not in `Meta.fields`
on either serializer at all** — they exist only on `TaskKanbanCardSerializer`
(`:519-540`), which backs the Kanban-specific endpoints (`/api/tasks/kanban/`,
`/api/tasks/{id}/move/`), not the create/update endpoints this page covers.

174
docs/api/tokens-and-api-keys.md Normal file
View file

@ -0,0 +1,174 @@
# Tokens and API keys
Two credential types exist alongside the JWT session described in
[Authentication](authentication.md), for callers that need to authenticate without a browser or an
interactive sign-in: personal access tokens and the per-org API key. This page documents the CRUD
surface for both. For what each one is, how it's resolved, and how its blast radius compares to the
others, see [Architecture: Authentication](../architecture/authentication.md#personal-access-tokens)
and [Security hardening](../self-hosting/security-hardening.md#token-surfaces).
## Personal access tokens
A personal access token (`bcrm_pat_…`, `PersonalAccessToken` in `backend/common/models.py:854-907`)
authenticates as the profile that created it and inherits that profile's role and org in full. It
carries a `scopes` field, but nothing in the request-handling code enforces it — read the model's
own comment before assuming otherwise:
```python
# NOTE: scopes are stored for forward-compatibility but are NOT enforced in
# Phase 1 — a token always inherits the owning profile's full role/permissions.
# Do not treat `scopes` as a trust boundary until enforcement lands.
```
(`backend/common/models.py:871-873`.) Set `scopes` if you like for your own bookkeeping; it does not
restrict what the token can do today.
`GET /api/profile/tokens/` (`PersonalAccessTokenListCreateView`,
`backend/common/views/pat_views.py:33-47`) lists the caller's own tokens — filtered on
`profile=request.profile` as well as `org=request.profile.org`, since this table carries no RLS
policy of its own (it's looked up by `token_hash` before any org context exists, so the explicit
filter here is the only tenant boundary, not a belt over RLS's braces):
```json
{
"error": false,
"tokens": [
{
"id": "<uuid>",
"name": "laptop mcp",
"token_prefix": "bcrm_pat_abc1",
"scopes": [],
"expires_at": null,
"last_used_at": "2026-07-30T12:00:00Z",
"created_at": "2026-07-01T09:00:00Z",
"revoked_at": null
}
]
}
```
`token_hash` is never serialized (`PersonalAccessTokenListSerializer`,
`backend/common/serializer.py:1054-1067`, whose `fields` is the exhaustive list above) — only
`token_prefix` identifies a token in any listing, on this endpoint or the admin one below.
## Creating and revoking
`POST /api/profile/tokens/` (same view, `:49-69`) creates a token for the caller:
```json
{"name": "laptop mcp", "scopes": [], "expires_at": null}
```
`name` is required (non-empty, at most 255 characters); `scopes`, if present, must be a list of
strings (at most 32); `expires_at` is optional, but if present must be in the future —
`validate_expires_at` rejects a past timestamp with `"expires_at must be in the future."`
(`backend/common/serializer.py:1092-1097`). On success (`201`):
```json
{
"error": false,
"id": "<uuid>",
"name": "laptop mcp",
"token_prefix": "bcrm_pat_abc1",
"scopes": [],
"expires_at": null,
"last_used_at": null,
"created_at": "2026-08-02T10:00:00Z",
"revoked_at": null,
"token": "bcrm_pat_abc1...<the full raw token>"
}
```
The `token` field is the only time the raw value is ever returned — it is generated, hashed, and
the plaintext discarded; there is no way to retrieve it again later, only to revoke it and create a
new one. A validation failure returns `400` with `{"error": true, "errors": {...}}`, the same shape
described in [Errors](errors.md#validation-errors).
`DELETE /api/profile/tokens/{id}/` (`PersonalAccessTokenDetailView`, `:72-86`) revokes a token —
sets `revoked_at`, it does not delete the row. It is scoped to `org=request.profile.org` **and**
`profile=request.profile` in the same lookup, so another user's token id returns `404`, not `403`:
revoking is unavailable to anyone but the token's own owner, and the lookup gives no sign a
different user's token even exists at that id. Revoking an already-revoked token is a no-op that
still returns `200`.
## Organization token oversight
`GET /api/org/tokens/` (`OrgAccessTokenListView`, `backend/common/views/pat_views.py:89-152`) is a
separate, admin-only endpoint — `permission_classes = (IsAuthenticated, HasOrgContext,
IsOrgAdmin)` — that lists every personal access token in the org, not just the caller's own. It is
deliberately kept apart from `profile/tokens/` above rather than folded into it with an
admin-widened filter, so the "a user manages only their own tokens" guard on the self-service
endpoint is never the thing that has to be loosened to support oversight:
```json
{
"error": false,
"tokens": [
{
"id": "<uuid>",
"name": "laptop mcp",
"token_prefix": "bcrm_pat_abc1",
"scopes": [],
"expires_at": null,
"last_used_at": "2026-07-30T12:00:00Z",
"created_at": "2026-07-01T09:00:00Z",
"revoked_at": null,
"owner": {"id": "<uuid>", "name": "Jane Doe", "role": "USER", "is_active": false},
"is_live": true
}
],
"totals": {"count": 12, "live": 9, "orphaned": 2, "unused_90d": 3}
}
```
`owner.is_active: false` with `is_live: true` is the specific case this view exists to surface: a
token belonging to a profile that has since been deactivated. Deactivating a profile invalidates its
tokens as a side effect, not through any separate revocation step —
`resolve_valid_pat` (`backend/common/pat_auth.py:29-46`) checks `pat.profile.is_active` (and
`pat.org.is_active`) on every single use, so the token in the example above cannot actually
authenticate again unless the owner's profile is reactivated first. `"orphaned"` in `totals` counts
exactly these: still unrevoked (`is_live`), but dormant because their owner is gone, which is what an
admin doing offboarding cleanup would revoke explicitly with the endpoint below.
`DELETE /api/org/tokens/{id}/` (`OrgAccessTokenDetailView`, `:155-176`) revokes any token in the
admin's own org, filtered only on `org=request.profile.org` (no `profile=` filter, unlike the
self-service delete above) — an admin can revoke a colleague's token, deactivated or not. A pk from
another org still `404`s. Revoking an already-revoked token is a no-op `200`, same as the
self-service endpoint.
## Organization API keys
Each org has exactly one API key (`Org.api_key`), managed through `GET`/`POST /api/org/api-key/`
(`OrgApiKeyView`, `backend/common/views/organization_views.py:244-326`), admin-only in both
directions — checked by hand in `_admin_org_or_error` (`:255-275`) rather than an `IsOrgAdmin`
permission class, but the same rule: `request.profile.role == "ADMIN"` or
`request.profile.is_organization_admin`.
Unlike a personal access token, this key doesn't authenticate as any particular person — a request
bearing it in the `Token` header resolves to the org's first active `ADMIN` profile
(`backend/common/middleware/get_company.py:157-183`), so a leaked key is equivalent to a leaked
admin session for that org, not one user's session, and there's no way to scope it down. The org is
always taken from `request.profile.org`, never from anything client-supplied, so a caller can only
ever read or rotate their own org's key.
`GET /api/org/api-key/`:
```json
{"error": false, "api_key": "<the current key>"}
```
`POST /api/org/api-key/` rotates it — generates a new key and immediately overwrites the old one:
```json
{
"error": false,
"message": "API key rotated. The previous key is no longer valid.",
"api_key": "<the new key>"
}
```
The message is not just cosmetic: the old key stops authenticating the moment this call returns,
with no grace period. A non-admin caller gets `403` with
`{"error": true, "errors": "Only organization admins can access the API key"}` from either method;
a caller with no org context at all gets `400` with
`{"error": true, "errors": "Organization context required"}`.

206
docs/architecture/authentication.md Normal file
View file

@ -0,0 +1,206 @@
# Authentication
BottleCRM has three separate credential types — a short-lived JWT for interactive clients, a
personal access token for agents and scripts, and a per-org API key for server-to-server
integrations — and this page describes how each one proves identity and how the server decides
what it's allowed to do. The rule underneath all three is the same one stated in
[Multi-tenancy and RLS](multi-tenancy-and-rls.md): identity, org, and role are facts the server
derives and verifies, never values a client supplies. Nothing in a request body, query string, or
custom header changes who you are authenticated as.
## Token model
Interactive sign-in (Google OAuth and passwordless magic-link/OTP) issues a JSON Web Token pair
built on `djangorestframework-simplejwt`. `SIMPLE_JWT` in `backend/crm/settings.py` sets:
```python
SIMPLE_JWT = {
"ACCESS_TOKEN_LIFETIME": timedelta(hours=1),
"REFRESH_TOKEN_LIFETIME": timedelta(days=14),
"ROTATE_REFRESH_TOKENS": True,
"BLACKLIST_AFTER_ROTATION": True,
"ALGORITHM": "HS256",
"SIGNING_KEY": SECRET_KEY,
...
}
```
Access tokens live one hour; refresh tokens live 14 days. Both are signed with `SECRET_KEY` itself
— there is no separate JWT signing key setting anywhere in this codebase, so rotating `SECRET_KEY`
invalidates every outstanding token immediately. See
[Security hardening](../self-hosting/security-hardening.md#secrets) for the operational
consequences of that. `rest_framework_simplejwt.token_blacklist` is in `INSTALLED_APPS`
specifically so that `RefreshToken.blacklist()` exists to call — without that app installed,
`BLACKLIST_AFTER_ROTATION` would be a silent no-op, which is exactly the kind of failure mode this
codebase's own comment on the `INSTALLED_APPS` entry calls out.
Tokens are minted by `OrgAwareRefreshToken` (`backend/common/serializer.py`), a `RefreshToken`
subclass whose `for_user_and_org(user, org, profile=None)` classmethod embeds org and role claims
directly in the token payload (see [Claims](#claims)) so the frontend doesn't need a follow-up
`/api/auth/profile/` call just to learn which org or role it's operating as.
Two other credential types exist alongside the JWT, covered later on this page:
[personal access tokens](#personal-access-tokens) (`bcrm_pat_…`, self-service, scoped to one
profile) and [organization API keys](#organization-api-keys) (one per org, admin-managed,
equivalent to an admin session). See
[Security hardening](../self-hosting/security-hardening.md#token-surfaces) for how their blast
radii compare.
## Claims
`OrgAwareRefreshToken.for_user_and_org` embeds, beyond simplejwt's own default claims
(`token_type`, `exp`, `iat`, `jti`, and the user id under `USER_ID_CLAIM = "user_id"`):
- `user_email`, `user_name` (derived from the email, since `User` has no first/last name fields),
`user_profile_pic`
- `org_id`, `org_name` — only when an org is present; a token minted before the caller has picked
an org (see [Organization switching](#organization-switching)) carries neither
- `org_settings` — `default_currency`, `currency_symbol`, `default_country`
- `role` — only when a `Profile` was supplied, i.e. only once an org is known
`org_id` is the claim that matters most: it's what `GetProfileAndOrg`
(`backend/common/middleware/get_company.py`) reads to resolve `request.org`, and by extension it's
what `RequireOrgContext` uses to set the PostgreSQL session variable every RLS policy checks — see
[Overview](overview.md#request-lifecycle). Critically, the middleware does not simply trust the
claim: it re-validates it against an active `Profile` row on every single request —
```python
profile = Profile.objects.select_related("org").get(
user_id=user_id, org_id=org_id, is_active=True
)
```
— so a token minted while a user was a member of an org keeps working only as long as that
membership stays active. If the membership is revoked after the token was issued, this lookup
raises `Profile.DoesNotExist` and the middleware turns that into a 403 asking the user to log in
again, rather than trusting a claim that's gone stale. `role` in the token is a display
convenience, not the source of truth for authorization — every permission check in
[Permissions and roles](permissions-and-roles.md) reads `request.profile.role` fresh from the
database-backed `Profile` the middleware just re-validated, not from the JWT payload.
## Refresh and rotation
The old documentation for this project asserted refresh-token rotation and server-side tracking
without pointing at code. Here is what the code actually does. `POST /api/auth/refresh-token/`
(`OrgAwareTokenRefreshView`, `backend/common/views/auth_views.py`, wired at
`backend/common/urls.py:58-62`) is the only refresh endpoint — there's no separate route using
simplejwt's stock `TokenRefreshView`, so the `ROTATE_REFRESH_TOKENS`/`BLACKLIST_AFTER_ROTATION`
settings shown above describe the intent, but the actual rotation logic is hand-written in this
view rather than delegated to simplejwt's default serializer:
```python
with transaction.atomic():
token.blacklist()
new_token = OrgAwareRefreshToken.for_user_and_org(user, org, profile)
```
The presented refresh token is decoded and validated, its `org_id` claim (if any) is re-checked
against an active `Profile` exactly as in [Claims](#claims) above — a revoked membership gets a 403
here too, not a fresh token — and then, in one atomic transaction, the presented token is
blacklisted and a replacement is minted carrying the same org context. Both writes share a
transaction specifically so a failure can never leave the caller tokenless while the old token
remains usable. This is real rotation with real server-side state: `RefreshToken.blacklist()`
writes to simplejwt's own `OutstandingToken`/`BlacklistedToken` tables (from the
`token_blacklist` app), which is what a stolen refresh token being replayed after the legitimate
client has already refreshed would hit — the blacklisted token fails validation on reuse. Those
bookkeeping rows aren't kept forever; `flush_expired_refresh_tokens`
(`backend/common/tasks.py`) deletes `OutstandingToken` rows once their own `expires_at` has passed,
scheduled nightly — see [Background jobs](background-jobs.md).
An expired or already-blacklisted refresh token fails with `TokenError`, which this view turns into
a 401. A refresh token for a `User` whose account has since been deactivated (`user.is_active`)
also fails, explicitly, with a 403.
## Organization switching
A signed-in user can belong to more than one org, and a single JWT only ever carries one `org_id`.
`POST /api/auth/switch-org/` (`OrgSwitchView`) is how a client moves to a different one: it
requires only `IsAuthenticated` (not `HasOrgContext`), because it has to work for a token that has
no org claim at all yet — the state a fresh Google OAuth login is in, since
`GoogleOAuthCallbackView`/`GoogleIdTokenView` both mint their initial token with `org=None` and
leave org selection to a separate step.
The view re-validates membership the same way the refresh endpoint does —
`Profile.objects.get(user=request.user, org_id=org_id, is_active=True)` — and 403s
(with an audit-logged `permission_denied` entry) if the caller has no active profile in the
requested org. It then optionally retires the refresh token the client is switching away from:
```python
if str(token.get("user_id")) != str(request.user.id):
return
token.blacklist()
```
That ownership check is not incidental — without it, `refresh` would let any authenticated caller
blacklist *any* refresh token they could observe, turning a convenience parameter into a remote
logout for someone else's session. Retiring the old token is optional by design (existing clients
don't all send it, and failing the switch over a missing optional field would be worse than letting
one token expire on its own 14-day schedule), but when it is sent and does belong to the caller, it
stops working immediately rather than staying valid against the previous org for the rest of its
life. The org switch itself — retiring the old token and minting the replacement — happens inside
one transaction for the same reason the refresh endpoint's does.
## Personal access tokens
Personal access tokens (`bcrm_pat_…`, `PersonalAccessToken` in `backend/common/models.py`) are
for agents and scripts — the MCP server integration in particular — that need to authenticate as a
specific user without going through the JWT/OAuth flow on every call. A token is generated once
(`PersonalAccessToken.generate(profile, name, ...)`), shown to the caller exactly once, and stored
only as a SHA-256 hash (`token_hash`); the plaintext is not recoverable from the database
afterward.
Resolution happens twice per request, deliberately: `GetProfileAndOrg` resolves the PAT first (so
`request.org` — and, downstream, the RLS context — is set before `RequireOrgContext` runs), and
DRF's `PATAuthentication` class (`backend/common/pat_auth.py`) reuses that same resolved token
(stashed as `request._pat`) rather than looking it up a second time. Both share one validation
function, `resolve_valid_pat`, which checks the token isn't revoked or expired
(`PersonalAccessToken.is_valid()`) and that both the owning profile and its org are still active —
so deactivating a profile cuts off its personal access tokens immediately, with no separate
revocation step.
A token authenticates **as** its owning profile and inherits that profile's role and org in full.
`PersonalAccessToken` has a `scopes` field, but read the model's own comment before assuming it
restricts anything:
```python
# NOTE: scopes are stored for forward-compatibility but are NOT enforced in
# Phase 1 — a token always inherits the owning profile's full role/permissions.
# Do not treat `scopes` as a trust boundary until enforcement lands.
```
Treat a personal access token as exactly as powerful as the user who created it — an admin's PAT
can do everything that admin can do through the UI. Self-service management lives at
`profile/tokens/` (a user manages only their own); admin oversight of every token in the org —
including a deactivated colleague's — is a deliberately separate, admin-only endpoint at
`org/tokens/`, so the self-service guard is never the thing that has to be loosened to support
oversight. See [Security hardening](../self-hosting/security-hardening.md#token-surfaces) for that
distinction in more depth.
## Organization API keys
Each `Org` has exactly one API key (`Org.api_key`, generated at creation), managed through
`GET`/`POST /api/org/api-key/` (`OrgApiKeyView`), admin-only in both directions. Unlike a personal
access token, this key doesn't authenticate as any particular person: `GetProfileAndOrg`'s
`_process_api_key_auth` resolves a request bearing this key (in a `Token` header) to the org's
**first active `ADMIN` profile** —
```python
profile = Profile.objects.filter(
org=organization, role="ADMIN", is_active=True
).first()
```
— which makes a leaked org API key equivalent to a leaked admin session for that org, not a leaked
individual user session. There's no way to scope it down to less than full admin, so it's the
highest-privilege of the three credential types this page describes. Rotating it
(`POST /api/org/api-key/`) invalidates the previous key immediately — the response is explicit
that the old key is no longer valid. It's excluded from every nested API representation
(`OrganizationSerializer` never embeds it) and only ever served by this one endpoint.
One piece of related configuration to be aware of: `GOOGLE_REDIRECT_URI` is defined in
`backend/crm/settings.py:360` but is dead — grep the backend and it's the only place that name
appears. The actual redirect URI used in the Google OAuth code exchange
(`GoogleOAuthCallbackView.post`) comes from the client's request body on each call
(`request.data.get("redirect_uri")`), not from this setting. Don't spend time configuring
`GOOGLE_REDIRECT_URI` expecting it to do anything; see
[Google OAuth](../self-hosting/google-oauth.md) for the setting that actually matters.

119
docs/architecture/background-jobs.md Normal file
View file

@ -0,0 +1,119 @@
# Background jobs
This page is for anyone writing a new Celery task. If you're operating a deployment rather than
writing code, see [Email and Celery](../self-hosting/email-and-celery.md#rls-in-background-tasks)
instead — it covers running the worker and beat processes and has the same rule below from the
operator's side. This page exists because the rule is easy to get right by copying an existing
task and easy to get wrong by writing one from scratch.
## Topology
`backend/crm/celery.py` defines the app: `Celery("crm")`, configured from Django settings under the
`CELERY_` namespace, so `CELERY_BROKER_URL` and `CELERY_RESULT_BACKEND`
(`backend/crm/settings.py`, both defaulting to `redis://localhost:6379/0`) are what it actually
connects with — there's no separate Celery-only config file. `app.autodiscover_tasks()` finds
`tasks.py` in each app; a second call, `app.autodiscover_tasks(related_name="celery_tasks")`,
exists specifically because the `tasks` Django app's own task module is named `celery_tasks.py`
to avoid colliding with the app's own name.
Two processes run against this one app definition, and neither is required for the API itself to
answer requests:
- **`celery -A crm worker`** executes tasks queued by the API — welcome emails, magic-link
delivery, assignment notifications, and more.
- **`celery -A crm beat`** fires the schedule in `app.conf.beat_schedule` (ten entries — recurring
invoice generation, overdue-invoice checks, payment reminders, expired-estimate checks,
stale-opportunity and goal-milestone scans, case SLA-breach scanning every five minutes,
stale-timer cleanup, and nightly cleanup of read notifications and expired refresh-token
records). Nothing runs any of these unless `beat` is also running.
See [Email and Celery](../self-hosting/email-and-celery.md#running-a-worker) for the actual run
commands and what each scheduled entry does operationally.
## Writing a task
Celery workers never go through Django's HTTP middleware stack. `GetProfileAndOrg` and
`RequireOrgContext` — the two middleware classes that establish tenancy for a normal request, see
[Overview](overview.md#request-lifecycle) — simply never run for a task, so nothing sets the
PostgreSQL session variable `app.current_org` on the worker's connection for you. A fresh worker
connection has an empty context, and per RLS's fail-safe design (see
[Multi-tenancy and RLS](multi-tenancy-and-rls.md#policies-and-org_scoped_tables)) an empty context
makes every org-scoped query on a correctly-configured, non-superuser database role return zero
rows — not another org's rows, but not the rows you wanted either.
The fix is one line, and it has to be the first thing a task that touches an org-scoped table does.
`set_rls_context(org_id)`, defined in `backend/common/tasks.py`, sets the same session variable the
middleware would have:
```python
# backend/contacts/tasks.py
from celery import shared_task
from common.tasks import set_rls_context
from contacts.models import Contact
@shared_task
def send_email_to_assigned_user(recipients, contact_id, org_id):
"""Send Mail To Users When they are assigned to a contact"""
set_rls_context(org_id)
contact = Contact.objects.get(id=contact_id)
...
```
Any task queued from a view already has `org_id` available (`request.profile.org.id`) to pass along
as an argument — pass it, and call `set_rls_context` with it before the first query, every time.
A task that has to sweep every org rather than act on one record calls `set_rls_context` once per
org, inside the loop, immediately before that org's queries — for example
`cases.tasks.scan_for_breached_cases`:
```python
for org in Org.objects.filter(id__in=list(org_ids)):
set_rls_context(org.id)
try:
total += _scan_org(org)
except Exception:
logger.exception("Escalation scan failed for org=%s", org.id)
```
Two kinds of task legitimately skip this call. First, a task whose table isn't org-scoped at all —
`flush_expired_refresh_tokens` operates on simplejwt's `OutstandingToken` table, which carries no
`org_id` column and isn't in `ORG_SCOPED_TABLES`, so there's no per-org context to set in the first
place. Second, a task that already sets context per-org inside a loop, as above, doesn't need it
again outside the loop. What doesn't qualify as an exception is "the query looks like it filters by
something else" — see below.
## The RLS rule
Stated plainly, because it's the one rule this page exists to make impossible to miss: **any Celery
task that reads or writes an org-scoped table must call `set_rls_context(org_id)` before its first
query against that table.** Nothing else establishes tenancy for a worker process.
This codebase has a live example of what happens when a task skips it.
`purge_read_notifications` (`backend/common/tasks.py:381-397`) deletes read notifications older
than a cutoff:
```python
@shared_task
def purge_read_notifications(days=NOTIFICATION_PURGE_DAYS):
"""... Runs once across all orgs — RLS does not need a per-org context
here because the query targets `read_at`, which is intrinsic to the row,
not org-scoped logic."""
cutoff = timezone.now() - timedelta(days=days)
deleted, _ = Notification.objects.filter(
read_at__isnull=False, read_at__lt=cutoff
).delete()
...
```
The docstring's reasoning doesn't hold: whether a *query condition* is org-specific has nothing to
do with whether the *table* is org-scoped, and `notification` is — it's in `ORG_SCOPED_TABLES`
(`backend/common/rls/__init__.py`) and carries the same `FORCE ROW LEVEL SECURITY` policy as every
other table on that list. This task never calls `set_rls_context`, so on the non-superuser database
role this project requires, it runs with an empty `app.current_org`, and its `DELETE` matches zero
rows across every org, silently, on every scheduled run. It doesn't fail loudly and it doesn't
purge the wrong org's data — it just never purges anything, which is its own quietly bad outcome:
the table grows unbounded while the schedule entry that's supposed to prevent that keeps reporting
success. Read the code a task actually executes, not the comment sitting above it, before trusting
either one.

129
docs/architecture/data-model.md Normal file
View file

@ -0,0 +1,129 @@
# Data model
This page describes the shape every model in this codebase follows, the handful of ways core CRM
entities connect to each other, and a few conventions that are easy to assume rather than verify —
which is exactly why they're written down here with the file and line that makes each one true.
## Core entities
- **Lead** (`backend/leads/models.py`) — a prospect not yet qualified into a customer.
`LeadPipeline`/`LeadStage` give each org its own configurable Kanban columns.
- **Account** (`backend/accounts/models.py`) — a company or customer. `AccountEmail`/
`AccountEmailLog` support account-linked inbound mailboxes.
- **Contact** (`backend/contacts/models.py`) — a person, optionally linked to an `Account`.
- **Opportunity** (`backend/opportunity/models.py`) — a sales deal in progress.
`OpportunityLineItem` holds quoted products; `StageAgingConfig` is per-org "how many days is too
long in this stage" configuration; `SalesGoal` is a quota/target tracked against `Opportunity`
data.
- **Case** (`backend/cases/models.py`) — a support ticket. By far the largest cluster of supporting
models: `CaseWatcher`, `CsatSurvey`, `Solution` (knowledge base), `CasePipeline`/`CaseStage`,
`ReopenPolicy`, `EscalationPolicy`, `InboundMailbox`/`EmailMessage` (email-to-ticket),
`RoutingRule`/`RoutingRuleState`, and `TimeEntry`.
`Case.account` is a direct foreign key (not nullable through an intermediate table); Cases carry
their own SLA fields (`sla_first_response_hours`, `sla_resolution_hours`,
`first_response_at`, `resolved_at`) directly on the model.
- **Task** (`backend/tasks/models.py`) — a to-do item, with its own configurable
`TaskPipeline`/`TaskStage`, and a separate `Board`/`BoardColumn`/`BoardTask`/`BoardMember` family
for freeform Kanban boards distinct from the Task pipeline.
- **Invoice** (`backend/invoices/models.py`) — billing, alongside `Estimate`, `RecurringInvoice`,
`Product`, `Payment`, and `InvoiceTemplate` in the same app. This app also owns the anonymous
client-portal views (see [Multi-tenancy and RLS](multi-tenancy-and-rls.md#portal-tokens)).
- **Order** (`backend/orders/models.py`) — `Order`/`OrderLineItem`.
## Relationships
The lead-to-customer path is a one-time conversion, not a standing foreign key.
`leads.services.convert_lead_to_account(lead_obj, request, create_opportunity=True)`
(`backend/leads/services.py`) runs as a side effect when a lead's `status` is updated to
`"converted"` through the normal lead update views (`backend/leads/views/lead_views.py`) — there is
no dedicated "convert" endpoint. It:
1. `get_or_create`s an `Account` (case-insensitive name match within the org).
2. `get_or_create`s a `Contact` from the lead's email, if the lead has one, and links it to the
account both by direct FK and via `Account.contacts`.
3. Creates an `Opportunity` if requested and the lead carries enough data to justify one, copying
`assigned_to`/`teams`/`tags` across.
4. Reassigns the lead's `Comment`/`Attachments` rows (generic-relation `ContentType`/`object_id`
pairs) from the lead onto the new account.
5. Sets `lead_obj.status = "converted"`.
There is deliberately no persisted link back from the created `Account`/`Contact`/`Opportunity` to
the originating `Lead` — `Lead`'s own model file documents this: earlier `converted_account`/
`converted_contact`/`converted_opportunity`/`conversion_date` fields were removed because they were
"never populated, conversion just sets status." If you need to trace a record back to the lead it
came from, there's nothing in the schema to follow; only the account/contact name or email is
likely to match.
`Contact` and `Account` are linked two separate ways at once: `Contact.account` is a nullable
foreign key (`on_delete=SET_NULL`), and `Account.contacts` is an independent many-to-many. Lead
conversion populates both. Code that needs "the contacts belonging to this account" should decide
deliberately which of the two it means, rather than assuming they're always in sync.
`Opportunity.account` is a nullable FK to `Account`; `Opportunity.contacts` is a many-to-many to
`Contact`. `Case.account` is a required-in-practice (blank/null allowed, but `on_delete=CASCADE`)
FK to `Account`, and `Case.contacts` is a many-to-many to `Contact`.
## Conventions
Nearly every model in this codebase inherits `BaseModel` (`backend/common/base.py`), which via
`AuditModel` (`backend/common/mixins.py`) provides:
- `id` — a `UUIDField` primary key (`default=uuid.uuid4`), not an auto-incrementing integer.
- `created_at`/`updated_at` — set automatically (`auto_now_add`/`auto_now`).
- `created_by`/`updated_by` — foreign keys to **`User`**, not `Profile`, set from
`crum.get_current_user()` in `BaseModel.save()`. This is worth being precise about: a permission
check that compares `request.profile` directly to `some_object.created_by` is comparing a
`Profile` instance to a `User` foreign key and will always be `False` — see
[Permissions and roles](permissions-and-roles.md#object-level-checks) for the bug class that
causes and how to compare them correctly (`profile.user_id == obj.created_by_id`).
Three models opt out of this entirely and inherit plain `models.Model` instead: `User`
(`backend/common/models.py:34`), `MagicLinkToken` (`:524`), and `PortalAccessToken` (`:910`). All
three still declare their own `id = models.UUIDField(default=uuid.uuid4, primary_key=True)` by
hand, so the UUID-primary-key convention holds even for them — but none of the three has
`created_at`/`updated_by`/`created_by` fields, which makes sense for what they are: `User` predates
and sits outside the org-membership model entirely (`Profile` is what's org-scoped), and
`MagicLinkToken`/`PortalAccessToken` are short-lived, auth-bootstrap rows with no meaningful
"who last touched this" to record.
Org-scoped models add one more field: an `org` foreign key to `common.Org`, either inherited from
`BaseOrgModel` or (the more common pattern in this codebase) declared by hand on top of
`BaseModel`. See [Multi-tenancy and RLS](multi-tenancy-and-rls.md#baseorgmodel) for which models do
which and why both are valid as long as the table is also registered and migrated for RLS.
**`Opportunity.stage` is a fixed enum; `Lead`, `Case`, and `Task` stages are configurable
per-org pipelines — these are not the same kind of field even though they sound similar.**
`Opportunity.stage` is a plain `CharField` over `common.utils.STAGES` — `PROSPECTING`,
`QUALIFICATION`, `PROPOSAL`, `NEGOTIATION`, `CLOSED_WON`, `CLOSED_LOST` — the same six stages for
every organization, with no model backing a per-org column set. `Lead.stage`, `Case.stage`, and
`Task.stage`, by contrast, are nullable foreign keys to a per-org `LeadStage`/`CaseStage`/
`TaskStage` row (itself belonging to a `LeadPipeline`/`CasePipeline`/`TaskPipeline`), so each org
can define its own Kanban columns for leads, cases, and tasks. All three of those models also keep
a separate, fixed-choices `status` field (`Lead.status`, `Case.status`, `Task.status`) alongside the
configurable `stage` FK — a stage's `maps_to_status` field is what keeps the two in sync when a
record moves between stages. A deal in `Opportunity` has no equivalent second field, because it has
no configurable stage to keep in sync with anything.
Several models — `Lead`, `Opportunity`, `Case` among them — carry a `custom_fields = JSONField()`
whose values are validated against `common.CustomFieldDefinition`, a per-org schema extension
mechanism that doesn't require a migration to add a field for one org.
Finally, one validation gotcha worth knowing before you rely on `Model.clean()`: DRF's
`ModelSerializer` calls `instance.save()`, not `instance.full_clean()`, so a model's `clean()`
method is invoked through the API only if that model's own `save()` explicitly calls
`self.full_clean()` itself — most don't. `Lead.clean()`, `Opportunity.clean()`, and `Case.clean()`
(`backend/cases/models.py:180`) all define real validation rules this way, and by default those
rules are silently unenforced through the API, because none of `Lead.save()`, `Opportunity.save()`,
or `Case.save()` calls `full_clean()`. A handful of other models do call it from their own
`save()`, so their `clean()` rules reach the API as a Django
`ValidationError` — a 500 unless the view happens to catch it, not a clean DRF 400: `Task.save()`
(`backend/tasks/models.py:403`), and `Comment.save()`/`Attachments.save()`
(`backend/common/models.py:316-318` and `:430-432`, both there specifically to enforce that a
comment or attachment's `org` matches the org of whatever record it's attached to). There's no
blanket rule for which models do this — check the individual `save()` method rather than assuming.
Where an unenforced `clean()` rule turned out to matter through the API, it's been reimplemented
directly in the serializer instead of relying on `full_clean()` — `opportunity/serializer.py`'s
`validate()` duplicates the two rules from `Opportunity.clean()` for exactly this reason, per its
own comment. If you add a `clean()` method to a model, verify whether that model's `save()` calls
`full_clean()` before assuming the rule is enforced anywhere outside a unit test that calls
`full_clean()` directly.

211
docs/architecture/multi-tenancy-and-rls.md Normal file
View file

@ -0,0 +1,211 @@
# Multi-tenancy and RLS
This page is for anyone about to add an org-scoped model, or trying to understand why one query
returned no rows when it should have returned some. It explains the tenancy model from the
application-code side; [PostgreSQL and RLS](../self-hosting/postgresql-and-rls.md) covers the
same mechanism from the operator's side — creating the database role, running `manage_rls`, and
proving isolation holds against real data. Read that page too before you deploy anything; this one
is about what to do when you're writing a new model.
## The model
Every tenant's data lives in the same database, the same tables, distinguished by an `org_id`
column. Isolation is enforced twice, independently:
1. **In application code**, every queryset that touches an org-scoped table filters explicitly —
`Model.objects.filter(org=request.profile.org)` — and every `create()`/`serializer.save()`
passes `org=request.profile.org` rather than trusting a client-supplied value.
2. **In PostgreSQL**, via Row-Level Security. Each org-scoped table carries a policy that compares
its `org_id` column to the session variable `app.current_org`
(`backend/common/rls/__init__.py`), which `RequireOrgContext` sets from the caller's validated
JWT on every request — see [Overview](overview.md#request-lifecycle) for exactly when.
Neither layer is optional, and neither is a superset of the other in practice: see
[The two-layer contract](#the-two-layer-contract) below for why both have to hold.
## BaseOrgModel
`BaseOrgModel` (`backend/common/base.py`) is a `BaseModel` subclass that adds an `org` foreign key
to `common.Org`, an `OrgScopedManager` (giving you `Model.objects.for_org(org)` and
`Model.objects.for_request(request)` helpers), and a `["org", "-created_at"]` index. Its own
docstring states the intent plainly: "All tenant-owned models should inherit from this instead of
`BaseModel`."
That said, verify this against the actual model files before assuming every org-scoped model
follows it, because most don't. Searching the codebase for classes that actually inherit
`BaseOrgModel` turns up four: `Order` and `OrderLineItem` (`backend/orders/models.py`), and
`PersonalAccessToken` and `PackApplication` (`backend/common/models.py`). Every core CRM model —
`Lead`, `Account`, `Contact`, `Opportunity`, `Case`, `Task` — and most of the supporting tables
around them (`CaseWatcher`, `CsatSurvey`, `Macro`, `BusinessCalendar`, and others) instead inherit
plain `BaseModel` and declare their own `org = models.ForeignKey(Org, on_delete=models.CASCADE,
related_name="...")` field by hand, shaped identically to what `BaseOrgModel` would have given
them. This isn't drift or oversight: several of these models say so directly in their own
docstrings, citing a coordination decision to prefer a hand-declared FK over `BaseOrgModel`. For
example, `backend/cases/models.py` on `CaseWatcher`:
```python
class CaseWatcher(BaseModel):
"""A profile subscribed to updates on a case.
Per `docs/cases/COORDINATION_DECISIONS.md` D2 we inherit BaseModel and
declare our own org FK rather than using BaseOrgModel — RLS still
applies via the migration that adds the `case_watcher` table.
"""
```
(The `docs/cases/COORDINATION_DECISIONS.md` file that comment and two others like it — in
`backend/business_hours/models.py` and `backend/macros/models.py` — cite does not exist in this
checkout; treat the reasoning captured in the code comments themselves as the source of truth.)
The practical takeaway: inheriting `BaseOrgModel` is a correct and recommended way to add an `org`
field to a new model — doing so also gets you the `OrgScopedManager` helpers
(`Model.objects.for_org(org)`, `Model.objects.for_request(request)`) for free, which is itself a
reason to prefer it — and you should default to it unless you have a specific reason to match this
codebase's more common hand-rolled-FK convention instead (for consistency with a sibling model in
the same app, for instance). But inheriting it is not what makes RLS protection work,
and it is not sufficient on its own — `PersonalAccessToken` inherits `BaseOrgModel` and is
*deliberately excluded* from RLS (see [Policies and ORG_SCOPED_TABLES](#policies-and-org_scoped_tables)
below). What actually determines whether a table is protected is the next section.
## Policies and ORG_SCOPED_TABLES
`ORG_SCOPED_TABLES` in `backend/common/rls/__init__.py` is the single list that determines which
tables get RLS policies — at the time of writing it has 61 entries. `get_enable_policy_sql(table)`
generates the SQL that protects each one:
```sql
ALTER TABLE "<table>" ENABLE ROW LEVEL SECURITY;
ALTER TABLE "<table>" FORCE ROW LEVEL SECURITY;
DROP POLICY IF EXISTS org_isolation ON "<table>";
DROP POLICY IF EXISTS org_insert_check ON "<table>";
CREATE POLICY org_isolation ON "<table>"
FOR ALL
USING (org_id::text = (select NULLIF(current_setting('app.current_org', true), '')));
CREATE POLICY org_insert_check ON "<table>"
FOR INSERT
WITH CHECK (org_id::text = (select NULLIF(current_setting('app.current_org', true), '')));
```
`FORCE ROW LEVEL SECURITY` matters because without it the table owner — typically the role that ran
migrations — would be exempt from its own table's policies. `NULLIF` turns an unset (empty-string)
session variable into SQL `NULL`, and `org_id::text = NULL` is never true, so an empty context
yields zero rows rather than every org's rows: fail-safe by construction, not by convention.
Adding RLS to a new org-scoped model is exactly three steps, and skipping any one of them leaves
the table looking protected while it isn't:
1. **Give the model an `org` field.** Inherit `BaseOrgModel`, or — following this codebase's
dominant pattern, see [BaseOrgModel](#baseorgmodel) above — declare `org =
models.ForeignKey(Org, on_delete=models.CASCADE, related_name="...")` directly.
2. **Add the table's `db_table` name to `ORG_SCOPED_TABLES`** in `backend/common/rls/__init__.py`.
3. **Write a migration** that calls `get_enable_policy_sql('your_table')` inside a `RunPython`
operation, guarded so it's a no-op on SQLite (the test database has no concept of row security):
Here is a real one, `backend/opportunity/migrations/0012_enable_rls_sales_goal.py`, in full:
```python
from django.db import connection, migrations
from common.rls import get_disable_policy_sql, get_enable_policy_sql
TABLE = "sales_goal"
def enable_rls(apps, schema_editor):
if connection.vendor != "postgresql":
return
with connection.cursor() as cursor:
cursor.execute(get_enable_policy_sql(TABLE))
def disable_rls(apps, schema_editor):
if connection.vendor != "postgresql":
return
with connection.cursor() as cursor:
cursor.execute(get_disable_policy_sql(TABLE))
class Migration(migrations.Migration):
atomic = False
dependencies = [
("opportunity", "0011_opportunity_kanban_order"),
]
operations = [
migrations.RunPython(enable_rls, disable_rls),
]
```
`get_enable_policy_sql` opens with the two `ALTER TABLE` statements, then — before either
`CREATE POLICY` — drops both policies with `DROP POLICY IF EXISTS`, so re-running this kind of
migration is idempotent — useful if a table's policy needs to be recreated after a schema change.
Skip step 2 and the consequence is worse than "no protection": `manage_rls --status` only reports
on tables that are actually in `ORG_SCOPED_TABLES`, so an unregistered table doesn't show up as
"disabled" — it doesn't show up at all, and the gap is invisible to the one tool that's supposed to
catch it. The one deliberate exception in this codebase is `personal_access_token`: it inherits
`BaseOrgModel` (so it has an `org` field) but is intentionally left out of `ORG_SCOPED_TABLES`,
because it's looked up by `token_hash` before any tenant context can exist — an "auth-bootstrap"
table, in the same category as `Org` itself. Its isolation is enforced instead by explicit
`org`/`profile` filters in `backend/common/views/pat_views.py`. That exception is the proof that
inheriting `BaseOrgModel` and being RLS-protected are two separate facts, not one.
## The two-layer contract
RLS is a safety net, not a replacement for the ORM filter. Every view and serializer still writes
`Model.objects.filter(org=request.profile.org)` and passes `org=request.profile.org` explicitly on
create — that ORM filter is the contract; RLS is what catches the case where a filter is missing,
forgotten, or wrong. The two layers fail differently, which is exactly why both are required:
- If the ORM filter is missing but RLS is correctly configured (a non-superuser database role, see
[PostgreSQL and RLS](../self-hosting/postgresql-and-rls.md#why-rls)), the query still returns
only the caller's org's rows — RLS silently saved you.
- If the database role is a superuser — which is the default (`DBUSER` unset falls back to
`postgres`) unless you deliberately change it — RLS is bypassed entirely, silently, with no error
and no log line. In that configuration, the ORM filter is the *only* thing standing between a
request and another org's data. A development environment where the DB role happens to be a
superuser will look identical whether the ORM filter is present or not; only
`manage_rls --verify-user` against a real, correctly-configured role tells you which one you're
actually running.
Treat every new queryset the same way regardless of which layer you're thinking about: write the
`org=` filter as if RLS didn't exist, because on a misconfigured deployment, it doesn't.
## Portal tokens
The client portal — invoice and estimate links emailed to customers, and CSAT survey links — is the
one place in this codebase that has to read org-scoped rows with no authenticated user and no JWT
at all. That's a genuine chicken-and-egg problem for RLS: you cannot set `app.current_org` until
you know the org, and with RLS enforcing isolation, you cannot read the row that would tell you the
org until the context is already set.
`common.models.PortalAccessToken` (`backend/common/models.py`, defined starting at line 910) breaks
the cycle. It's a small, deliberately **unscoped** table — absent from `ORG_SCOPED_TABLES`, no RLS
policy — that maps `sha256(url_token)` to an `org_id`, populated at the same moment each portal
token is minted (`common/portal_tokens.py:register_portal_token`). A public view hashes the token
it was handed in the URL, looks up the org in this unscoped table, and only then sets the RLS
context and queries the actual resource:
```python
# backend/invoices/public_views.py
def _resolve_org_context(token, resource_type):
org_id = resolve_portal_org(token, resource_type)
if org_id:
set_rls_context(org_id)
return org_id
```
An unknown or malformed token leaves the RLS context empty; the subsequent scoped query then
returns nothing under RLS's fail-safe default, and the caller gets the same 404 a disabled link
would produce — a stranger probing token values learns nothing about whether one is real. This is
the same "resolve org first via an unscoped, narrow lookup, then call `set_rls_context`" pattern
Celery tasks use (see [Background jobs](background-jobs.md)); it adds a dedicated, narrow surface
rather than weakening the isolation policy on `invoice`, `estimate`, or `csat_survey` themselves.
`backend/docs/PORTAL_RLS.md` documents the reasoning and the alternatives that were deliberately
rejected — notably, granting the anonymous path `BYPASSRLS` — but its "what is still open" section
describes the state *before* `PortalAccessToken` existed; the resolution above is what actually
runs today.

120
docs/architecture/overview.md Normal file
View file

@ -0,0 +1,120 @@
# Architecture overview
This section is for developers extending or auditing BottleCRM, not for operators deploying it —
see Self-hosting for that. It explains how the backend is put together: what runs before a view
ever sees a request, what each Django app owns, and where to go for the detail on tenancy,
authentication, permissions, the data model, and background jobs.
## The three clients
A SvelteKit web app (`frontend/`), a Flutter mobile app (`mobile/`), and any third-party
integration all talk to the same Django REST Framework backend (`backend/`) over the same JSON
API — there is no separate mobile API or web-only endpoint set. `crm/urls.py` mounts the entire
API under `/api/`, split into `common.app_urls` (auth, org/profile management, teams, tags,
notifications, documents, custom fields, and one route each into `accounts`, `contacts`, `leads`,
`opportunity`, `tasks`, `cases`, `invoices`, boards, business hours, and macros) plus
`invoices.public_urls` for the anonymous client-portal endpoints. Because every client shares one
API, an authorization rule enforced in a view protects all three clients at once — and,
conversely, a rule enforced only in the SvelteKit UI protects none of them, since a script or a
modified mobile build reaches the API directly.
## Request lifecycle
Tenancy is established in middleware, before any view code runs, in a fixed order defined in
`MIDDLEWARE` in `backend/crm/settings.py`:
```python
MIDDLEWARE = [
"django.middleware.security.SecurityMiddleware",
"whitenoise.middleware.WhiteNoiseMiddleware",
"django.middleware.common.CommonMiddleware",
"django.contrib.sessions.middleware.SessionMiddleware",
"django.middleware.csrf.CsrfViewMiddleware",
"django.contrib.auth.middleware.AuthenticationMiddleware",
"django.contrib.messages.middleware.MessageMiddleware",
"django.middleware.clickjacking.XFrameOptionsMiddleware",
"corsheaders.middleware.CorsMiddleware",
"crum.CurrentRequestUserMiddleware",
"common.middleware.get_company.GetProfileAndOrg",
"common.middleware.rls_context.RequireOrgContext",
]
```
The two that matter for multi-tenancy run last, in this order:
1. **`GetProfileAndOrg`** (`backend/common/middleware/get_company.py`) resolves who the caller is
and which org they're acting in. For a request bearing a personal access token it resolves the
token first — deliberately before the JWT branch, so a PAT is never handed to the JWT decoder.
For a JWT-bearing request it decodes the access token, reads the `org_id` claim out of the
signed payload (never out of a header or body — see
[Authentication](authentication.md#claims)), and looks up an active `Profile` for that
`user_id`/`org_id` pair; only if that profile exists does it set `request.profile` and
`request.org`. For an org-API-key request it resolves the key to the org's first active
`ADMIN` profile. In every path, the org on the request is a fact the server derived and
verified — never a value taken as-is from the client.
2. **`RequireOrgContext`** (`backend/common/middleware/rls_context.py`) runs next and does two
things: it rejects any request to a non-exempt path with a 403 if `request.org` was never set
by the step above, and — this is the step that actually enables tenant isolation — it sets the
PostgreSQL session variable `app.current_org` to that org's id with
`SELECT set_config('app.current_org', %s, false)`, and resets it to an empty string after the
response. Every Row-Level Security policy in the schema keys off this one session variable; see
[Multi-tenancy and RLS](multi-tenancy-and-rls.md) for how.
So the order is: identity and org are established and validated from the JWT first, and only once
that has succeeded is the database told which org's rows the rest of this request may see. A view
never runs before both steps have completed — `RequireOrgContext` sits after `GetProfileAndOrg` in
the list above and returns a response itself (the 403) rather than calling `get_response` when the
context is missing, so a request that fails tenancy resolution never reaches URL dispatch.
This file also defines an older class, `SetOrgContext`, whose own docstring describes it as the
one wired into `MIDDLEWARE` — that docstring is stale. `SetOrgContext` sets the session variable
but never rejects a context-less request; `RequireOrgContext` is what `settings.py` actually
lists, and it is the one enforced.
One layer above middleware, DRF's own authentication runs at view dispatch time
(`DEFAULT_AUTHENTICATION_CLASSES` in `REST_FRAMEWORK`: `PATAuthentication`, then simplejwt's
`JWTAuthentication`, then `APIKeyAuthentication`) to populate `request.user` for `IsAuthenticated`
and similar permission classes. This is a second, independent pass over the same credential —
`PATAuthentication` reuses the PAT the middleware already resolved (stashed as `request._pat`) to
avoid a second lookup, but a JWT is decoded once by the middleware for org context and again by
`JWTAuthentication` for `request.user`. Every view still needs `permission_classes` set correctly;
authentication alone only establishes identity, not what that identity may do — see
[Permissions and roles](permissions-and-roles.md).
## Backend layout
```
backend/
├── crm/ # Project config: settings.py, urls.py, celery.py
├── common/ # Cross-cutting: User/Org/Profile, middleware, RLS config, permissions,
│ # auth views, personal access tokens, notifications, tags, teams, custom
│ # fields, documents, packs, security audit log
├── accounts/ # Account (company/customer) records
├── contacts/ # Contact (person) records
├── leads/ # Leads, lead pipelines/stages, lead-to-account conversion
├── opportunity/ # Sales pipeline: Opportunity, line items, stage aging, sales goals
├── cases/ # Support tickets: Case, SLAs, CSAT, escalation/routing, inbound email
├── tasks/ # Task management and Kanban boards (Board/BoardColumn/BoardTask)
├── invoices/ # Billing: invoices, estimates, recurring invoices, products, payments,
│ # and the anonymous client-portal views (invoices.public_urls)
├── orders/ # Sales orders and order line items
├── business_hours/ # Per-org working-hours calendars that SLA timers honor
└── macros/ # Canned responses / saved replies
```
`common` is the one app every other app depends on for tenancy itself: it owns the
`User`/`Org`/`Profile` models, the RLS configuration (`common/rls/`), the permission classes
(`common/permissions.py`), and every authentication path (`common/views/auth_views.py`,
`common/pat_auth.py`, `common/external_auth.py`). Every domain app's *models* build on `common`
(`BaseModel`/`BaseOrgModel` from `common.base`, `Org`/`Profile` from `common.models`) and never the
other way around. That one-directional rule doesn't extend to `common`'s own *views*, though:
several of them are cross-cutting by nature and import domain-app models directly — the dashboard
(`common/views/dashboard_views.py`), global search (`common/search_views.py`), tag usage counts
(`common/views/tags_views.py`), and vertical-pack application (`common/packs/applier.py`) all pull
in `Lead`, `Opportunity`, `Case`, and others, because aggregating across every domain app is
exactly what those views are for. Only the model layer is strictly one-directional.
The `teams` Django app referenced in some older documentation and comments no longer exists as a
separate app; `Teams` is a model in `common/models.py`, and `INSTALLED_APPS` in
`backend/crm/settings.py` has a commented-out `# "teams", # Merged into common app` line marking
where it used to be.

188
docs/architecture/permissions-and-roles.md Normal file
View file

@ -0,0 +1,188 @@
# Permissions and roles
Authentication (covered in [Authentication](authentication.md)) establishes who is making a
request. This page is about the separate question every view still has to answer for itself: given
that identity, is this specific action on this specific object allowed? DRF's permission classes
answer the first, coarse half of that; object-level checks inside the view answer the rest.
## Roles
`Profile.role` (`backend/common/models.py`) is a two-value `CharField`:
`common.utils.ROLES = (("ADMIN", "ADMIN"), ("USER", "USER"))`, defaulting to `USER`. That's the
role a person holds *within one org* — a user with profiles in two orgs can be `ADMIN` in one and
`USER` in the other, because a role lives on the `Profile` (the user-org membership row), not on
`User` itself.
A second, independent flag exists alongside it: `Profile.is_organization_admin`, a plain boolean,
exposed as the `.is_admin` property. It is genuinely a separate field from `role` — nothing
enforces that the two agree — and a fair amount of permission-checking code treats either one as
sufficient for "this person is an admin":
```python
# common/permissions.py — IsOrgAdmin
return request.profile.role == "ADMIN" or request.profile.is_organization_admin
```
```python
# cases/access.py
def is_org_admin(profile):
"""True for the two ways this codebase spells "admin"."""
return profile.role == "ADMIN" or bool(getattr(profile, "is_admin", False))
```
When you write a new admin check, check both, the way these two do — a check that only tests
`role == "ADMIN"` will silently deny someone whose admin status was granted only through
`is_organization_admin`, and vice versa.
Separately from any org role, `IsSuperAdmin` (`backend/common/permissions.py`) gates
platform-level access on `User.is_superuser` — an explicit flag set with `manage.py
createsuperuser`, the Django admin, or another audited path, never inferred from an email address
or domain. The class's own docstring explains why that matters: deriving super-admin status from
an email domain would hand platform-wide access — every org, every user — to anyone who can obtain
an account at that domain, turning an ordinary signup into vertical privilege escalation. This flag
is unrelated to any `Profile.role`; a `User` can be `is_superuser` with no org membership at all.
## Permission classes
`backend/common/permissions.py` defines the building blocks every view composes:
- **`HasOrgContext`** — requires `request.profile` and `request.org` to both be set (by the
middleware described in [Overview](overview.md#request-lifecycle)) and `request.profile.is_active`
to be true. This is the check that turns "the JWT didn't carry a valid org claim" into a clean
403 with the message "Organization context is required. Please login again." at the view layer,
mirroring what `RequireOrgContext` already enforces at the middleware layer.
- **`IsOrgAdmin`** — `role == "ADMIN"` or `is_organization_admin`, as shown above.
- **`IsSuperAdmin`** — `user.is_superuser`, as shown above.
The standard combination on nearly every org-scoped view in this codebase is
```python
permission_classes = (IsAuthenticated, HasOrgContext)
```
— for example `CaseListView`, `CaseAttachmentView`, and `CaseCommentView` in
`backend/cases/views.py`, or `TagsListView` in `backend/common/views/tags_views.py`. Notice what
that combination does *not* do: it doesn't check role, and it doesn't check which specific object
the request is about. `IsAuthenticated` + `HasOrgContext` gets you "a real, active member of this
org is making this request" and nothing more specific than that. Role checks
(`request.profile.role == "ADMIN"`) and object-level checks are layered on inside the view methods
themselves — DRF's declarative `permission_classes` list doesn't see the object being acted on
until the view has already fetched it, so per-object authorization has to live in code, which is
what the next section covers.
## Object-level checks
`backend/cases/access.py` is the clearest example in the codebase of object-level authorization
done deliberately, in one place, rather than re-implemented per view. Its own module docstring
explains the problem it fixes: before this module existed, "who may read/write/delete a case" was
answered independently in five different call sites, and had drifted into three different answers
by accident — a watcher could see their case in the list but got a 403 opening it, because the list
view's queryset included watchers and the detail view's permission check didn't. That's the
consequence of not centralizing this kind of check: verbs disagree with each other silently, and
the person who notices is a user hitting a wall the product didn't intend.
The module now states three explicit, different rules and enforces them from one place:
```python
def is_org_admin(profile):
return profile.role == "ADMIN" or bool(getattr(profile, "is_admin", False))
def has_case_write_access(profile, case):
if is_org_admin(profile):
return True
if profile.user_id == case.created_by_id:
return True
return profile.id in {p.id for p in case.assigned_to.all()}
def assert_case_delete_access(profile, case):
"""Narrower than writing on purpose: an assignee is somebody the work was
handed to, which is a reason to let them work the ticket and not a reason
to let them erase it."""
if is_org_admin(profile):
return
if profile.user_id == case.created_by_id:
return
raise PermissionDenied(_DENIED)
```
Two things worth noticing in `has_case_write_access`: the comparison is `profile.user_id ==
case.created_by_id` — a `User` id compared to a `User` id — not `profile == case.created_by`.
`Case.created_by` (from `common/mixins.py`'s `AuditModel`, which nearly every model inherits
transitively through `BaseModel` — see [Data model](data-model.md#conventions) for the few
exceptions) is a foreign key to `User`, not to `Profile`. Comparing a `Profile` instance directly to
a `User` foreign key is always `False` — a comparison that type-checks in Python but can never be
true, which quietly turns an intended "creator or admin" check into an "admin only" check with no
error anywhere.
`backend/cases/views.py`'s own `CaseAttachmentView.delete` docstring documents two separate defects
having been live in that endpoint before the fix — an unscoped `objects.get(pk=pk)` lookup (no org
filter at all), and this same `created_by` type mismatch — and its closing line says only that "the
same one-line **lookup** bug is still open in `leads`, `tasks` and `opportunity`." That sentence is
about the missing-org-filter defect specifically, not the `created_by` comparison, and — checked
against the current source of all three apps rather than taken on the docstring's word — it's now
stale for one of the three:
- **`leads`** (`leads/views/lead_interactions.py`) — the unscoped lookup is still open (`.objects
.get(pk=pk)` at line 204, no `org=`). The `created_by` comparison here is correct, though:
`request.profile.user == self.object.created_by` (line 208) compares a `User` to a `User`, not a
`Profile` to a `User`, so this endpoint's `created_by` check works.
- **`tasks`** (`tasks/views/task_views.py`) — both defects are fixed: the lookup is
org-filtered (`.filter(pk=pk, org=request.profile.org)`, lines 975-977) and the comparison is
`request.profile.user_id == self.object.created_by_id` (line 987), with a comment at lines 982-984
explaining why. The cases docstring's claim that the lookup bug is "still open" in `tasks` is
stale.
- **`opportunity`** (`opportunity/views/opportunity_interactions.py` and `kanban_views.py`) — both
defects are open, and the type mismatch exists at two separate call sites: `request.profile ==
self.object.created_by` in the attachment delete (`opportunity_interactions.py:158` for the
unscoped lookup, `:162` for the comparison) and again as `is_owner = request.profile ==
opportunity.created_by` in the Kanban move endpoint (`kanban_views.py:162`).
So as of this writing: the `created_by` type-mismatch bug itself is open in `opportunity` only (two
call sites); the separate unscoped-lookup bug is open in `leads` and `opportunity`. Neither defect
is something this documentation task fixes — it's flagged here, precisely, because a docstring that
lists which apps still have a bug goes stale exactly like any other comment does, and the fix in
this case is checking the current source yourself, the way this paragraph did, rather than trusting
either the docstring or a prior summary of it. When you write a `created_by` comparison, compare
`profile.user_id` to `obj.created_by_id`, and write a test that proves the check can return both
`True` and `False`.
## Read wide, write narrow
A pattern recurs across this codebase enough to name: for a given kind of resource, who may *read*
it is broader than who may *change* it, which in turn is broader than who may *destroy* it. Two
concrete shapes of it:
**Per-object access**, as in `cases/access.py` above:
```
read admin · creator · assignee · watcher
write admin · creator · assignee
delete admin · creator
```
Watching a case is subscribing to updates, not being handed the keys to it — so the watcher clause
that widens *read* access is deliberately not carried into *write* or *delete*. Each verb states its
own rule; none is inferred from another.
**Shared, org-wide configuration** — tags, vertical packs, and similar org-level settings — follows
a simpler but equally deliberate version of the same shape: any active member of the org may read
it, only an admin may change it. `TagsListView` (`backend/common/views/tags_views.py`) is a clean
example: `get` has no additional check beyond the standard `(IsAuthenticated, HasOrgContext)`, but
`post` gates explicitly:
```python
def post(self, request, *args, **kwargs):
"""Create a new tag (admin only)."""
if request.profile.role != "ADMIN" and not request.user.is_superuser:
return Response(
{"error": True, "errors": "Only admins can create tags"},
status=status.HTTP_403_FORBIDDEN,
)
```
`common/views/pack_views.py` states the same shape as a doc comment before a single line of code:
"any authenticated member may list packs; only an org ADMIN may apply one or clear sample data."
When you're adding a new endpoint over something that's genuinely shared, org-wide configuration
rather than a record with a specific owner, this is the default to reach for: don't gate the read
at all beyond org membership, and gate every write and destroy behind an explicit admin check in
the view — not a `permission_classes` entry that would also block the read.

256
docs/contributing/adding-an-org-scoped-model.md Normal file
View file

@ -0,0 +1,256 @@
# Adding an org-scoped model
This page walks through adding a new tenant-owned model end to end, using a worked example — a
`CallLog` model recording calls made against a `Lead` — so every code block is something you could
actually drop into the tree, not pseudocode. It assumes you've read
[Multi-tenancy and RLS](../architecture/multi-tenancy-and-rls.md), which explains *why* each of
these steps exists; this page is the checklist for *doing* it.
## The checklist
1. **Inherit `BaseOrgModel`** (`common.base`) so the model gets an `org` foreign key, the UUID
primary key, audit fields, and the `OrgScopedManager` helpers for free.
2. **Add the table's `db_table` name to `ORG_SCOPED_TABLES`** in `backend/common/rls/__init__.py`.
3. **Write a migration** that calls `get_enable_policy_sql('your_table')` inside a `RunPython`
operation, with a `dependencies` entry that guarantees the table already exists when it runs.
4. **Filter every queryset** that touches the model by `org=request.profile.org`.
5. **Pass `org=...` explicitly** on every `create()` / `serializer.save()` — never trust a
client-supplied org.
6. **Write a test for both the allowed and the denied path** — an in-org request that succeeds, and
a cross-org request that doesn't see or can't write the row.
Steps 2 and 3 are both required and neither substitutes for the other: registering the table
without a migration that actually calls `get_enable_policy_sql` leaves the table with no policies at
all, and a migration that calls `get_enable_policy_sql` without registering the table in
`ORG_SCOPED_TABLES` means `manage_rls --status` can never report on it, so a gap there is invisible
to the one tool meant to catch it. See
[Common mistakes](../self-hosting/postgresql-and-rls.md#common-mistakes) for both failure modes
described in more detail.
## The model
```python
# backend/leads/models.py
from django.db import models
from common.base import BaseOrgModel
class CallLog(BaseOrgModel):
"""A record of an outbound or inbound call made against a lead."""
DIRECTION_CHOICES = (
("OUTBOUND", "Outbound"),
("INBOUND", "Inbound"),
)
lead = models.ForeignKey(Lead, on_delete=models.CASCADE, related_name="call_logs")
direction = models.CharField(max_length=10, choices=DIRECTION_CHOICES)
notes = models.TextField(blank=True, default="")
duration_seconds = models.PositiveIntegerField(default=0)
class Meta:
verbose_name = "Call Log"
verbose_name_plural = "Call Logs"
db_table = "call_log"
ordering = ("-created_at",)
indexes = [models.Index(fields=["org", "-created_at"])]
```
`BaseOrgModel` is abstract, and — this is easy to get wrong, and this codebase has a live example of
getting it wrong — a subclass's own `Meta` class does **not** automatically inherit the abstract
parent's `indexes` just because the model inherits `BaseOrgModel`. `Order` (`backend/orders/models.py`,
one of the few models here that actually inherits `BaseOrgModel` rather than hand-declaring its own
`org` foreign key) declares its own `Meta` with `db_table`, `verbose_name`, and `ordering`, but no
`indexes` — so `Order._meta.indexes` is `[]` at runtime, not the `["org", "-created_at"]` index
`BaseOrgModel.Meta` defines. `backend/orders/migrations/0003_remove_order_orders_org_id_9026a7_idx_and_more.py`
is the receipt: it drops that exact index from both `order` and `order_line_item`, tables that
otherwise get every other query filtered by `org`. `PersonalAccessToken` (`backend/common/models.py`)
is the codebase's real example of doing this correctly — its own `Meta` explicitly redeclares
`indexes = [models.Index(fields=["org", "-created_at"])]` rather than assuming it's inherited, which
is exactly what the `CallLog` example above copies. `BaseOrgModel` reliably gives you `id` (UUID
primary key), `org`, `created_at`, `updated_at`, `created_by`, and `updated_by` regardless of whether
your model declares its own `Meta` — those are fields and a `save()` override on the class itself, not
something `Meta` mediates. The `["org", "-created_at"]` index is the one thing on `BaseOrgModel` that
your own `Meta` can silently drop, so redeclare it (as above) any time your model needs its own
`Meta` for a `db_table`, `ordering`, or anything else.
## Registering the table
Add the table's `db_table` name — `call_log`, matching `Meta.db_table` above — to
`ORG_SCOPED_TABLES` in `backend/common/rls/__init__.py`. This list currently has 61 entries (see the
file itself for the current count — it changes as the project grows, so don't rely on a number
copied from documentation); a new entry looks like any other:
```python
# backend/common/rls/__init__.py
ORG_SCOPED_TABLES = [
...
"task",
"invoice",
"call_log", # CallLog — call records against a lead
# Supporting entities
"comment",
...
]
```
## The RLS migration
Write a dedicated migration for the new table's policies, modeled on a real one already in the tree,
`backend/opportunity/migrations/0012_enable_rls_sales_goal.py`:
```python
# backend/leads/migrations/0017_enable_rls_call_log.py
from django.db import connection, migrations
from common.rls import get_disable_policy_sql, get_enable_policy_sql
TABLE = "call_log"
def enable_rls(apps, schema_editor):
if connection.vendor != "postgresql":
return
with connection.cursor() as cursor:
cursor.execute(get_enable_policy_sql(TABLE))
def disable_rls(apps, schema_editor):
if connection.vendor != "postgresql":
return
with connection.cursor() as cursor:
cursor.execute(get_disable_policy_sql(TABLE))
class Migration(migrations.Migration):
atomic = False
dependencies = [
("leads", "0016_call_log"), # the migration that actually creates call_log
]
operations = [
migrations.RunPython(enable_rls, disable_rls),
]
```
`atomic = False` matches every RLS migration in this codebase — `DROP`/`CREATE POLICY` each take an
`ACCESS EXCLUSIVE` lock, so committing per-statement instead of holding one long transaction keeps
each lock brief. `get_enable_policy_sql` itself opens with `DROP POLICY IF EXISTS` before creating
either policy, so the migration is idempotent if it's ever re-run.
!!! warning "The dependency is not decorative — get it wrong and the migration silently does nothing"
`dependencies = [("leads", "0016_call_log")]` above is what guarantees Django applies this
migration *after* the one that runs `CreateModel` for `CallLog`. If that dependency were missing
or pointed at an earlier migration, Django would be free to apply this one first — and
`ALTER TABLE "call_log" ENABLE ROW LEVEL SECURITY` against a table that doesn't exist yet would
raise a hard error, not fail silently, *unless* the migration is written defensively with an
existence check first. `backend/common/migrations/0034_rls_pipeline_tables.py` is exactly that
defensive style — it checks `get_check_table_exists_sql()` before stamping each table, because
it registers six tables across three different apps in one migration and couldn't express a
single clean dependency chain for all of them. Its own comment explains the cost: get the
dependencies wrong there and the check silently prints `Skipping <table> (table does not exist)`
and moves on — the migration "succeeds," CI is green, and the table simply never gets a policy.
For a single new model in one app, prefer the plain `opportunity/0012` shape above (a hard
dependency, no existence check, a real error if the ordering is ever wrong) over copying
`0034`'s defensive pattern — a loud failure during development is much cheaper than a policy
that quietly never gets created.
## The view
Every view touching the new model needs both the standard permission classes and an explicit org
filter — RLS is the safety net, not the contract (see
[The two-layer contract](../architecture/multi-tenancy-and-rls.md#the-two-layer-contract)):
```python
# backend/leads/views/call_log_views.py
from django.shortcuts import get_object_or_404
from rest_framework import status
from rest_framework.permissions import IsAuthenticated
from rest_framework.response import Response
from rest_framework.views import APIView
from common.permissions import HasOrgContext
from leads.models import Lead, CallLog
from leads.serializer import CallLogSerializer
class CallLogListView(APIView):
permission_classes = (IsAuthenticated, HasOrgContext)
def get(self, request, lead_id, format=None):
lead = get_object_or_404(Lead, id=lead_id, org=request.profile.org)
logs = CallLog.objects.filter(org=request.profile.org, lead=lead)
return Response(CallLogSerializer(logs, many=True).data)
def post(self, request, lead_id, format=None):
lead = get_object_or_404(Lead, id=lead_id, org=request.profile.org)
serializer = CallLogSerializer(data=request.data)
serializer.is_valid(raise_exception=True)
serializer.save(org=request.profile.org, lead=lead, created_by=request.user)
return Response(serializer.data, status=status.HTTP_201_CREATED)
```
Two things to notice: the `Lead` lookup itself is org-filtered (`get_object_or_404(Lead, id=lead_id,
org=request.profile.org)`), so a cross-org caller gets a 404 on the parent object before ever
reaching the `CallLog` query — and `org` is passed to `serializer.save()` explicitly rather than
accepted from `request.data`, the same [server-owned field](security-rules.md#server-owned-fields)
rule that applies everywhere else.
## The tests
Write both directions, using the shared fixtures in `backend/conftest.py` (`org_a`, `org_b`,
`admin_client`, `org_b_client`, and the rest) — modeled on the real pair in
`opportunity/tests/test_sales_goals.py`. `org_a`, `org_b`, `admin_client`, and `org_b_client` are
real, shared fixtures already defined in `backend/conftest.py`; `lead_in_org_a` below is not — it's a
local, file-scoped fixture this example defines itself, the same way `leads/tests/
test_lead_access_and_totals.py` has its own local `_lead()` helper rather than a shared one (there is
no org-scoped `Lead` fixture shared across the suite, because what fields a test needs on its `Lead`
varies too much to make one worth sharing):
```python
import pytest
from leads.models import CallLog, Lead
@pytest.fixture
def lead_in_org_a(org_a):
return Lead.objects.create(
first_name="Ada",
last_name="Lovelace",
email="ada@example.com",
status="in process",
company_name="Analytical Engines",
org=org_a,
)
@pytest.mark.django_db
def test_admin_can_create_call_log(admin_client, org_a, lead_in_org_a):
response = admin_client.post(
f"/api/leads/{lead_in_org_a.id}/call-logs/",
{"direction": "OUTBOUND", "notes": "Left voicemail"},
format="json",
)
assert response.status_code == 201
assert CallLog.objects.filter(lead=lead_in_org_a, org=org_a).exists()
@pytest.mark.django_db
def test_org_b_cannot_read_org_a_call_logs(org_b_client, org_a, lead_in_org_a):
"""The denied path — mirrors test_org_isolation in test_sales_goals.py."""
CallLog.objects.create(lead=lead_in_org_a, org=org_a, direction="OUTBOUND")
response = org_b_client.get(f"/api/leads/{lead_in_org_a.id}/call-logs/")
# The parent Lead lookup is org-scoped, so a foreign org gets a 404 on the
# lead itself, before the CallLog query ever runs.
assert response.status_code == 404
```
The second test is the one that actually earns its keep: a test suite that only ever checks the
allowed path can pass at 100% coverage while the org filter above is missing entirely — see
[Testing](testing.md#the-sqlite-caveat) for why this particular pair of tests is also the only thing
proving the ORM filter is correct, independent of whether RLS is even enforced in the environment the
test runs in.

91
docs/contributing/code-style.md Normal file
View file

@ -0,0 +1,91 @@
# Code style
## Python
`backend/ruff.toml` exists and is a real, checked-in configuration:
```toml
indent-width = 4
[format]
indent-style = "space"
line-ending = "lf"
quote-style = "double"
[lint]
select = ["E", "F", "I"]
ignore = ["E501"]
```
That selects pycodestyle errors (`E`), pyflakes (`F`), and import sorting (`I`), and explicitly
ignores line-length (`E501`) — so this project has decided not to enforce a line-length limit, not
that it forgot to.
**But read this before running anything:** as of this writing, none of `black`, `isort`, or `ruff`
is declared as a dependency anywhere in `backend/pyproject.toml` — not in `[project.dependencies]`,
not in the `dev` or `docs` groups under `[dependency-groups]`, not in the `mcp` extra, and not
pulled in transitively (`backend/uv.lock` has no entry for any of the three). `uv sync` on a fresh
clone will not install any of them, and `uv run black .` fails on a fresh clone with `error: Failed
to spawn: 'black' / Caused by: No such file or directory`. The same is true of `isort` and `ruff`.
CI does not run any of them either:
`.github/workflows/tests.yml`'s `backend-tests` job runs a migration check and `pytest`, and nothing
else — there is no lint or format step for Python anywhere in this repository's CI. In practice,
Python style in this project is currently enforced by human review, not tooling.
Two things follow from that:
1. **Don't `uv add black isort ruff` to make the documented commands work.** Adding a dependency
changes `pyproject.toml` and re-locks `uv.lock` for every future `uv sync` — including CI's — so
it's a decision to make deliberately, in its own PR, not something to smuggle in while fixing an
unrelated bug. See [Development setup](development-setup.md#backend) for what `uv add` actually
does.
2. **The codebase does not currently conform to what these tools would enforce**, so even running
them read-only produces a large, unrelated diff: `uvx ruff check .` — `uvx` runs a tool from
PyPI in an ephemeral environment without touching this project's lockfile — reports on the order
of 150+ findings across the tree today, and `uvx black --check --diff .` reports on the order of
170 files that would be reformatted. Neither number is a defect to fix in passing; it reflects
that these tools have not been run consistently against this codebase's history. If you want to
check your own changes without installing anything project-wide, scope `uvx` to the files you
touched (`uvx ruff check path/to/your_file.py`, `uvx black --check path/to/your_file.py`) rather
than the whole tree — running `black .` or `ruff check --fix .` across everything will reformat
files you didn't mean to touch and turn a focused PR into an unreviewable one.
Match the surrounding file's conventions by eye — 4-space indentation, double-quoted strings,
Django/DRF import grouping (standard library, then third-party, then local `common`/app imports) —
the way `ruff.toml`'s settings describe, even without the tool enforcing it for you.
## Frontend
`frontend/package.json` defines the real commands:
```bash
pnpm run lint # prettier --check . && eslint .
pnpm run format # prettier --write .
pnpm run check # svelte-kit sync && svelte-check --tsconfig ./jsconfig.json
```
Unlike the backend, these tools are installed (`prettier`, `eslint`, `svelte-check`, and their
plugins are all in `frontend/package.json`'s `devDependencies`) and CI runs all three, but not
symmetrically: `.github/workflows/tests.yml`'s `frontend-checks` job runs the Prettier check with a
`|| echo "::warning::..."` fallback, so a formatting difference shows up as a CI **warning**, not a
failed build. The ESLint step (`pnpm eslint .`) has no such fallback — a lint error fails the job.
`pnpm check` (svelte-check) and `pnpm build` run unconditionally after that and also fail the job on
error. So in practice: fix ESLint errors and type errors before opening a PR, but don't be surprised
if Prettier differences are flagged as a warning rather than a hard failure — run `pnpm run format`
yourself rather than relying on CI to catch it.
## Flutter
```bash
cd mobile
dart format .
flutter analyze --no-fatal-infos
```
`mobile/analysis_options.yaml` includes `package:flutter_lints/flutter.yaml` with no
project-specific rules added or removed. Neither of this repository's two GitHub Actions workflows
(`tests.yml`, `codeql-analysis.yml`) mentions Flutter, Dart, or `mobile/` at all, so — like the
backend's formatters — nothing in CI enforces Dart formatting or lint cleanliness; running these
commands yourself before opening a PR is the only check that happens. See
[Build and configure](../mobile/build-and-configure.md#tests-and-analysis) for the full command set
including `flutter test`.

123
docs/contributing/development-setup.md Normal file
View file

@ -0,0 +1,123 @@
# Development setup
This page gets a clone of the three BottleCRM projects — `backend/`, `frontend/`, `mobile/` — running
locally for development. For exact required versions (Python, PostgreSQL, Node, pnpm) see
[Requirements](../self-hosting/requirements.md); this page assumes you already have them installed.
For a from-scratch install walkthrough (including creating the database and `.env`), see
[Manual setup](../getting-started/manual-setup.md) — this page focuses on the parts specific to
contributing: running the test suite, adding dependencies, and running the mobile app alongside the
other two.
## Backend
The backend is managed with [`uv`](https://docs.astral.sh/uv/), not pip or a manually-activated
virtualenv. `uv sync` reads `backend/pyproject.toml` and `backend/uv.lock` and creates
`backend/.venv/` with the exact locked dependency versions; every command after that runs through
`uv run <cmd>`, which resolves that venv automatically without you activating it.
```bash
cd backend
uv sync
cp .env.example .env
uv run python manage.py migrate
uv run python manage.py runserver
```
Edit `.env` first if your local PostgreSQL role, database name, or Redis URL differ from the
defaults — see [Manual setup](../getting-started/manual-setup.md) and
[Environment variables](../self-hosting/environment-variables.md).
Run the test suite from `backend/` — see [Testing](testing.md) for the fast-iteration flags and
what the markers mean:
```bash
uv run pytest
```
To add a dependency, use `uv add`, not a manual edit to `pyproject.toml` — `uv add` resolves the
version, writes it to `pyproject.toml`, and re-locks `uv.lock` in one step, so the two files never
drift out of sync:
```bash
uv add <package> # runtime dependency, added to [project.dependencies]
uv add --group dev <package> # dev-only dependency (tests, tooling), added to [dependency-groups]
```
To refresh every locked version against the current `pyproject.toml` constraints without adding or
removing a dependency — for example, to pick up a transitive security fix — use `uv lock --upgrade`
instead; it re-resolves and rewrites `uv.lock` in place.
`backend/pyproject.toml` currently declares two dependency groups — `dev` (`pytest`,
`pytest-django`, `pytest-cov`) and `docs` (`mkdocs-material`) — plus one optional extra, `mcp`, for
the hosted MCP server (`uv sync --extra mcp`, not installed by default). `dev` is treated as a
default group by `uv`, so `uv sync` on its own installs it; a plain `uv sync --group docs` installs
`docs` **in addition to** `dev`, not instead of it — that's why CI's `docs-build` job, which runs
exactly `uv sync --frozen --group docs`, is able to run `uv run pytest ...` in the very next step
(see [Pull requests](pull-requests.md#what-ci-runs)). `mcp` is the one group/extra that stays out
unless you ask for it explicitly with `--extra mcp`. See [Code style](code-style.md) before reaching
for `uv add black` or similar — it is not as simple as it sounds in this repository.
## Frontend
The frontend uses [pnpm](https://pnpm.io/), pinned to major version 10 in CI
(`.github/workflows/tests.yml`).
```bash
cd frontend
pnpm install
pnpm run dev
```
`pnpm run dev` starts the Vite dev server on `http://localhost:5173`. `frontend/vite.config.js`
configures no dev-server proxy — there is no `server.proxy` entry in it — so the browser calls the
Django API directly, cross-origin: `frontend/src/lib/api.js` builds its base URL from
`PUBLIC_DJANGO_API_URL`, defaulting to `http://localhost:8000/api` when that variable is unset. That
means your backend's `CORS_ALLOWED_ORIGINS` has to include `http://localhost:5173` (it does by
default in `backend/.env.example` — see [Manual setup](../getting-started/manual-setup.md)) or every
request from the dev server fails as a CORS error, not a 404 or connection-refused, which is easy to
misdiagnose as a backend problem.
## Mobile
```bash
cd mobile
flutter pub get
flutter run
```
That gets the app compiling and running against whatever emulator or device is attached, but **it
will not talk to your local backend until you edit a compile-time constant first** — see
[Build and configure](../mobile/build-and-configure.md) and
[Connect to a self-hosted backend](../mobile/connect-to-self-hosted.md) before running it if you
intend to exercise real API calls rather than just confirm the app builds.
## Running everything
The fastest way to bring up the full stack — PostgreSQL, Redis, the Django API, a Celery worker, a
Celery beat scheduler, and the frontend dev server — together is Docker Compose, from the repo
root:
```bash
docker compose up --build
```
See [Docker](../self-hosting/docker.md) for what each service does and how to reach them. Running
the pieces natively instead, for faster iteration on the backend or frontend, means starting
PostgreSQL and Redis yourself and then running each of these in its own terminal:
```bash
# Terminal 1 — API
cd backend && uv run python manage.py runserver
# Terminal 2 — background jobs (recurring invoices, SLA scans, notification emails, ...)
cd backend && uv run celery -A crm worker --loglevel=INFO
# Terminal 3 — frontend
cd frontend && pnpm run dev
```
The Celery worker is optional for exercising most CRUD endpoints, but several features send email or
run scheduled work through it — see [Background jobs](../architecture/background-jobs.md) for what
runs where, and note that `celery -A crm beat` (the scheduler that actually fires the ten recurring
jobs described there) is a fourth, separate process that Docker Compose starts for you but the
native flow above does not.

81
docs/contributing/pull-requests.md Normal file
View file

@ -0,0 +1,81 @@
# Pull requests
## Before you open one
`CONTRIBUTING.md` (repo root) lays out the process this page assumes: branch from `master` with a
descriptive name, keep the change focused rather than mixing in unrelated edits, include tests where
practical, and update documentation when behavior, setup, or a public API changes. Its own checklist
is worth re-reading right before you open a PR:
- The change is focused and does not include unrelated edits.
- Relevant backend and frontend checks pass — see [Development setup](development-setup.md) and
[Testing](testing.md) for what to run locally before pushing.
- Tests cover new or changed behavior where practical — see
[Security rules](security-rules.md#prove-permission-checks-can-fail) specifically if the change
touches a permission check.
- Required migrations are included — see [What CI runs](#what-ci-runs) below for what happens if
one is missing.
- Documentation is updated.
- No credentials, tokens, or private data are committed.
In the PR description itself: explain what changed and why, how you tested it, and include
screenshots for any visible UI change.
## What CI runs
`.github/workflows/tests.yml` defines three jobs — `backend-tests`, `frontend-checks`, and
`docs-build` — triggered by two events, both scoped to the same path list (`backend/`, `frontend/`,
`docs/`, `mkdocs.yml`, `.readthedocs.yaml`, `.coveragerc`, or the workflow file itself): a pull
request against `master`, or a **push to `master` specifically** (`branches: [master]` on the `push`
trigger). Pushing a feature branch on its own doesn't run any of this — you'll only see these jobs
once you open the PR against `master`, so don't expect CI feedback from `git push` alone.
**`backend-tests`** installs WeasyPrint's system libraries, runs `uv sync --frozen` (installing
exactly what's locked in `uv.lock` — no re-resolution), then two steps in order:
```bash
uv run python manage.py makemigrations --check --dry-run
uv run pytest -m "not postgres_only" --tb=short -v
```
**A model change without a matching migration fails the first step**, before the test suite even
runs — `makemigrations --check --dry-run` exits non-zero if Django's autodetector finds any model
change that isn't already captured in a migration file. Generate and commit the migration
(`uv run python manage.py makemigrations`) before you push. See [Testing](testing.md#what-ci-runs)
for what the test step does and doesn't cover — notably, this job runs on SQLite, so it exercises no
RLS policy regardless of what you changed. On a push to `master` (not on pull requests) this job
additionally regenerates and commits a coverage badge.
**`frontend-checks`** installs with `pnpm install --frozen-lockfile`, then runs a Prettier check (a
formatting difference is reported as a warning, not a failure — see
[Code style](code-style.md#frontend)), `pnpm eslint .` (this one does fail the job), `pnpm check`
(svelte-check), and finally `pnpm build`.
**`docs-build`** installs the `docs` dependency group (`uv sync --frozen --group docs`) and runs
```bash
uv run mkdocs build --strict -f ../mkdocs.yml -d site
```
`--strict` fails the build on a broken nav entry or a broken internal link target that doesn't
exist. A second step then runs `common/tests/test_docs_build_excludes_internal.py`, which rebuilds
the site again and asserts that nothing under `docs/superpowers/` or `docs/design/` — internal specs
and plans, excluded via `exclude_docs` in `mkdocs.yml` — ends up published in the output. If you add
a documentation page, `mkdocs build --strict` is the same command you can run locally first; see
[Development setup](development-setup.md#backend) for the `uv run --group docs` invocation.
## Review expectations
Backend Python style is not enforced by CI (see [Code style](code-style.md#python) for why) — a
reviewer may still ask you to match the surrounding file's formatting even though no bot will flag
it automatically. Frontend style mostly is enforced (ESLint failures and type errors block the
build; Prettier differences only warn). Flutter has no CI at all, so `dart format .` and `flutter
analyze` before opening a PR are the only check that happens for mobile changes.
Beyond style, expect review to focus on the two things this codebase treats as non-negotiable: every
new or changed queryset filtering by `org=request.profile.org` (or being covered by RLS with a test
proving it — see [Adding an org-scoped model](adding-an-org-scoped-model.md)), and every new or
changed permission check being provable in both directions — see
[Security rules](security-rules.md#prove-permission-checks-can-fail). A PR that adds an endpoint
without a test for the rejected/forbidden path, not just the happy path, is the single most common
thing to get asked to add here.

105
docs/contributing/repo-layout.md Normal file
View file

@ -0,0 +1,105 @@
# Repository layout
This page is a map, not a tutorial — it tells you where to look, and links to the pages that explain
*why* each part is shaped the way it is.
## Top level
```
Django-CRM/
├── backend/ # Django REST Framework API — see Backend apps below
├── frontend/ # SvelteKit web app — see Frontend below
├── mobile/ # Flutter app — see Mobile below
├── docs/ # This documentation site (MkDocs Material)
├── docker/ # Docker-specific config: docker/postgres/init-rls-user.sql (creates the
│ # non-superuser RLS role) and docker/backend/entrypoint.sh — that's the
│ # whole directory, no Redis config lives here
├── mcp_server/ # The hosted MCP server (backend's optional `mcp` extra) — see
│ # Integrations → MCP server
├── mkdocs.yml # Docs site config and nav
├── docker-compose.yml # Full local stack — see Self-hosting → Docker
├── CONTRIBUTING.md # The short version of this Contributing section
├── RLS_SETUP.md # Stubbed pointer to PostgreSQL and RLS — kept only so links made before
│ # this documentation site existed still resolve; not a setup guide
│ # itself anymore
└── schema.yml # Generated OpenAPI schema (drf-spectacular) — not hand-maintained
```
`docs/` also holds internal specs and plans under `docs/superpowers/` and `docs/design/` — these are
excluded from the built site (`exclude_docs` in `mkdocs.yml`) and are not part of the published
documentation.
## Backend apps
`backend/crm/settings.py`'s `INSTALLED_APPS` lists the project's Django apps in this order (minus
Django/DRF/third-party entries):
| App | Owns |
|---|---|
| `common` | `User`/`Org`/`Profile` models, the RLS configuration, permission classes, every authentication path (JWT, personal access tokens, org API keys), notifications, tags, teams, custom fields, documents, and vertical packs |
| `accounts` | Account (company/customer) records |
| `cases` | Support tickets: `Case`, SLAs, CSAT, escalation/routing, inbound email |
| `contacts` | Contact (person) records |
| `leads` | Leads, lead pipelines/stages, lead-to-account conversion |
| `opportunity` | The sales pipeline: `Opportunity`, line items, stage aging, sales goals/quotas |
| `tasks` | Task management and Kanban boards (`Board`/`BoardColumn`/`BoardTask`) |
| `invoices` | Billing — invoices, estimates, recurring invoices, products, payments — and the anonymous client-portal views |
| `orders` | Sales orders and order line items |
| `business_hours` | Per-org working-hours calendars that SLA timers honor |
| `macros` | Canned responses / saved replies |
`common` is the one app every other app depends on for tenancy: it owns `BaseModel` and
`BaseOrgModel` (`common/base.py` — the abstract base classes every org-scoped model builds on, see
[Adding an org-scoped model](adding-an-org-scoped-model.md)), the tenancy middleware
(`common/middleware/get_company.py`'s `GetProfileAndOrg` and
`common/middleware/rls_context.py`'s `RequireOrgContext`), and the RLS utilities
(`common/rls/__init__.py` — `ORG_SCOPED_TABLES`, `get_enable_policy_sql`, and the rest of the SQL
generators). See [Architecture overview](../architecture/overview.md#backend-layout) for how these
pieces run together on a request, and [Multi-tenancy and RLS](../architecture/multi-tenancy-and-rls.md)
for the tenancy model itself.
Each domain app follows a similar internal shape: `models.py` (or a `models/` package for larger
apps), `serializer.py`, one or more view modules, `migrations/`, and `tests/`. `common` is larger and
less uniform than the others because it also holds cross-cutting views — the dashboard, global
search, tag usage counts — that pull in models from every domain app; see
[Architecture overview](../architecture/overview.md#backend-layout) for why that one-directional
model-layer rule doesn't extend to views.
## Frontend
```
frontend/src/
├── lib/
│ ├── api.js, api-helpers.js # API client (fetch/axios wrappers)
│ ├── components/ # Shared UI components (shadcn-svelte based)
│ ├── server/ # Server-side load-function helpers
│ ├── shell/ # App shell (nav, layout chrome)
│ ├── stores/ # Svelte stores
│ └── v2/ # Shared v2 design-system pieces (components, styles, formatters)
└── routes/
├── (app)/ # Authenticated routes — leads, accounts, contacts, opportunities
│ # (as "pipeline"), tasks, tickets ("support"), invoices, and more
├── (no-layout)/ # Public routes — login and the rest of the pre-auth flow
└── api/ # SvelteKit server routes (not the Django API)
```
Every page under `routes/(app)/` reads and writes the Django API described in the rest of this
site — there is no separate frontend-only data model.
## Mobile
```
mobile/lib/
├── config/ # API base URL and other compile-time configuration
├── core/ # Cross-cutting utilities
├── data/ # Data layer
├── providers/ # State management
├── routes/ # App routing
├── screens/ # UI screens
├── services/ # API and auth service classes
├── widgets/ # Shared widgets
└── main.dart # Entry point
```
See [Build and configure](../mobile/build-and-configure.md) for how this connects to a backend, and
[Release builds](../mobile/release-builds.md) for producing a signed build.

182
docs/contributing/security-rules.md Normal file
View file

@ -0,0 +1,182 @@
# Security rules
BottleCRM is a multi-tenant application where every org's data sits in the same database, the same
tables, next to every other org's. The rules on this page exist because the two ways that goes wrong
— one tenant reading or writing another tenant's data, and a non-admin reaching admin- or
platform-level actions — are the two outcomes every change touching auth, permissions, or
serializers has to rule out before it merges.
## The trust boundary
The backend (`backend/`) is the only trust boundary in this system.
[Architecture overview](../architecture/overview.md#the-three-clients) describes why that has to be
true structurally, not just as policy: a SvelteKit web app, a Flutter mobile app, and any
third-party integration all talk to the *same* Django REST Framework API over the same JSON
endpoints — there is no separate, more-trusted internal API. A rule enforced only in the SvelteKit
UI (disabling a button, hiding a form field, client-side validation) protects none of the three
clients, because a browser DevTools console, a modified mobile build, or a plain `curl` request
reaches the same endpoint directly, skipping whatever the UI would have prevented. Every constraint
that matters has to be enforced in the DRF view or serializer, full stop.
## Identity is server-derived
Who is making a request, which org they're acting in, and what role they hold are facts the server
establishes and re-validates on every request — never values read from the request body, a query
string, or a custom header. [Authentication](../architecture/authentication.md#claims) shows the
concrete mechanism: the `org_id` claim inside a caller's signed JWT is not simply trusted — the
middleware that resolves it re-checks it against an active `Profile` row on every single request:
```python
profile = Profile.objects.select_related("org").get(
user_id=user_id, org_id=org_id, is_active=True
)
```
If that lookup fails — the membership was revoked, the profile was deactivated — the request gets a
403 asking the caller to sign in again, rather than proceeding on a claim that's gone stale. The
practical rule for any view or serializer you write: identity comes from `request.user` and
`request.profile`, both set by middleware before your code runs (see
[Overview](../architecture/overview.md#request-lifecycle)) — never from a field the client sent.
`role`, in particular, is a display convenience when it appears in a JWT payload; every real
permission check reads `request.profile.role` fresh from the database, not from the token.
## Tenant scoping
Pair every permission check with an explicit org filter — the two are independent controls, and
neither substitutes for the other. `HasOrgContext` (`backend/common/permissions.py`) is the standard
permission class on nearly every org-scoped view:
```python
class HasOrgContext(permissions.BasePermission):
def has_permission(self, request, view):
if not hasattr(request, "profile") or request.profile is None:
return False
if not hasattr(request, "org") or request.org is None:
return False
if not request.profile.is_active:
return False
return True
```
That establishes "a real, active member of this org is making this request" — it does not filter
which rows the view returns. Every queryset still has to add
`Model.objects.filter(org=request.profile.org)` itself, and every `create()` / `serializer.save()`
still has to pass `org=request.profile.org` explicitly, never accept it from the request body. RLS
(PostgreSQL Row-Level Security) backs this up at the database layer, but it's a safety net for when
the filter is missing, not a substitute for writing it — see
[The two-layer contract](../architecture/multi-tenancy-and-rls.md#the-two-layer-contract) for why
both layers are required, and [Adding an org-scoped model](adding-an-org-scoped-model.md) for the
full checklist when the view is for a brand-new model.
## Server-owned fields
Any field the server is supposed to derive — `org`, `created_by`, `profile`, a publish/visibility
flag — must never be settable from `request.data`. There are two real mechanisms in this codebase
for that, not one:
**Declare the field `read_only=True` or list it in `read_only_fields`.** `TeamCreateSerializer` in
`backend/common/serializer.py` declares `read_only_fields = ("created_at", "created_by", "org")`;
`leads/serializer.py` sets `read_only_fields = ("id", "created_at", "updated_at", "org")` on
`LeadStageSerializer` and `LeadPipelineSerializer`; and `TimeEntrySerializer`
(`backend/cases/serializer.py`) declares the field itself read-only — `profile =
ProfileSerializer(read_only=True)` — for exactly the reason this section opens with: `profile` is
who logged the time entry, and it has to be a fact the server sets, not one a client requests.
**Or don't accept it through the serializer at all, and set it explicitly at the call site.**
`TimeEntryCreateSerializer`'s own docstring states this directly: "`case`, `profile`, and `org` are
injected by the view and not accepted from the client" — and indeed none of the three appear in its
`Meta.fields`. The view (`backend/cases/time_views.py`) backs that up:
```python
entry = TimeEntry.objects.create(
org=request.profile.org,
case=case,
profile=request.profile,
**serializer.validated_data,
)
```
`serializer.validated_data` can only ever contain the fields the serializer declared in the first
place, so there's nothing for a client to override here even by guessing field names — the
equivalent of popping a key out of `request.data` before `save()`, but enforced by what the
serializer's `fields` tuple omits rather than by a runtime pop.
The sharpest example of *why* this matters is `is_sample` — a flag shared, via one imported
`SAMPLE_DATA_HELP_TEXT` constant, by six models: `Lead`, `Account`, `Contact`, `Case`, `Opportunity`,
and `Task` (`grep -rl SAMPLE_DATA_HELP_TEXT backend/` finds it in each app's `models.py`, plus
`common/base.py` where it's defined once). `leads/serializer.py` declares
`read_only_fields = ("is_sample",)`. The help text itself, defined once in `backend/common/base.py`
and shared by every model that has the field, states the reasoning as a security invariant rather
than documentation:
```python
SAMPLE_DATA_HELP_TEXT = (
"True only for demo rows created by common.packs.applier._apply_sample_data. "
"Server-set exclusively — never expose this as a writable field on any "
"serializer. It is the sole key common.packs.applier.clear_sample_data uses "
"to decide what to delete, so a client-writable path here would let a user "
"mark an arbitrary real record as sample and have it deleted."
)
```
If a field like this were writable, a hostile request wouldn't need a bug in the delete logic at
all — it would just flip the flag on someone else's real record and let the legitimate "clear sample
data" action do the deleting. When you add a field the server is meant to own, ask what a client
could achieve by setting it directly, not just whether the happy path needs it to be read-only.
## Prove permission checks can fail
A permission check that always evaluates to the same answer isn't a check — it's dead code wearing
a check's shape, and this codebase has a well-documented history of exactly that happening by
accident. `Case.created_by` (like nearly every model's `created_by`) is a foreign key to `User`, not
to `Profile`. Comparing a `Profile` instance directly against it —
```python
request.profile == self.object.created_by # always False: Profile compared to a User FK
```
— type-checks in Python and never raises, but is never `True`, which silently turned an intended
"creator or admin" check into "admin only" with no error anywhere. This isn't hypothetical in this
codebase: `git log -L` on `CaseAttachmentView.delete` in `backend/cases/views.py` shows a commit
(`c9250dd`) whose diff replaces exactly this —
```diff
- request.profile.role == "ADMIN"
- or request.profile.is_admin
- or request.profile == self.object.created_by
+ is_org_admin(request.profile)
+ or request.profile.user_id == self.object.created_by_id
```
— confirming from the actual commit history, not from a comment describing it, that the
`Profile`-vs-`User` comparison was live code before it was replaced with the type-correct
`request.profile.user_id == self.object.created_by_id`.
[Permissions and roles](../architecture/permissions-and-roles.md#object-level-checks) documents this
pattern in depth, including which apps had it fixed and which — as of that page's own last check
against source — still don't; read it before writing a new `created_by` comparison, and compare
`profile.user_id` to `obj.created_by_id`, never `profile` to `obj.created_by` directly.
The only way to know a check can fail is to write a test that makes it fail. `leads/tests/
test_lead_access_and_totals.py` states the rule its own tests follow: "Every permission test asserts
both directions. A check that can only return False is indistinguishable from a check that is never
reached." In practice that means a pair of tests, not one — for example
`opportunity/tests/test_sales_goals.py`:
```python
def test_create_goal_admin(self, admin_client, org_a):
...
response = admin_client.post(self.GOALS_URL, data, format="json")
assert response.status_code == 201
def test_create_goal_non_admin_forbidden(self, user_client):
...
response = user_client.post(self.GOALS_URL, data, format="json")
assert response.status_code == 403
```
A single test asserting only the `403` case would pass identically whether the admin check works
correctly or whether it denies *everyone*, admin included, because of a mistake like the one above.
Only the paired `True` test catches that. See [Testing](testing.md) for the fast-iteration commands,
and [Adding an org-scoped model](adding-an-org-scoped-model.md#the-tests) for the same pairing
applied to tenant isolation specifically.

113
docs/contributing/testing.md Normal file
View file

@ -0,0 +1,113 @@
# Testing
## Running tests
From `backend/`:
```bash
uv run pytest
```
`backend/pytest.ini` sets `addopts` to include coverage by default —
`--cov=. --cov-config=../.coveragerc --cov-report=term-missing --cov-report=html:htmlcov
--cov-report=xml:coverage.xml` — alongside `--strict-markers --tb=short -v`. So a plain `uv run
pytest` always computes coverage and writes `backend/htmlcov/` and `backend/coverage.xml`, even for
a single test file.
Scope a run to one app, one file, or a keyword the same way you would with any pytest project:
```bash
uv run pytest accounts/tests/ # one app's test package
uv run pytest leads/tests/test_leads_kanban.py # one file
uv run pytest -k "test_org_isolation" # by keyword — matches the cross-tenant
# isolation test present in most API test
# modules (leads, cases, contacts, ...)
```
## Fast iteration
Coverage instrumentation and the three report formats above add real overhead. For the
edit-run-edit loop, drop coverage and stop at the first failure:
```bash
uv run pytest --no-cov -x
```
`--strict-markers` (also set in `addopts`) means a typo'd **custom** marker fails the run immediately
rather than being silently ignored — every custom marker you use has to be declared in `pytest.ini`'s
`markers` list. It doesn't touch markers a plugin registers itself, like `@pytest.mark.django_db`
(from `pytest-django`) — those work without appearing in `pytest.ini` at all, which is why every
example on this page uses `django_db` freely alongside the two markers that actually are declared.
## Markers
`pytest.ini` declares exactly two custom markers:
```ini
markers =
slow: marks tests as slow
postgres_only: marks tests that require PostgreSQL
```
`postgres_only` is in active use — exactly six tests carry it, confirmed by collecting the marker
directly (`uv run pytest -m postgres_only --collect-only` → `6/2776 tests collected`), across four
files: `common/tests/test_pat_auth.py` (one test), `common/tests/test_pat_model.py` (one),
`common/tests/test_rls_pipeline_tables.py` (two), and `invoices/tests/test_portal_rls.py` (two).
(`common/tests/test_pat_integration.py` mentions `postgres_only` only in a comment explaining that
its own tests deliberately *don't* carry the marker and run on SQLite instead — it contributes zero
of the six.) Four of the six assert RLS policy state directly (`pg_policies`,
`pg_class.relrowsecurity`) — the two in `test_rls_pipeline_tables.py`, plus one each in
`test_pat_model.py` and `test_portal_rls.py` confirming `personal_access_token` and
`portal_access_token` deliberately carry *no* policy. The other two assert isolation *behaviour*
under a real, RLS-enforcing role instead of reading policy metadata:
`test_pat_auth.py::test_cross_org_pat_cannot_read_other_orgs_leads` calls the real `/api/leads/`
endpoint and checks a foreign org's lead never appears, and
`test_portal_rls.py::test_resolution_reads_estimate_that_empty_context_hides` exercises the full
portal-token resolution path. `slow` is declared but,
as of this writing, applied to nothing (`grep -rn "pytest.mark.slow" --include='*.py' .`, excluding
`venv/`, returns no matches in application code) — it's reserved for future use, not a marker you'll
see filtering anything today.
## The SQLite caveat
`crm.test_settings` (`backend/crm/test_settings.py`) is the settings module `pytest.ini` points at,
and its `DATABASES` is hardcoded to an in-memory SQLite database — not read from an environment
variable, not conditional on anything. Its own module docstring states why: "RLS (Row-Level
Security) is PostgreSQL-only and is skipped on SQLite." That means **the default `uv run pytest`
run never exercises a single RLS policy** — every test runs against a database engine that has no
concept of row security at all. Everything you'd normally rely on RLS to catch, an org-filter bug
included, is invisible to this default suite; only the explicit `Model.objects.filter(org=...)` in
application code is being tested. See [Multi-tenancy and RLS](../architecture/multi-tenancy-and-rls.md#the-two-layer-contract)
for why that ORM filter has to be correct on its own merits, independent of RLS.
The `postgres_only`-marked tests exist to close part of that gap, and they self-skip safely rather
than fail when there's no PostgreSQL connection: each one checks `connection.vendor != "postgresql"`
and calls `pytest.skip(...)` if so. Under the SQLite settings this repository ships, that check is
always true, so these six tests always skip — they don't fail, and a green `uv run pytest` run tells
you nothing about whether they'd pass. One of them — `test_portal_rls.py`'s module docstring —
describes an intended way to actually run them for real, against a non-superuser PostgreSQL role:
`RLS_ENFORCE=1`, pointed at a `crm.test_settings_postgres` module. A comment in `backend/.env`
(gitignored — `.gitignore:20` — and absent from `.env.example`, so a fresh clone has neither the
file nor the reference) names the same two pieces. Neither source is authoritative, because that
settings module does not exist anywhere in this checkout as of this writing, so this documented path
does not currently work. If you need to prove RLS isolation holds against real data rather than
just reading the policy SQL, use `manage.py manage_rls --test` against a real, non-superuser-configured PostgreSQL
database instead — see [PostgreSQL and RLS](../self-hosting/postgresql-and-rls.md#verifying-isolation)
for exactly what it proves and how to set up that role.
## What CI runs
`.github/workflows/tests.yml`'s `backend-tests` job runs on the same SQLite settings described
above — there's no PostgreSQL service in that job — so `postgres_only` tests skip there too, and
CI itself has no path that runs them for real. Two steps matter, in this order:
```bash
uv run python manage.py makemigrations --check --dry-run
uv run pytest -m "not postgres_only" --tb=short -v
```
The migration check runs first and fails the build if any model change is missing its migration —
see [Pull requests](pull-requests.md#what-ci-runs) for what that means for your workflow. The
`-m "not postgres_only"` filter on the test step is redundant with the tests self-skipping, but
makes the exclusion explicit in the command itself rather than relying on every future
`postgres_only` test remembering its own skip guard.

101
docs/getting-started/demo-data.md Normal file
View file

@ -0,0 +1,101 @@
# Demo data and packs
BottleCRM has two independent ways to populate an organization with non-empty data: the
`seed_data` management command, which generates a large, randomized dataset for an entire org, and
**vertical packs**, which apply a small, curated, industry-specific set of pipelines and sample
records to an org that already exists.
## Seeding a demo organization
```bash
uv run python manage.py seed_data --email you@example.com
```
Run from `backend/` (or `docker compose exec backend python manage.py seed_data --email
you@example.com` against the Docker stack). This is the command referenced in
[Docker quick start](docker-quick-start.md) and [First sign-in](first-sign-in.md).
The command (`backend/common/management/commands/seed_data.py`) creates an organization and fills
it with realistic-looking fictional data — contacts, accounts, leads, opportunities, cases, tasks,
products, invoices, estimates, recurring invoices, sales goals, teams and tags — using curated
name/company/address pools and reserved demo domains (`example.com`, `.example`) so nothing it
generates resembles a real person or business. The **first** organization it creates is always
named **`MicroPyramid`**, specifically so local-dev workflows have a stable, known org name to
sign in against with `devlogin --org MicroPyramid`. An **admin profile** is created for whichever
`--email` you pass. Re-running the command with the same email reuses the existing org and user
rather than creating duplicates — it's safe to run more than once.
Counts for every entity type (`--leads`, `--accounts`, `--opportunities`, `--invoices`, and so on)
and locale (`--currency`, `--country`) are configurable flags; see `--help` for the full list. A
`--clear` flag deletes existing CRM data for re-seeding (but never users, orgs or profiles) — it's
a separate, broader operation from the sample-data clearing described below, and is not scoped to
records a pack created.
## Vertical packs
A vertical pack is a small, declarative JSON manifest — not a data generator — that shapes an org
toward one industry: a starter pipeline (with stages) for leads, cases and tasks; a handful of
custom field definitions; tags; products; some terminology overrides (for example relabeling
"Lead" as "Enquiry"); and a short list of named sample records (accounts, contacts, deals,
tickets, leads, tasks) that reference each other by name within the manifest.
Four packs ship in `backend/packs/`, one manifest file per pack:
| Pack id | File | Description |
|---|---|---|
| `real-estate` | `real-estate.json` | Brokers, agencies and developers selling residential and commercial property. |
| `professional-services` | `professional-services.json` | Agencies, consultancies and accounting firms selling billable work. |
| `education-admissions` | `education-admissions.json` | Coaching centres, training institutes and colleges managing admissions. |
| `events-weddings` | `events-weddings.json` | Wedding planners, event agencies and venues taking enquiries through to a delivered event. |
Every manifest is validated strictly at load time (`backend/common/packs/schema.py`) — an unknown
key, an invalid `stage_type`, a `sample_data` reference to an account or stage name that isn't
declared elsewhere in the same pack, and similar mistakes all fail validation rather than being
silently ignored or half-applied.
## Applying a pack
Applying a pack is handled by `backend/common/packs/applier.py`'s `apply_pack()`, reachable
through:
- **The UI** — the Settings → Organization page (for an existing org) and the organization
creation page (for a brand-new org) both offer a pack picker, calling the same backend endpoint.
- **The API** — `POST /api/packs/<pack_id>/apply/`, restricted to an organization admin
(`request.profile.role == "ADMIN"`); any authenticated member can list available packs via
`GET /api/packs/`, but only an admin can apply one.
**Applying a pack is strictly additive.** The applier's own description of itself: it creates
what's missing and skips what already exists, and it "never calls `.update()`, never calls
`.save()` on a row it did not just construct, and never deletes." A pipeline whose name already
exists in the org is skipped entirely (along with its stages); a custom field, tag or product that
already exists by its matching key is skipped individually. This makes applying the same pack
twice — or applying a second pack to an org that already has one — safe: it can only add what's
still missing, never overwrite or remove something you've since changed. The one exception is
terminology and the org's `vertical` label, which are merged in (only keys not already set are
added) rather than duplicated.
Sample records created by a pack (accounts, contacts, deals/opportunities, tickets/cases, leads
and tasks) are marked with an `is_sample` boolean field, set only by the applier — it is
`read_only` on every serializer that exposes it, so it can never be set by a client request. Every
sample row is also tagged with a "Sample data" tag and assigned to whichever admin applied the
pack, purely for visibility in the UI; the tag carries no special authority and isn't what the
system uses to know a row is sample data.
If an org already has any sample rows (from this pack or a previous one), applying `sample_data`
again is skipped — a pack's sample data is only ever created once per org, regardless of how many
times you re-apply.
## Clearing sample data
**Only `clear_sample_data()` deletes sample records**, via `DELETE /api/packs/sample-data/`
(also admin-only, also reachable from the Settings → Organization page). It is the sole
counterpart to pack-applied sample data: nothing else in the system deletes rows just because
`is_sample` is set.
Clearing is scoped strictly to rows where `is_sample=True` for the calling org, deleted in an
order that removes child records (tasks, leads) before the parents they can reference (tickets,
deals, contacts, accounts). It never removes anything else — a real record you created yourself is
untouched even if it happens to reference a sample record, and a sample row is **retained rather
than deleted** if a non-sample record still depends on it (for example, a real invoice raised
against a demo account). Retained rows are reported back, not silently skipped, so the caller
knows some sample data was intentionally kept rather than assuming the clear was incomplete.

136
docs/getting-started/docker-quick-start.md Normal file
View file

@ -0,0 +1,136 @@
# Docker quick start
The fastest way to run BottleCRM locally is Docker Compose. It brings up PostgreSQL, Redis, the
Django API, two Celery processes and the SvelteKit frontend from one file, using the checked-in
`.env.docker` for configuration.
## Prerequisites
- Docker Engine with the Compose v2 plugin (the `docker compose` subcommand, not the standalone
`docker-compose` binary — `docker-compose.yml` is written for the plugin).
- A clone of the repository, with `docker-compose.yml` at its root.
## Clone and configure
All commands below run from the repository root — the directory that contains
`docker-compose.yml` and `Dockerfile`.
`.env.docker` is checked into the repository with working development defaults and is loaded by
every service in `docker-compose.yml`. You don't need to copy or edit anything to get a working
stack. If you do want to override a value locally (for example to add Google OAuth credentials),
create `.env.docker.local` next to it — `docker-compose.yml` loads it as an optional, gitignored
second `env_file` whose values win over `.env.docker`.
The variables `.env.docker` actually defines are:
| Variable | Purpose |
|---|---|
| `SECRET_KEY` | Django secret key. The shipped value is an insecure dev placeholder. |
| `DEBUG` | `True` in the dev compose file — this is also what allows `devlogin` to run (see [First sign-in](first-sign-in.md)). |
| `ENV_TYPE` | `dev`. |
| `ALLOWED_HOSTS` | `localhost,127.0.0.1,backend`. |
| `DOMAIN_NAME`, `SWAGGER_ROOT_URL` | Both `http://localhost:8000`. |
| `DBNAME`, `DBUSER`, `DBPASSWORD`, `DBHOST`, `DBPORT` | Connection details the Django `backend` (and Celery) containers use to reach `db`. |
| `POSTGRES_DB`, `POSTGRES_USER`, `POSTGRES_PASSWORD` | Used only by the `db` service itself to create the database and its own superuser role. |
| `CELERY_BROKER_URL`, `CELERY_RESULT_BACKEND` | Both point at the `redis` service, database `0`. |
| `DEFAULT_FROM_EMAIL`, `ADMIN_EMAIL`, `ADMIN_PASSWORD` | Console email backend (dev), and the credentials used to bootstrap a Django admin superuser — see [What you got](#what-you-got). |
| `CORS_ALLOW_ALL` | `True` for local development. |
| `GOOGLE_CLIENT_ID`, `GOOGLE_CLIENT_SECRET`, `GOOGLE_REDIRECT_URI` | Blank by default; Google sign-in is disabled until these are set. |
| `PUBLIC_DJANGO_API_URL` | The API base URL the frontend container is built against, `http://localhost:8000`. |
## Start the stack
```bash
docker compose up --build
```
This builds and starts six services, defined in `docker-compose.yml`:
| Service | Image / build | Host port |
|---|---|---|
| `db` | `postgres:16-alpine` | `5432` |
| `redis` | `redis:7-alpine` | `6379` |
| `backend` | built from the repo root `Dockerfile` | `8000` |
| `celery-worker` | same image as `backend`, runs `celery -A crm worker --loglevel=info` | — |
| `celery-beat` | same image as `backend`, runs `celery -A crm beat --loglevel=info` | — |
| `frontend` | built from `frontend/Dockerfile` | `5173` |
`backend`, `celery-worker` and `celery-beat` all wait for `db` and `redis` to report healthy
before starting. The `backend` container's entrypoint (`docker/backend/entrypoint.sh`) waits for
PostgreSQL to accept connections, runs `migrate`, runs `create_default_admin`, collects static
files, and then starts the Django dev server on `0.0.0.0:8000`.
**The database role matters.** `db` starts as the `postgres` superuser (from `POSTGRES_USER`) —
that account exists only so the container can create the `crm_db` database. On first boot,
`docker/postgres/init-rls-user.sql` runs automatically (via Postgres's
`/docker-entrypoint-initdb.d/` mechanism) and creates a second, non-superuser role, `crm_user`,
granting it the privileges it needs on `crm_db`. The Django containers connect as `crm_user`
(`DBUSER` in `.env.docker`), never as `postgres`. This is required, not incidental: PostgreSQL
Row-Level Security is bypassed entirely for a superuser connection, so if the app ever connected
as `postgres`, every RLS policy in the schema would be silently inert and tenant isolation would
depend on application-level filtering alone.
## Migrate and seed
`entrypoint.sh` already runs `migrate` on every container start, so the schema is up to date as
soon as `backend` reports it's serving. To load demo data, run the seeder inside the running
`backend` container:
```bash
docker compose exec backend python manage.py seed_data --email you@example.com
```
`seed_data` creates an organization named **`MicroPyramid`** (the first organization it creates
is always named this, specifically so local-dev workflows have a known, stable name to sign in
against) and an **admin profile** for the user you passed with `--email`, along with a full set
of demo leads, accounts, contacts, opportunities, cases, tasks, invoices and more. Re-running the
command reuses the existing `MicroPyramid` org and admin user rather than creating duplicates.
You can also run migrations manually, the same way you'd run any other management command against
the container:
```bash
docker compose exec backend python manage.py migrate
```
## Sign in
With `DEBUG=True` (the default in `.env.docker`), the fastest way in is `devlogin` — a
management command that mints a JWT pair directly, without going through Google OAuth or email.
Full details, including how it resolves `--org` and what it requires of the target user, are in
[First sign-in](first-sign-in.md). For the org and user you just seeded:
```bash
docker compose exec backend python manage.py devlogin you@example.com --org MicroPyramid
```
This prints an access token, a refresh token, and the `MicroPyramid` org's UUID, already bound
into the token — see [First sign-in](first-sign-in.md) for how to use them from the frontend at
`http://localhost:5173`.
## What you got
After `docker compose up --build` and one `seed_data` run, you have:
- A PostgreSQL 16 database (`crm_db`) with RLS-enforcing policies applied by the migrations.
- A Django API on `http://localhost:8000`, running under the dev server (not a production WSGI
server — this compose file is for development).
- A Celery worker and a Celery beat scheduler, both connected to the `redis` service, running the
periodic and background tasks registered in `backend/crm/celery.py`.
- A SvelteKit frontend on `http://localhost:5173`, built against `PUBLIC_DJANGO_API_URL=http://localhost:8000`.
- A Django admin superuser, created automatically by `create_default_admin` on first boot, using
`ADMIN_EMAIL` / `ADMIN_PASSWORD` from `.env.docker` (`admin@localhost` / `admin` unless you
override them). This account can sign in to Django's own `/admin/` site; it does not have a CRM
organization profile, so it can't sign in to the CRM app itself unless you also give it a
profile via `seed_data` or the org-creation flow.
- If you ran `seed_data`, an org named `MicroPyramid` with an admin profile for the email you
passed, plus a large set of demo records.
## Next steps
- [First sign-in](first-sign-in.md) — the three ways to get a session, including the exact
`devlogin` invocation and what Google OAuth and magic links need.
- [Demo data and packs](demo-data.md) — what `seed_data` creates in detail, and how vertical packs
layer industry-specific pipelines and sample records on top of it.
- If Docker isn't the right fit, [Manual setup](manual-setup.md) covers running the backend,
database, Redis, Celery and frontend directly on your machine.

141
docs/getting-started/first-sign-in.md Normal file
View file

@ -0,0 +1,141 @@
# First sign-in
BottleCRM issues sessions as JWT access/refresh pairs (`rest_framework_simplejwt`), not
server-side sessions or cookies set by a traditional username/password login endpoint — there is
no such endpoint. `backend/common/urls.py` registers exactly three ways to obtain a token pair.
## The three sign-in paths
| Path | Where it's implemented | Typical use |
|---|---|---|
| `devlogin` management command | `backend/common/management/commands/devlogin.py` | Local development only |
| Google OAuth | `GoogleOAuthCallbackView` (web, PKCE) and `GoogleIdTokenView` (mobile ID token), both in `backend/common/views/auth_views.py` | Self-hosted deployments with Google OAuth configured |
| Magic links (passwordless email) | `MagicLinkRequestView`, `MagicLinkVerifyView`, `MagicLinkVerifyCodeView`, all in `backend/common/views/auth_views.py` | Any deployment — needs only outbound email |
All three ultimately mint tokens through the same `OrgAwareRefreshToken.for_user_and_org()` helper,
so the resulting access/refresh pair behaves identically regardless of which path produced it.
## Local development: devlogin
`devlogin` mints an access/refresh token pair for an existing user directly from the command
line, with no browser round-trip and no OAuth provider involved. It **refuses to run unless
`settings.DEBUG` is `True`** — the very first thing the command does is raise a `CommandError` if
`DEBUG` is falsy. That guard is also what makes the command safe to ship in the same codebase that
runs in production: `DEBUG=False` there, and the command becomes a no-op.
Invocation:
```bash
uv run python manage.py devlogin <email> --org <name-or-uuid>
```
Run from `backend/` (or, against a Docker stack, as
`docker compose exec backend python manage.py devlogin <email> --org <name>`).
It prints three things: an **access token**, a **refresh token**, and — when `--org` is given —
the **org's UUID** it bound the token to.
A few specifics worth knowing:
- `--org` accepts either a name or a UUID, and resolves by **id first, then name**: it tries an
exact `Org.objects.get(id=org_arg)` lookup first, and only if that fails — not found, or the
string isn't a valid UUID at all (`Org.objects.get(id=...)` raises Django's `ValidationError`,
not `ValueError`, for a non-UUID string, so both are caught alongside `Org.DoesNotExist`) —
falls back to matching on `Org.name`. If more than one org shares that name, it refuses to guess
and asks you to pass an ID instead.
- The target **user must already have an active `Profile` in that org** — `devlogin` does not
create one. If it doesn't find one, it raises a `CommandError` naming the missing
user/org pair rather than silently proceeding without org context. (Passing `--create` will
create the *user* if it doesn't exist yet, but that's independent of the profile requirement —
a freshly created user still needs a `Profile` in the org before `--org` will succeed for them.)
- A token minted with `--org` **already carries the `org_id` claim**, so there's no follow-up
org-switch call needed — the frontend can use it immediately. Omit `--org` and you get a token
with no org bound; you'd then need to pick one interactively (see
[Choosing an organization](#choosing-an-organization) below) before the app has anything to show.
### Using the tokens in the browser
The snippet `devlogin` prints (`localStorage.setItem('access_token', ...)`,
`localStorage.setItem('refresh_token', ...)`) only writes `localStorage` — correct for
`frontend/src/lib/api.js`'s own `STORAGE_KEYS`, which is what the API client reads for direct,
client-side fetches, but **not enough on its own to sign you in through the SvelteKit app**.
`frontend/src/hooks.server.js` authenticates every server-rendered request from three **cookies**
instead — `event.cookies.get('jwt_access')`, `('jwt_refresh')` and `('org')` — and nothing
server-side reads `localStorage` at all. Without those cookies set, `hooks.server.js` treats the
request as signed out and redirects to `/login`, no matter what `localStorage` holds.
To get past the guard, open the devtools console on `http://localhost:5173`, run the printed
`localStorage.setItem(...)` snippet, and set the matching cookies yourself:
```js
const oneYear = 60 * 60 * 24 * 365;
document.cookie = `jwt_access=${ACCESS_TOKEN}; path=/; max-age=${oneYear}; samesite=lax`;
document.cookie = `jwt_refresh=${REFRESH_TOKEN}; path=/; max-age=${oneYear}; samesite=lax`;
document.cookie = `org=${ORG_UUID}; path=/; max-age=${oneYear}; samesite=lax`;
```
Substitute the printed access token, refresh token, and (if you passed `--org`) org UUID for
`ACCESS_TOKEN` / `REFRESH_TOKEN` / `ORG_UUID`; skip the `org` cookie entirely if you omitted
`--org`. Then reload. The root route `/` **is** the dashboard —
`frontend/src/routes/(app)/+page.svelte` sits directly at the route group's root — there is no
`/dashboard` route anywhere in the route tree.
## Google OAuth
Two endpoints handle Google sign-in, for two different clients:
- **`POST /api/auth/google/callback/`** (`GoogleOAuthCallbackView`) — the web flow, used by the
SvelteKit frontend. It's a PKCE authorization-code exchange: the frontend sends `code`,
`code_verifier` and `redirect_uri`; the backend exchanges them with Google for tokens, decodes
the ID token payload, and requires `email_verified` to be true before it will create or sign in
a user — an unverified Google email is rejected outright.
- **`POST /api/auth/google/`** (`GoogleIdTokenView`) — the mobile flow. The client sends an
already-obtained Google ID token (`idToken`); the backend verifies it against Google
(`google.oauth2.id_token.verify_oauth2_token`) rather than doing a code exchange, and applies the
same `email_verified` check.
Both paths get-or-create a `User` by email, reject sign-in for a deactivated account
(`user.is_active is False`), backfill the user's name from the Google profile if it wasn't set
yet, and mint an org-unbound token (no `--org` equivalent here — see
[Choosing an organization](#choosing-an-organization)). Google OAuth is only usable once
`GOOGLE_CLIENT_ID` / `GOOGLE_CLIENT_SECRET` are configured; in the Docker quick start's
`.env.docker` both are blank, which disables it.
## Magic links
Passwordless sign-in by email, in two deliveries:
- **Link delivery** — `POST /api/auth/magic-link/request/` creates a token, emails a sign-in link,
and `POST /api/auth/magic-link/verify/` (`MagicLinkVerifyView`) exchanges the token in that link
for an access/refresh pair.
- **Code delivery** — the same request endpoint, with `delivery: "code"`, instead emails a 6-digit
one-time code. `POST /api/auth/magic-link/verify-code/` (`MagicLinkVerifyCodeView`) checks the
code (max 5 attempts before the code is burned) and returns tokens. This is the mobile-friendly
variant — no link-tap round-trip through a browser.
The request endpoint always returns `200` with a generic "if this email is valid…" message,
whether or not the address has an account — this is deliberate, to avoid leaking which emails are
registered. Requests are rate-limited to 5 tokens per email per hour, and each token expires 10
minutes after it's issued. Verifying creates the user if the email doesn't exist yet (same
deactivated-account check as the OAuth paths applies), and, like OAuth, mints an org-unbound
token unless the user already has an active profile in exactly one org — in that case the
response includes that org's tokens and a `current_org` field directly.
In a self-hosted deployment, magic links require a working outbound email backend; in the Docker
quick start's dev configuration, email is printed to the container's stdout instead of actually
sent (a console backend), so you'd read the link or code from the `backend` container logs.
## Choosing an organization
A token minted without an org (any sign-in path except `devlogin --org`) still needs one before
the app has anything to show — every org-scoped view requires org context. The SvelteKit frontend
handles this at its `/org` route: it calls `GET /api/auth/me/` to list the organizations the
signed-in user has an active profile in, and lets them pick one. Selecting an org calls
`POST /api/auth/switch-org/` (`OrgSwitchView`) with the chosen `org_id`; the backend verifies the
user has an active `Profile` in that org and returns a fresh access/refresh pair with the org
claim bound in, the same shape `devlogin --org` produces directly. If the presented refresh token
is also sent, the backend blacklists it as part of the switch so the old, org-less token can't be
reused afterward.
The same `/api/auth/switch-org/` endpoint is what a user with access to multiple organizations
uses later to move between them — it isn't only a first-sign-in step.

171
docs/getting-started/manual-setup.md Normal file
View file

@ -0,0 +1,171 @@
# Manual setup
Running the backend, database, Redis, Celery and frontend directly on your machine, without
Docker. Useful if you want faster iteration on backend code than a bind-mounted container gives
you, or if you're setting up on a host where Docker isn't available.
## Prerequisites
- **Python 3.12 or later** — `backend/pyproject.toml` declares `requires-python = ">=3.12"`, and
`backend/.python-version` pins `3.12`.
- **[uv](https://docs.astral.sh/uv/)** — the backend is managed with uv, not pip or a manually
created virtualenv. `uv sync` reads `backend/pyproject.toml` and `backend/uv.lock` and creates
a `.venv/` for you; every backend command in this guide runs via `uv run <cmd>`, which resolves
that venv automatically. Install it once, system-wide:
```bash
curl -LsSf https://astral.sh/uv/install.sh | sh
# or, on macOS: brew install uv
```
- **PostgreSQL** — the only supported database backend (`django.db.backends.postgresql` in
`backend/crm/settings.py`); Row-Level Security depends on it.
- **Redis** — used as the Celery broker and result backend.
- **Node.js and [pnpm](https://pnpm.io/)** — for the SvelteKit frontend in `frontend/`.
## Backend
From `backend/`:
```bash
cd backend
uv sync
```
This creates `.venv/` and installs everything listed in `pyproject.toml`'s `dependencies`
(Django, DRF, Celery, `djangorestframework-simplejwt`, `weasyprint`, etc.) at the versions locked
in `uv.lock`.
Copy the example environment file and adjust it for your local database and Redis:
```bash
cp .env.example .env
```
`backend/crm/settings.py` loads this file via `python-dotenv` (`load_dotenv()` at import time), so
anything you put in `.env` is picked up the next time you run a management command. At minimum,
review `DBNAME`, `DBUSER`, `DBPASSWORD`, `DBHOST`, `DBPORT` against the database role you create in
the next section.
Once `.env` points at a reachable database, apply migrations:
```bash
uv run python manage.py migrate
```
And run the dev server:
```bash
uv run python manage.py runserver
```
By default this serves on `http://127.0.0.1:8000`.
## Database
The Django `DATABASES` setting (`backend/crm/settings.py`) reads `DBNAME`, `DBUSER`,
`DBPASSWORD`, `DBHOST` and `DBPORT` from the environment, defaulting to a local `postgres`
superuser connection if they're unset. Don't leave it on that default: **the role Django connects
as must not be a PostgreSQL superuser**, or Row-Level Security is bypassed for every query that
role makes, regardless of what the RLS policies say.
Create a dedicated, non-superuser role and database, for example:
```bash
sudo -u postgres psql
```
```sql
CREATE DATABASE crm_db;
CREATE USER crm_user WITH PASSWORD 'crm_password';
GRANT ALL PRIVILEGES ON DATABASE crm_db TO crm_user;
\connect crm_db;
GRANT ALL ON SCHEMA public TO crm_user;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON TABLES TO crm_user;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON SEQUENCES TO crm_user;
```
Then point `.env` at it (`DBNAME=crm_db`, `DBUSER=crm_user`, `DBPASSWORD=crm_password`,
`DBHOST=localhost`, `DBPORT=5432`) and run `uv run python manage.py migrate`.
You can confirm the role is configured correctly with the RLS management command:
```bash
uv run python manage.py manage_rls --status
uv run python manage.py manage_rls --verify-user
```
## Redis and Celery
Redis needs to be reachable at the URL in `CELERY_BROKER_URL` / `CELERY_RESULT_BACKEND` (both
default to `redis://localhost:6379/0` in `.env.example`). Install it locally through your package
manager, or run the same image the Docker setup uses:
```bash
docker run -p 6379:6379 redis:7-alpine
```
With Redis reachable, start a Celery worker from `backend/`:
```bash
uv run celery -A crm worker --loglevel=INFO
```
`backend/crm/celery.py` also defines a set of periodic tasks (recurring-invoice generation,
overdue-invoice and expired-estimate checks, stale-opportunity and goal-milestone checks, SLA
breach scanning, notification and refresh-token cleanup) on cron-style schedules. To run those on
schedule you also need Celery beat, in a separate terminal:
```bash
uv run celery -A crm beat --loglevel=INFO
```
Neither process is required for the API itself to respond to requests — they only matter for
background and scheduled work.
## Frontend
From `frontend/`:
```bash
cd frontend
pnpm install
```
Copy the example environment file:
```bash
cp .env.example .env
```
At minimum, `PUBLIC_DJANGO_API_URL` should point at your backend (`http://localhost:8000` if
you're following the ports in this guide). Then start the dev server:
```bash
pnpm run dev
```
This serves the SvelteKit app on `http://localhost:5173` (`vite dev` under the hood, per the
`dev` script in `frontend/package.json`).
## Running all three
A full local stack is four long-running processes across (at least) three terminals, all started
from the state above:
```bash
# Terminal 1 — from backend/
uv run python manage.py runserver
# Terminal 2 — from backend/
uv run celery -A crm worker --loglevel=INFO
# Terminal 3 — from frontend/
pnpm run dev
```
Redis and PostgreSQL need to already be running before any of these start (either installed
locally or run as standalone containers, as shown above). Add a fourth terminal for
`uv run celery -A crm beat --loglevel=INFO` from `backend/` if you need scheduled tasks to fire.
Once all of this is up, see [First sign-in](first-sign-in.md) for how to get an authenticated
session, and [Demo data and packs](demo-data.md) for loading sample data.

23
docs/index.md Normal file
View file

@ -0,0 +1,23 @@
# BottleCRM
BottleCRM is a multi-tenant CRM covering leads, accounts, contacts, opportunities, support
tickets (cases), tasks and invoices.
## What BottleCRM is
BottleCRM is MIT licensed and self-hosted: you run it on your own infrastructure, there are no
per-seat limits, and there is no feature paywall.
## How it fits together
Three clients — a SvelteKit web app, a Flutter mobile app, and the REST API — share one Django
REST Framework backend. Tenant isolation is enforced by PostgreSQL Row-Level Security keyed on
the `app.current_org` session variable, not by application code alone: application middleware
sets that variable from the authenticated user's JWT on every request, and the database enforces
it independently of the ORM's own org filters.
## Where to start
- **Operators**, deploying and running BottleCRM: [Docker quick start](getting-started/docker-quick-start.md)
- **Integrators**, building against the REST API: [Conventions](api/conventions.md)
- **Contributors**, working on the BottleCRM codebase: [Development setup](contributing/development-setup.md)

179
docs/integrations/csv-import.md Normal file
View file

@ -0,0 +1,179 @@
# CSV import
## What can be imported
**There are two, unrelated ways to bulk-create records from a CSV, plus one gap.** Contacts and
cases (tickets) share a modern, two-phase preview/commit importer. Leads have a separate, older,
single-step bulk uploader that predates it and works nothing like it. Accounts have neither —
confirmed by grepping the denied concept, not just one filename, across the whole app:
`grep -rniE "\bupload\b|csv_import|import_views|ImportPreview|ImportCommit" --include="*.py"
backend/accounts/` (excluding migrations/tests) returns nothing. If you need to bulk-create
accounts, use `POST /api/accounts/` directly (see [Accounts](../api/accounts.md)) — the
[Python](python-sample.md) / [JavaScript](javascript-sample.md) samples on this site call the
equivalent lead endpoint, but the same obtain-token/list/create pattern applies unchanged.
**The lead uploader:** `POST /api/leads/upload/` (`LeadUploadView`,
`backend/leads/views/lead_interactions.py`) takes `multipart/form-data` with the file under the
field name `leads_file` — not `file` like the importers below. It only requires a `title` column
(`LeadListForm`/`csv_doc_validate`, `backend/leads/forms.py`); every other column is optional and
free-form. The file is decoded as **`iso-8859-1`, not UTF-8** — unlike the contacts/cases
importers below, so a UTF-8 file with multi-byte characters (accented names, curly quotes) won't be
rejected, it will be silently mis-decoded. There's no preview step: a valid upload returns
`{"error": false, "message": "Leads created Successfully"}` immediately and hands the parsed rows
to a Celery task (`create_lead_from_file`, `backend/leads/tasks.py`) that creates the `Lead` rows
in the background — by the time you get the `200`, nothing has necessarily been written yet, and
there's no per-row error report or count. Rows with a blank `title` are dropped before the task
even runs, and the task itself additionally expects every row to have an `email` value it can
match against a basic format check; rows that fail that check are silently skipped, and a file
with no `email` column in it at all fails the whole background job outright, not just the rows
missing one, because the check isn't guarded. A file that has no `title` column, isn't valid CSV,
or where every row is invalid is rejected synchronously with `{"error": true, "errors": {...}}`.
**Permission gating is not the same across the three surfaces, and this is worth knowing before
you rely on it.** `LeadUploadView.permission_classes = (IsAuthenticated, HasOrgContext)` — no
admin check, no `has_sales_access` check. **Any authenticated org member can bulk-create leads**
through this endpoint. The contacts and cases importers below both gate on `_can_import`
(admin or `has_sales_access`) — a deliberately narrower surface than a single `POST` to their
respective list endpoints. The lead uploader has no equivalent restriction.
Both importers below share the same design: a stateless two-phase flow (`parse_and_validate` then
`commit_rows`, `backend/contacts/services/csv_import.py` and `backend/cases/services/csv_import.py`),
a 5 MB / 5,000-row cap, UTF-8-only decoding (with or without a BOM — anything else returns a
`header_error` asking you to re-save the file, rather than silently producing mojibake that then
passes validation), and gating to admins or users with sales access:
```python
def _can_import(profile) -> bool:
if profile is None:
return False
if getattr(profile, "role", None) == "ADMIN":
return True
if getattr(profile, "is_admin", False):
return True
return bool(getattr(profile, "has_sales_access", False))
```
Anyone else gets `403 {"error": true, "message": "Permission denied"}` from either endpoint of
either importer — this mass-create surface is deliberately not open to every org member the way a
single `POST` is.
## Contacts
`POST /api/contacts/import/preview/` and `POST /api/contacts/import/commit/`
(`ContactImportPreviewView` / `ContactImportCommitView`, `backend/contacts/import_views.py`), both
`multipart/form-data` with the file under the field name `file`.
Required headers: `first_name`, `last_name`. Optional: `email`, `phone`, `organization`, `title`,
`department`, `do_not_call`, `linkedin_url`, `address_line`, `city`, `state`, `postcode`,
`country`, `description`, `account_name`, `assigned_emails`, `team_names`, `tags`. Any header not
in that list — including a misspelling — fails the whole file with a `header_error` before any row
is read; there's no partial-header tolerance.
`assigned_emails`, `team_names` and `tags` accept multiple values separated by `;` in one cell.
`account_name`, `assigned_emails` and `team_names` must resolve to an existing account, active
profile or team in your org — an unresolved reference is a row error, not a silent skip or an
auto-create (tags are the one exception: an unrecognized tag name is auto-created at commit).
`country`, if present, must be one of the codes documented on the [Leads](../api/leads.md#fields)
page's `country` field (same `COUNTRIES` list, shared across the codebase).
## Tickets
`POST /api/cases/import/preview/` and `POST /api/cases/import/commit/`
(`CaseImportPreviewView` / `CaseImportCommitView`, `backend/cases/import_views.py`), same
`multipart/form-data` shape.
Required headers: `name`, `status`, `priority`. Optional: `description`, `case_type`,
`account_name`, `contact_emails`, `assigned_emails`, `team_names`, `tags`, `closed_on`.
`status` and `priority` are matched case-insensitively against the same `STATUS_CHOICE` /
`PRIORITY_CHOICE` enums the API uses (`New`, `Assigned`, `Pending`, `Closed`, `Rejected`,
`Duplicate`; `Low`, `Normal`, `High`, `Urgent`) — a value outside those is a row error, not a
free-text fallback. `case_type`, if present, must be one of `Question`, `Incident`, `Problem`.
`closed_on` must be `YYYY-MM-DD`. `contact_emails` (plural, `;`-separated) must resolve to
existing contacts in your org — this importer does not auto-create contacts the way inbound email
does (see [Inbound email](inbound-email.md#how-inbound-email-becomes-a-ticket)).
## Matching
Every reference field — `account_name`, `contact_emails`/`assigned_emails`/`team_names`, and each
importer's own duplicate check — is resolved against your org only. Both importers bulk-prefetch
every distinct reference value in the file once (one query per reference type, not one per row), so
a 5,000-row file with several reference columns runs on the order of ten queries during validation,
not tens of thousands — and, just as importantly, the lookups can't reach across tenants: a CSV
that names another org's account or a team that doesn't exist in your org fails to resolve the same
way a typo would, because the prefetch never looks outside `org=`.
Duplicate detection differs by importer, because "duplicate" means something different for a
person than for a ticket:
- **Contacts**: `email` is a hard error against both the file (another row with the same address)
and the org's existing contacts — the database enforces a per-org, case-insensitive unique
constraint, and the importer mirrors it so the failure surfaces as a row error instead of an
`IntegrityError` at commit. `phone` is a hard error too, normalized with the same digits-only,
last-10-characters comparison the rest of the codebase uses (`common.validators.normalize_phone`),
even though there's no database constraint backing it. A full-name collision (`first_name` +
`last_name`) is **only** an error when the row has neither an email nor a phone to disambiguate
it from an existing same-named contact — two people can legitimately share a name, so the check
only fires when there's nothing else to tell them apart.
- **Tickets**: `name` is a hard error against both the file and the org's existing case names —
there is no email/phone-style disambiguation exception, because a ticket name isn't expected to
double as a person's identity the way a contact's is.
## Response shape
Both endpoints of both importers return the same shape. `preview` never writes to the database —
it re-runs the identical validation `commit` does and returns:
```json
{
"header_error": null,
"valid": [{"row": 2, "first_name": "Ada", "last_name": "Lovelace", "...": "..."}],
"errors": [{"row": 3, "field": "email", "message": "'not-an-email' is not a valid email"}],
"summary": {"total": 2, "valid": 1, "invalid": 1}
}
```
`row` is 1-based against the data rows (the header is not row 1, so the first data row is `row: 1`).
`header_error` is non-null instead of `valid`/`errors` for a whole-file problem — a missing
required header, an unrecognized header, non-UTF-8 content, an empty file, or more than 5,000 data
rows — and when it's set, `valid` and `errors` are always empty; there's no partial parse of a file
that fails at the header stage.
`commit` re-parses and re-validates the file from scratch — it does not trust a client-side
"these rows already passed preview" claim — and refuses to write anything if *any* row is invalid:
```json
{"error": false, "created": 42, "ids": ["<uuid>", "..."]}
```
```json
{
"error": true,
"message": "Fix the invalid rows before importing",
"errors": [{"row": 3, "field": "email", "message": "..."}],
"created": 0
}
```
A `header_error` at commit time returns the same field, `created: 0`, and no `errors` array.
The two importers handle one edge case differently: a conflicting row created by someone else
between your preview and your commit call, most plausibly the same email landing twice from two
concurrent contact imports. The **contacts** importer's `commit_rows` wraps the write in a
`try`/`except IntegrityError` and turns that race into a clean response instead of a `500`:
```json
{
"error": true,
"message": "A contact was created concurrently that conflicts with this import (likely a duplicate email). Re-run preview and try again.",
"detail": "...",
"created": 0
}
```
The **cases (tickets)** importer's `commit_rows` has no equivalent `try`/`except` — because it
doesn't need one the same way: `Case` has no database-level uniqueness constraint on `name` (unlike
`Contact.email`, which the database itself enforces per org), so the same race there wouldn't raise
an `IntegrityError` at all. It would instead create two same-named cases, since the importer's
"duplicate name" check only ever looked at what existed *at validation time*. This is a narrow
window — you'd need two concurrent commits importing the same ticket name at once — but it's worth
knowing the two importers aren't symmetric here.

157
docs/integrations/inbound-email.md Normal file
View file

@ -0,0 +1,157 @@
# Inbound email
BottleCRM can turn email sent to a support address into a case (support ticket), and thread
replies onto the same case automatically. This is the **only** webhook in the product, and it runs
in one direction only: BottleCRM *receives* mail through it. Nothing in this codebase posts a
webhook out to a URL you configure — there is no equivalent "notify my system when a case closes"
hook. If you need BottleCRM to push events elsewhere, you're integrating through the
[MCP server](mcp-server.md) or polling the REST API; nothing here does that for you.
## How inbound email becomes a ticket
The pipeline (`backend/cases/inbound/pipeline.py`, entry point `ingest()`) runs the same four
steps for every message that reaches a configured mailbox:
1. **Spam/noise filtering** (`cases/inbound/spam.py`). A message is dropped — recorded, but no
case touched — if it looks like a bounce, carries `Auto-Submitted: <anything but "no">`,
`Precedence: bulk|list|junk`, `X-Autoreply: yes`, or both `List-Id` and `List-Unsubscribe`
headers (a mailing-list signature; either header alone isn't enough, since transactional mail
sometimes carries `List-Unsubscribe` on its own). A message with no `Message-ID` at all is also
treated as a drop — without one there's nothing to thread or de-duplicate on.
2. **Threading** (`cases/inbound/threading.py`, `find_existing_case`). `In-Reply-To` and every id
in `References` are merged into one de-duplicated candidate list, and a **single** query
matches that whole list against prior messages' `Message-ID`s, ordered by most-recently-received
— so if more than one candidate id matches, the most recently received message wins; it is not
"`In-Reply-To` first, then `References`" as two separate lookups. Only after that does the match
fall through to a case's own thread id (`external_thread_id`, including one inherited from a case
merged into it, via `alt_thread_ids`) and, as a last resort, a `[Case #<8-hex-chars>]` prefix
surviving in the subject line. If the matched case was since merged into another, the reply is
attached to the surviving primary instead.
3. **Contact resolution** (`cases/inbound/contacts.py`). The `From:` address is matched
case-insensitively against existing contacts in the mailbox's org; if none matches, a new
`Contact` is auto-created from the display name (or the email's local part, if there's no
display name) and linked to the case.
4. **Case creation or update.** No match in step 2 creates a new `Case` — subject becomes the name
(truncated to 64 characters), body becomes the description, and **`status` is always `"New"`**
(`pipeline.py`: `status="New"` is hardcoded — the mailbox has no `default_status` field, so
don't go looking for one). `priority` and `case_type` do come from the mailbox's
`default_priority`/`default_case_type`, and if the mailbox has a `default_assignee`, that
profile is added to the new case's `assigned_to`. A match reuses the existing case, attaches the
contact if not already linked, and can reopen a closed case (subject to the org's reopen policy)
if the reply arrives inside the configured reopen window.
Every message — including dropped ones — is recorded as exactly one `EmailMessage` row
(`org` + `message_id` is the idempotency key, so a provider retry never creates a duplicate case).
That gives admins a forensic trail even for the mail that never became a ticket, and it's what
[the mailbox list's per-mailbox counts](#configuring-a-mailbox) are computed from.
## Configuring a mailbox
`InboundMailbox` (`backend/cases/models.py`) is a per-org row managed at
`GET`/`POST /api/cases/mailboxes/` and `GET`/`PUT`/`DELETE /api/cases/mailboxes/{id}/`
(`InboundMailboxListCreateView` / `InboundMailboxDetailView`,
`backend/cases/inbound_views.py`), admin-only for every write (`POST`, `PUT`, `DELETE` all check
`role == "ADMIN"` or `is_admin`; a non-admin gets `403`). Reads are open to any org member.
The model declares four provider choices — `ses`, `mailgun`, `postmark`, `imap` — but **only `ses`
is actually implemented today**. The webhook itself checks this: any mailbox whose `provider` is
not `"ses"` returns `501 Not Implemented` for every notification it receives
(`InboundMailboxWebhookView.post`, `backend/cases/inbound_views.py`). The `imap_host` /
`imap_port` / `imap_username` / `imap_password_enc` fields exist on the model already so that a
future IMAP implementation doesn't need a schema migration, but nothing reads them yet — setting
`provider` to anything but `ses` configures a mailbox that will reject every message sent to it.
Fields worth knowing when creating one:
| Field | Notes |
| --- | --- |
| `address` | The inbound email address. On **create**, `validate_address` pre-checks for a case-insensitive duplicate in your org and returns a clean `400` if one exists (`self.instance is None` gates this — it's explicitly a create-only check). On **update** (`PUT`), that pre-check does not run at all: renaming a mailbox's `address` onto one already used elsewhere in your org skips the serializer check entirely and goes straight to `.save()`, where the database's own `UniqueConstraint` on `(org, address)` rejects it — deterministically, on every such `PUT`, not just under a race, and with no `try`/`except` around the write to turn it into a clean error. Expect an unhandled `IntegrityError` (`500`), not a `400`, if you rename a mailbox onto an address your org already has. |
| `provider` | Must be `ses` for the mailbox to accept mail (see above). |
| `webhook_secret` | Auto-generated (`secrets.token_urlsafe(32)`) on create if you don't supply one. See [Securing the endpoint](#securing-the-endpoint) for what it does and does not do. |
| `topic_arn` | Leave blank on create — it's pinned automatically from the first verified SNS `SubscriptionConfirmation`, not set by hand. See below. |
| `default_priority`, `default_case_type` | Set directly on any new case this mailbox creates; both optional. There is no `default_status` — a new case's `status` is always `"New"`, not configurable per mailbox (see [above](#how-inbound-email-becomes-a-ticket)). |
| `default_assignee_id` | Optional. If set, that profile is added to the new case's `assigned_to` (a many-to-many "add", not a status or ownership field of its own). |
| `is_active` | An inactive mailbox's webhook returns `404` for every notification, same as a mailbox id that doesn't exist — set this to disable a mailbox without deleting its history. |
`GET /api/cases/mailboxes/` adds two per-mailbox fields on top of the serializer above —
`cases_last_30d` and `last_received_at` — plus an org-wide `totals` object
(`{"count", "active", "cases_last_30d"}`), all computed by `_mailbox_analytics`
(`backend/cases/inbound_views.py`) from the `EmailMessage.mailbox` foreign key, not stored on the
mailbox row itself. `cases_last_30d` counts distinct cases *created* in the last 30 days that have
an inbound message through that mailbox (a reply to an older case doesn't count as a new ticket);
`last_received_at` is the newest inbound message's `received_at`, including dropped ones — the
address still received mail even if the pipeline discarded it. Neither field is present on
`GET /api/cases/mailboxes/{id}/`, which returns only the plain serializer.
## The SNS webhook
Point an AWS SES receipt rule (via an SNS topic, "SNS Notification with full content" or the
older JSON-envelope form — the parser handles both) at:
```
POST /api/cases/inbound/<mailbox_id>/
```
This route is deliberately public — `authentication_classes = ()`, `permission_classes =
(AllowAny,)` — because SNS has no way to send your BottleCRM credentials. The `<mailbox_id>` in the
URL is what scopes each notification to one org: the view looks the mailbox up by that id (and
`is_active=True`) before doing anything else, and because the webhook bypasses
`RLSContextMiddleware` entirely (there's no authenticated session to derive an org from), the view
sets the RLS session variable by hand from `mailbox.org_id` before touching any other org-scoped
table.
On a `Notification`, the view acks with:
```json
{"ok": true, "case_id": "<uuid or null>", "dropped": false, "reason": "", "created_case": true}
```
`dropped`/`reason` reflect the spam-filtering outcome from [the pipeline](#how-inbound-email-becomes-a-ticket)
above; the response is still `200` even when the message was dropped, and deliberately so — SES
already accepted the message from the original sender, and returning a `4xx` here would just
trigger pointless provider retries of a message that was correctly classified as noise, not lost.
## Securing the endpoint
Because the endpoint takes no credential, trust is established entirely by verifying the message
itself, in two layers — and it takes both:
1. **SNS signature verification** (`cases/inbound/sns.py`, `verify_sns_message`). The payload's
`Signature` is checked against the certificate at its `SigningCertURL`, which is pinned to the
`sns.<region>.amazonaws.com` host family, fetched with redirects disabled and TLS verified
against the system CA store, and itself checked for a matching Common Name and a currently-valid
validity window before its public key is trusted. This proves the payload was genuinely signed
by *some* SNS topic in *some* AWS account — nothing more.
2. **TopicArn pinning.** A valid SNS signature alone isn't enough, because anyone with an AWS
account can create a topic and have SNS sign messages for it. Each mailbox is pinned to the
exact `TopicArn` it should accept: the pin is captured automatically, trust-on-first-use, from
the first signature-verified `SubscriptionConfirmation` a mailbox receives, and every
`Notification` after that must carry the exact same `TopicArn` or it's rejected. A mailbox with
no pin yet rejects every `Notification` outright (only a `SubscriptionConfirmation` can
establish the pin), and once pinned, a mailbox does not silently re-pin to a different topic —
without this, anyone who learned a mailbox's UUID could point their own SNS topic at it and have
AWS legitimately sign forged mail on their behalf, which — because the pipeline threads replies
onto existing cases — means injecting messages into a live customer conversation, not just
spam tickets.
Both failure modes return the same generic `403` (`"Signature verification failed"`) rather than
distinguishing "bad signature" from "wrong topic" — a caller who gets past the existence check
below can't tell which of the two rejected them, or what the pinned `TopicArn` is. That existence
check is a separate, earlier step: a `mailbox_id` that doesn't match any active mailbox returns
`404` before either check above ever runs, which does let a caller distinguish "this id belongs to
no mailbox" from "this id belongs to a mailbox, but the request failed verification" (a `403`, a
`501` for an unsupported provider, and so on) — a real, if narrow, way to enumerate valid mailbox
ids, notwithstanding the source comment's stated intent not to leak that. Mailbox ids are UUIDs,
not sequential, so this isn't practically brute-forceable, but it's not the airtight non-disclosure
the comment claims either.
**`webhook_secret` is not part of this verification.** It is stored on the mailbox, auto-generated
on create, and settable/rotatable by an admin through the mailbox CRUD endpoints above, but nothing
in `InboundMailboxWebhookView.post` reads or compares it — the two checks above are what actually
gate the endpoint. What `webhook_secret` (and `topic_arn`, held to the same bar) does get is
**stripped from every API response to a non-admin caller**: `InboundMailboxSerializer.to_representation`
pops both fields unless the requesting profile is an admin, so a regular org member who can list
mailboxes never sees either value. Treat both as admin-only secrets when deciding who in your org
should have access to the CRM at all — not because a leaked `webhook_secret` lets someone forge
mail today, but because a leaked `topic_arn` embeds your AWS account id, and because the field is
positioned to matter if a future provider integration does check it.

172
docs/integrations/javascript-sample.md Normal file
View file

@ -0,0 +1,172 @@
# JavaScript sample
## What this does
The same flow as the [Python sample](python-sample.md), in JavaScript: obtain an access token,
list the org's leads, and create one — using only the built-in `fetch`, no npm dependencies. It's
written for Node (tested for compatibility with Node 24), but the `fetch` calls themselves would
run unmodified in any modern browser's console against a CORS-enabled backend.
As with the Python sample, "obtains a token" means reading a personal access token (`bcrm_pat_…`)
from the environment rather than driving an interactive sign-in — see
[Tokens and API keys](../api/tokens-and-api-keys.md#personal-access-tokens) for what a PAT
authenticates as, and the [Python sample](python-sample.md#what-this-does) for how to mint one
locally.
## The script
```javascript
#!/usr/bin/env node
// List an org's leads and create one, using a BottleCRM personal access token.
//
// Usage:
// BCRM_BASE_URL=http://localhost:8000 BCRM_TOKEN=bcrm_pat_... node leads-sample.mjs
//
// Requires Node 18+ for the built-in `fetch`. The flag history, precisely:
// --experimental-fetch was a Node 17.5 flag; Node 18 made fetch available
// with no flag at all (still labeled experimental, with a startup warning);
// Node 21 dropped that warning and made fetch stable. Tested for Node 24.
// No npm dependencies.
//
// See docs/api/tokens-and-api-keys.md for how to mint BCRM_TOKEN, and
// docs/api/leads.md for the shape of the /api/leads/ responses this reads.
const BASE_URL = (process.env.BCRM_BASE_URL || "http://localhost:8000").replace(/\/$/, "");
const TOKEN = process.env.BCRM_TOKEN;
if (!TOKEN) {
console.error(
"Set BCRM_TOKEN to a bcrm_pat_... personal access token " +
"(see docs/api/tokens-and-api-keys.md)."
);
process.exit(1);
}
const headers = {
Authorization: `Bearer ${TOKEN}`,
Accept: "application/json",
"Content-Type": "application/json",
};
// GET /api/leads/ — the response splits leads into open/closed sections
// rather than one flat list; see docs/api/leads.md#list-leads.
async function listLeads() {
const res = await fetch(`${BASE_URL}/api/leads/`, { headers });
const data = await res.json();
if (!res.ok) {
throw new Error(`List failed (${res.status}): ${JSON.stringify(data)}`);
}
const openSection = data.open_leads;
const leads = openSection.open_leads;
console.log(
`${leads.length} open lead(s) on this page (of ${openSection.leads_count} total open):`
);
for (const lead of leads) {
const name = `${lead.first_name ?? ""} ${lead.last_name ?? ""}`.trim();
console.log(
` - ${name || "(no name)"} <${lead.email ?? "no email"}> [${lead.status}]`
);
}
return data;
}
// POST /api/leads/ — a 200 with {"error": false, ...} is success; the
// response does not echo the created record, so we look it up afterward.
async function createLead() {
const email = `sample.lead+${Date.now()}@example.com`;
const res = await fetch(`${BASE_URL}/api/leads/`, {
method: "POST",
headers,
body: JSON.stringify({
first_name: "Sample",
last_name: "Lead",
email,
status: "assigned",
source: "other",
}),
});
const body = await res.json();
if (!res.ok || body.error) {
throw new Error(`Create failed (${res.status}): ${JSON.stringify(body)}`);
}
console.log(`Create response: ${JSON.stringify(body)}`);
return email;
}
// GET /api/leads/?email=... — `email` is a case-insensitive contains filter
// (docs/api/conventions.md#filtering-and-search), fine here since the address
// we just created is unique in this org.
async function findByEmail(email) {
const url = new URL(`${BASE_URL}/api/leads/`);
url.searchParams.set("email", email);
const res = await fetch(url, { headers });
const data = await res.json();
if (!res.ok) {
throw new Error(`Lookup failed (${res.status}): ${JSON.stringify(data)}`);
}
const matches = data.open_leads.open_leads;
if (matches.length) {
console.log(`Found the new lead: id=${matches[0].id}`);
} else {
console.log("New lead not on this page (increase limit/offset to find it).");
}
}
async function main() {
console.log("Listing existing leads...");
await listLeads();
console.log("\nCreating a lead...");
const email = await createLead();
console.log("\nConfirming it was created...");
await findByEmail(email);
}
main().catch((err) => {
console.error(err.message);
process.exit(1);
});
```
## Running it
Save the script as `leads-sample.mjs`. It uses no `import`/`export` and no top-level `await` (the
`await` calls are all inside `async function main()`, invoked with `.catch()` rather than awaited
at the top level), so it runs the same way whether Node treats the file as CommonJS or an ES
module — the `.mjs` extension just removes any ambiguity from a `package.json` in the same
directory. You need a reachable
BottleCRM backend and a personal access token, minted as described in the
[Python sample](python-sample.md#what-this-does):
```bash
BCRM_BASE_URL=http://localhost:8000 \
BCRM_TOKEN=bcrm_pat_... \
node leads-sample.mjs
```
**A note on verification:** like the Python sample, this script was checked by static means —
every URL and field against [Leads](../api/leads.md) and [Conventions](../api/conventions.md), and
the whole file with `node --check` — rather than executed against a live database, to avoid
writing a row into a shared instance this documentation didn't provision. The example output is
illustrative, not a captured run, and is identical in shape to the
[Python sample's](python-sample.md#running-it) since both call the same endpoints:
```text
Listing existing leads...
3 open lead(s) on this page (of 3 total open):
- Jordan Blake <jordan.blake@example.com> [assigned]
- Priya Shah <priya.shah@example.com> [in process]
- (no name) <no email> [assigned]
Creating a lead...
Create response: {"error":false,"message":"Lead Created Successfully"}
Confirming it was created...
Found the new lead: id=3f2a1c9e-....
```
A validation failure comes back as a `400` and the script throws with the error body BottleCRM
returned, per [Errors](../api/errors.md#validation-errors):
```text
Create failed (400): {"error":true,"errors":{"email":["Another lead in this organisation already uses that email address."]}}
```

204
docs/integrations/mcp-server.md Normal file
View file

@ -0,0 +1,204 @@
# MCP server
`bcrm-mcp` is a [Model Context Protocol](https://modelcontextprotocol.io) server that lets an AI
agent (Claude Desktop, Cursor, Codex CLI, Gemini CLI, or any other MCP client) read and write CRM
data. It lives in its own top-level package, `mcp_server/`, next to `backend/` and `frontend/` —
not inside the Django project — and it is a thin HTTP client of the CRM's own REST API: it has no
database connection of its own and does not import Django (`mcp_server/src/bcrm_mcp/client.py`
depends only on `httpx`). Every tool call becomes one authenticated request to `/api/...`, and the
backend enforces RLS, RBAC and field validation exactly as it would for any other caller.
Source and further detail: [`mcp_server/README.md`](https://github.com/django-crm/Django-CRM/blob/master/mcp_server/README.md)
in the repository.
## What it does
The server exposes eight tools an agent can call — search/list, get, create, update, delete, a
generic non-CRUD action runner, a schema-describe helper, and an action-discovery helper — over
eight CRM entities: leads, contacts, accounts, opportunities, tasks, cases, invoices and
solutions. It does not add any capability the REST API doesn't already have; it's a uniform,
agent-friendly surface over the same endpoints documented in [API reference](../api/conventions.md).
## Installing
There are two things you can install, depending on how you want to run it.
**The standalone package**, for running the server as a local subprocess your MCP client launches
(the `stdio` transport, the default):
```bash
cd mcp_server
uv sync
```
`uv sync` creates `mcp_server/.venv/` from `mcp_server/pyproject.toml` + `uv.lock`. The package
requires Python 3.11+ and pulls in `fastmcp`, `httpx` and `pydantic` — nothing from the Django
project.
**The `mcp` extra on the backend**, for mounting the server inside the Django app so it's reachable
over HTTP without a local install at all:
```bash
cd backend
uv sync --extra mcp
```
This is declared in `backend/pyproject.toml` as `[project.optional-dependencies] mcp =
["bcrm-mcp"]`, with `bcrm-mcp` resolved via `[tool.uv.sources]` as an editable path dependency on
`../mcp_server`. It is opt-in specifically so the core backend install stays lean — a plain `uv
sync` in `backend/` never pulls in `fastmcp` or `starlette` at all. If the extra isn't installed,
`backend/crm/asgi.py` catches the resulting `ImportError` and serves Django alone; an incomplete or
skipped MCP install never takes the rest of the site down.
## Hosted at /mcp
`backend/crm/asgi.py` optionally mounts the MCP server at exactly `/mcp` on the same process that
serves the rest of the API — no separate deployment. This only activates when three things are all
true: the `mcp` extra is installed, `BCRM_MCP_ENABLED` is not set to a falsy value (`0`, `false`,
`no` or `off` — anything else, including unset, leaves it enabled), and the app is served under
**ASGI**, not WSGI:
```bash
# from backend/
uv sync --extra mcp
BCRM_BASE_URL=http://127.0.0.1:8000 \
uv run uvicorn crm.asgi:application --host 0.0.0.0 --port 8000
# MCP is now live at http://<host>:8000/mcp
```
`manage.py runserver` uses WSGI, so the mount is inactive under the ordinary local dev server —
this is purely an ASGI-server concern and doesn't change any other local workflow.
`BCRM_BASE_URL` is the loopback address the mounted tools call to reach the CRM API; it defaults
to `http://127.0.0.1:8000`, which is correct when MCP and Django share the same process, as they
do here.
The mount dispatches by path prefix rather than Starlette's `Mount` (which only matches
`/mcp/…` with a trailing segment and would let a bare `/mcp` fall through to Django) — both `/mcp`
and `/mcp/...` are routed to the MCP app, everything else goes to Django as before.
## Authenticating
The server has two transports, and they authenticate differently:
- **`stdio`** (default, used when your MCP client launches `bcrm-mcp` as a local subprocess) — the
whole process acts as one user. It reads a single token from `BCRM_TOKEN` at startup and reuses
one `CrmClient` for every call. `Settings.from_env()` refuses to start without it.
- **`http`** (the `/mcp` mount above, or a standalone server run with `BCRM_TRANSPORT=http`) — a
long-lived, multi-user server. There is no server-side token at all; `Settings.from_env()`
actively rejects a `BCRM_TOKEN` set alongside `BCRM_TRANSPORT=http`, because a shared token would
collapse every caller into one identity. Instead **every request carries its own
`Authorization: Bearer <bcrm_pat_…>` header**, and `ClientResolver.get()`
(`mcp_server/src/bcrm_mcp/server.py`) builds a fresh, uncached `CrmClient` from that header on
every single call — never reused across requests, so one caller's identity can never leak into
the next caller's response.
For the `/mcp` mount specifically, `crm/asgi.py` enforces this at the edge: any request to `/mcp`
or `/mcp/...` with no well-formed bearer token gets a `401` before it reaches the MCP layer at
all — even `initialize` and tool listing are unreachable without a token, not just the tool calls
that touch data. The token's actual validity (not expired, not revoked, owning profile and org
still active) is then re-checked by the backend on every API call the tool makes, the same way any
other PAT-authenticated request is checked — see
[Tokens and API keys](../api/tokens-and-api-keys.md#personal-access-tokens) for how a PAT is
minted, validated and revoked.
In both transports, the token is a normal BottleCRM personal access token
(`bcrm_pat_…`) — there is nothing MCP-specific about it.
## Available tools
| Tool | Kind | Description |
| --- | --- | --- |
| `crm_search` | read-only | List/search records of an entity. Takes `query`, `filters`, `limit`, `offset`. `limit` is capped at **50** regardless of what's requested (`MAX_LIMIT` in `mcp_server/src/bcrm_mcp/tools.py`). |
| `crm_get` | read-only | Fetch one record's full detail by `id`. |
| `crm_create` | write | Create a record from a `data` object, validated server-side exactly as `POST` to the underlying endpoint would be. |
| `crm_update` | write | Partially update a record (PATCH semantics) from a `data` object. **Returns an error for `invoices`** — see below. |
| `crm_delete` | destructive | Delete a record. Refuses to run unless called with `confirm=true`. **Also returns an error for `invoices`** — see below. |
| `crm_action` | write | `POST` a non-CRUD action to `/api/<entity>/<id>/<action>/`. Most of the actions `entities.py` lists don't resolve to a real route today — see below before relying on this. |
| `crm_describe` | read-only | Return an entity's fields, types, enums and which are required, derived from the live OpenAPI schema at `/schema/`. |
| `list_actions` | read-only | Return the allowed non-CRUD actions for every entity, exactly as `entities.py` declares them — see below for which of those are actually reachable. |
CRUD tools take an `entity` argument. `resolve_path` (`mcp_server/src/bcrm_mcp/entities.py`) maps
each to its real API path and the non-CRUD actions `entities.py` declares as allowed for it:
| Entity | API path | Non-CRUD actions declared |
| --- | --- | --- |
| `leads` | `/api/leads/` | `convert`, `add_comment` |
| `contacts` | `/api/contacts/` | `add_comment` |
| `accounts` | `/api/accounts/` | `add_comment` |
| `opportunities` | `/api/opportunities/` | `add_comment` |
| `tasks` | `/api/tasks/` | — |
| `cases` | `/api/cases/` | `add_comment` |
| `invoices` | `/api/invoices/` | `send` |
| `solutions` | `/api/cases/solutions/` | — |
Solutions are served under the cases app, not at a top-level `/api/solutions/` — `crm_search(entity="solutions", ...)` resolves to `/api/cases/solutions/` correctly, but it's worth knowing if you're
cross-referencing the [Endpoint index](../api/endpoint-index.md).
**Most of the actions in that table don't work, and neither does `crm_update`/`crm_delete` on
invoices — this is a real bug in `entities.py`, not a doc gap, and it isn't fixed here.**
`crm_action` builds `POST /api/<entity>/<id>/<action>/` (`tools.py`). Checked against the real
Django URLconfs (`leads/urls.py`, `contacts/urls.py`, `accounts/urls.py`, `opportunity/urls.py`,
`cases/urls.py`, `invoices/api_urls.py`):
- **`convert` on `leads`** — no such route exists anywhere in `leads/urls.py`. Converting a lead
is done by setting its status, not by calling an action:
`crm_update(entity="leads", id=..., data={"status": "converted"})`.
- **`add_comment`** on `leads`, `contacts`, `accounts`, `opportunities` and `cases` — none of these
resolve either. Every one of those apps routes comments at `comment/<id>/`
(e.g. `POST /api/leads/comment/<id>/`), not `<id>/add_comment/`, and Django's `<str:pk>` URL
converter doesn't match a `/` inside the segment, so `<id>/add_comment/` matches nothing. No tool
in this MCP server currently reaches the `comment/<id>/` routes at all.
- **`send` on `invoices`** is the only one that actually resolves —
`POST /api/invoices/<id>/send/` is a real route (`invoices/api_urls.py`).
Calling `crm_action` with any of the non-working entries returns a `CrmError` carrying the
backend's `404` — the MCP layer doesn't pre-validate that the action exists before posting.
Separately, `InvoiceDetailView` (`backend/invoices/api_views.py`) implements only `get` and `put`
— no `patch`, no `delete` — so `crm_update` (which sends `PATCH`) and `crm_delete` (which sends
`DELETE`) both come back as a `CrmError` carrying a `405` for `invoices`, regardless of the
`data`/`confirm` you pass. To change an invoice, use `crm_action(entity="invoices",
action="send", confirm=true)` for sending it, or a full `PUT` via the REST API directly (see
[Invoices and estimates](../api/invoices.md)) for anything else — there is no MCP tool that
partially updates or deletes an invoice today.
## Permissions
**The agent is exactly as privileged as the PAT it's given, no more and no less.** `PATAuthentication`
resolves the token to the owning `Profile` and sets `request.profile`/`request.org` from it
(`backend/common/pat_auth.py`) — the same code path any other PAT-authenticated request goes
through. There is no separate MCP identity, no elevated service account, and no way for the MCP
layer itself to grant a permission the token's owner doesn't already have. A PAT does carry a
`scopes` field, but nothing in the request-handling code enforces it yet (see
[Tokens and API keys](../api/tokens-and-api-keys.md#personal-access-tokens)) — so today there is no
way to hand an agent a narrower token than its owner's own full role and org access. Treat every
PAT you configure into an MCP client as equivalent to handing that agent your own login for as
long as the token is valid.
A few narrower guards live in the MCP layer itself, on top of (not instead of) backend
enforcement:
- `crm_search` caps `limit` at 50 no matter what the agent asks for, so a single call can't pull an
unbounded result set.
- `crm_delete` refuses to run without `confirm=true`.
- `crm_action` refuses an outward-facing action — currently just `send` (emailing a customer an
invoice) — without `confirm=true` too, mirroring the delete guard so a model can't trigger a
real-world side effect off a misread instruction.
Beyond that, the MCP server does no authorization of its own: it forwards each tool call to the
CRM API verbatim and surfaces whatever the API returns, success or error (`CrmClient._request`
raises `CrmError` whenever `resp.status_code >= 400`, carrying the DRF error body — a `3xx`
response isn't treated as an error by this check and instead falls through to `resp.json()`,
which would raise its own decode error if that response body isn't JSON).
RLS, role checks and per-object ownership rules are all enforced exactly once, in the backend, the
same as for the web app or the mobile app.
To cut off an agent immediately, revoke its PAT. There are two ways to do that, and which one
applies depends on your role. An admin can revoke any token in the org from the CRM at
**Settings → API tokens** (`/settings/api-tokens`); that page's revoke action calls the
admin-only `DELETE /api/org/tokens/{id}/`, and the whole page is gated on that same admin-only
oversight list (`GET /api/org/tokens/`) — a non-admin who opens it sees "Admins only", not a token
list, and can't reach any revoke control there at all. Any role can instead call
`DELETE /api/profile/tokens/{id}/` directly against their **own** token — a separate, self-service
endpoint that no UI on this settings page currently calls — which is the option to use if the PAT
you need to revoke is yours and you aren't an admin.

177
docs/integrations/python-sample.md Normal file
View file

@ -0,0 +1,177 @@
# Python sample
## What this does
A complete, runnable script that talks to the BottleCRM REST API from plain Python: it obtains an
access token, lists the org's leads, and creates one.
"Obtains a token" here means reading a **personal access token** (`bcrm_pat_…`) from the
environment, rather than driving one of the interactive sign-in flows on
[Authentication](../api/authentication.md). A PAT is what BottleCRM issues specifically so a script
doesn't need a browser, an OAuth client secret, or a magic-link email round trip just to make API
calls — see [Tokens and API keys](../api/tokens-and-api-keys.md#personal-access-tokens) for how it
authenticates and what it inherits. There are two ways to mint one, and which applies depends on
your role: an **admin** can do it from the CRM's **Settings → API tokens** page
(`/settings/api-tokens`) — that page is admin-only (it lists every token in the org, not just your
own), so a non-admin who opens it sees "Admins only" and never reaches the create form. **Any
role** can instead call `POST /api/profile/tokens/` directly — the same self-service endpoint the
settings page's own create form posts to — which is the option to use if you aren't an admin, or
you just want a token without the UI. For a local backend you're running yourself, the fastest way
is the Django shell:
```bash
cd backend
uv run python manage.py shell -c "
from common.models import Profile, PersonalAccessToken
p = Profile.objects.filter(role='ADMIN', is_active=True).first()
print(PersonalAccessToken.generate(p, 'python-sample')[0])
"
```
This prints the raw `bcrm_pat_…` token once — copy it, there's no way to retrieve it again later.
## The script
```python
#!/usr/bin/env python3
"""List an org's leads and create one, using a BottleCRM personal access token.
Usage:
BCRM_BASE_URL=http://localhost:8000 BCRM_TOKEN=bcrm_pat_... python leads_sample.py
See docs/api/tokens-and-api-keys.md for how to mint BCRM_TOKEN, and
docs/api/leads.md for the shape of the /api/leads/ responses this script reads.
"""
import os
import sys
import time
import requests
BASE_URL = os.environ.get("BCRM_BASE_URL", "http://localhost:8000").rstrip("/")
TOKEN = os.environ.get("BCRM_TOKEN")
if not TOKEN:
sys.exit(
"Set BCRM_TOKEN to a bcrm_pat_... personal access token "
"(see docs/api/tokens-and-api-keys.md)."
)
session = requests.Session()
session.headers.update(
{
"Authorization": f"Bearer {TOKEN}",
"Accept": "application/json",
}
)
def list_leads():
"""GET /api/leads/ — the response splits leads into open/closed sections
rather than one flat list; see docs/api/leads.md#list-leads."""
resp = session.get(f"{BASE_URL}/api/leads/")
resp.raise_for_status()
data = resp.json()
open_section = data["open_leads"]
leads = open_section["open_leads"]
print(
f"{len(leads)} open lead(s) on this page "
f"(of {open_section['leads_count']} total open):"
)
for lead in leads:
name = f"{lead.get('first_name') or ''} {lead.get('last_name') or ''}".strip()
print(
f" - {name or '(no name)'} <{lead.get('email') or 'no email'}> "
f"[{lead.get('status')}]"
)
return data
def create_lead():
"""POST /api/leads/ — a 200 with {"error": false, ...} is success; the
response does not echo the created record, so we look it up afterward."""
email = f"sample.lead+{int(time.time())}@example.com"
payload = {
"first_name": "Sample",
"last_name": "Lead",
"email": email,
"status": "assigned",
"source": "other",
}
resp = session.post(f"{BASE_URL}/api/leads/", json=payload)
body = resp.json()
if resp.status_code >= 400 or body.get("error"):
sys.exit(f"Create failed ({resp.status_code}): {body}")
print(f"Create response: {body}")
return email
def find_by_email(email):
"""GET /api/leads/?email=... — `email` is a case-insensitive contains
filter (docs/api/conventions.md#filtering-and-search), which is fine here
since the address we just created is unique in this org."""
resp = session.get(f"{BASE_URL}/api/leads/", params={"email": email})
resp.raise_for_status()
matches = resp.json()["open_leads"]["open_leads"]
if matches:
print(f"Found the new lead: id={matches[0]['id']}")
else:
print("New lead not on this page (increase limit/offset to find it).")
if __name__ == "__main__":
print("Listing existing leads...")
list_leads()
print("\nCreating a lead...")
new_email = create_lead()
print("\nConfirming it was created...")
find_by_email(new_email)
```
## Running it
Save the script as `leads_sample.py`. You need a reachable BottleCRM backend (local or
self-hosted) and a personal access token for a profile in the org you want to query, minted as
described [above](#what-this-does).
`requests` isn't a dependency of this documentation site or of the CRM's own frontend — the
simplest way to run the script without installing anything into a project is `uv run` with an
inline dependency:
```bash
BCRM_BASE_URL=http://localhost:8000 \
BCRM_TOKEN=bcrm_pat_... \
uv run --with requests python leads_sample.py
```
Or, with a regular virtualenv: `pip install requests` and then
`python leads_sample.py` with the same two environment variables set.
**A note on verification:** this script was checked by static means — every URL and field against
[Leads](../api/leads.md) and [Conventions](../api/conventions.md), and the whole script for valid
Python syntax — rather than executed against a live database, to avoid writing a row into a shared
instance this documentation didn't provision. The example output below is illustrative, not a
captured run:
```text
Listing existing leads...
3 open lead(s) on this page (of 3 total open):
- Jordan Blake <jordan.blake@example.com> [assigned]
- Priya Shah <priya.shah@example.com> [in process]
- (no name) <no email> [assigned]
Creating a lead...
Create response: {'error': False, 'message': 'Lead Created Successfully'}
Confirming it was created...
Found the new lead: id=3f2a1c9e-....
```
A validation failure — for example, running the script twice in the same second so the generated
email collides — comes back as a `400` and the script exits with the error body BottleCRM returned,
per [Errors](../api/errors.md#validation-errors):
```text
Create failed (400): {'error': True, 'errors': {'email': ['Another lead in this organisation already uses that email address.']}}
```

109
docs/mobile/build-and-configure.md Normal file
View file

@ -0,0 +1,109 @@
# Build and configure
This page covers getting the Flutter source running from a clone — dependencies, local runs, and
the test/lint commands. It does not cover pointing the app at your own backend or producing a
signed release build; see [Connect to a self-hosted backend](connect-to-self-hosted.md) and
[Release builds](release-builds.md) for those.
## Prerequisites
`mobile/pubspec.yaml` declares:
```yaml
environment:
sdk: ^3.8.1
```
This is the only entry under `environment:` — there is no `flutter:` key. In a Flutter
`pubspec.yaml`, the `sdk:` constraint under `environment:` pins the **Dart** SDK, not the Flutter
SDK; a `flutter: ">=x.y.z"` line would pin Flutter separately, and this project doesn't have one.
So the project itself declares no explicit Flutter version floor — `^3.8.1` only says "Dart SDK
3.8.1 or newer, below 4.0.0."
That said, the checked-in `mobile/pubspec.lock` records the SDK constraints actually resolved
against the full dependency graph, in its trailing `sdks:` block:
```yaml
sdks:
dart: ">=3.12.0 <4.0.0"
flutter: ">=3.44.0"
```
This is narrower than the project's own `^3.8.1` floor because several dependencies (for example
`flutter_riverpod: ^3.3.1`, `go_router: ^17.2.3`, `google_fonts: ^8.1.0`) have their own, higher
SDK requirements, and `pub` resolves to the intersection of all of them. In practice, **Flutter
3.44.0 or newer** (which is what ships Dart 3.12+) is the version that will actually resolve this
project's dependencies — that number comes from the lockfile, not from a guess at which Flutter
release maps to which Dart version. There is no `.fvmrc`, and neither of the two GitHub Actions
workflows in this repository (`.github/workflows/tests.yml`, `codeql-analysis.yml`) mentions
Flutter, Dart, or `mobile/` at all, so there's no CI-enforced version either — the lockfile is the
only pin that exists.
(`mobile/README.md` states "Flutter SDK: 3.8.1 or later" — the same Dart-constraint-copied-as-a-
Flutter-version error. Use the lockfile numbers above instead.)
Beyond the SDK itself:
- **Android builds** need the Android SDK (via Android Studio or the standalone command-line
tools). `mobile/android/app/build.gradle.kts` sets `compileSdk = flutter.compileSdkVersion` and
`targetSdk = flutter.targetSdkVersion` — both come from whatever your installed Flutter SDK
defines as current, not from a number hardcoded in this project. It does pin two things itself:
`ndkVersion = "28.2.13676358"` (the comment notes this was "bumped to satisfy jni plugin
requirement") and Java/Kotlin compatibility at **17** (`compileOptions.sourceCompatibility` /
`targetCompatibility = JavaVersion.VERSION_17`, mirrored in the `kotlin { compilerOptions {
jvmTarget } }` block) — you need a JDK 17-compatible toolchain available to Gradle.
- **iOS builds** need Xcode and CocoaPods, and only work on macOS. `mobile/README.md` states "iOS:
iOS 11.0+", but the actual deployment target in `mobile/ios/Runner.xcodeproj/project.pbxproj` is
`IPHONEOS_DEPLOYMENT_TARGET = 12.0`, set identically across all three build configurations
(Debug, Release, Profile) — 12.0 is the real floor.
## Getting dependencies
```bash
cd mobile
flutter pub get
```
This resolves and downloads the packages declared in `pubspec.yaml` into your local pub cache and
writes/updates `pubspec.lock` (already checked in — `flutter pub get` should reproduce it exactly
if nothing has changed upstream). Android's Firebase/Crashlytics integration
(`firebase_core`, `firebase_crashlytics`) is wired up for Android only — see
[Connect to a self-hosted backend](connect-to-self-hosted.md#google-sign-in-configuration) for
what that does and doesn't require from you.
## Running against a local backend
`flutter run` starts the app in debug mode against whichever device or emulator is attached
(`flutter devices` lists them). **Before you do this, read [Connect to a self-hosted
backend](connect-to-self-hosted.md) first** — the backend URL the debug build talks to is a
compile-time constant in `lib/config/api_config.dart`, and as shipped it does **not** point at
`localhost` or any backend you're likely to be running yourself. Running `flutter run` without
changing that constant first will build successfully and simply talk to a stranger's backend.
Once `lib/config/api_config.dart` points at your own backend:
```bash
flutter run
```
Press `r` for hot reload, `R` for hot restart, `q` to quit. If you change the URL constant while
the app is already running, restart the run (`R`, or stop and re-run) rather than relying on hot
reload — see the compile-time-constant caveat on the linked page.
## Tests and analysis
```bash
flutter test # run the test suite
flutter analyze --no-fatal-infos # static analysis (analysis_options.yaml + flutter_lints)
dart format . # format all Dart source
```
`mobile/README.md` describes a `test/unit/`, `test/widget/`, `test/integration/` structure. As of
this writing the actual `test/` tree holds one file,
`test/screens/deals/deals_list_screen_test.dart`, a widget test for the deals list screen — the
three-way split described in the README doesn't exist yet. `flutter test` runs whatever is there
regardless of directory layout, so the command itself is accurate even though the described
structure isn't.
`flutter analyze --no-fatal-infos` runs against the rules in `analysis_options.yaml`, which
includes `package:flutter_lints/flutter.yaml` with no project-specific rules added or disabled.

221
docs/mobile/connect-to-self-hosted.md Normal file
View file

@ -0,0 +1,221 @@
# Connect to a self-hosted backend
Read this page before you run or build the mobile app against anything other than
`api.bottlecrm.io`. The short version: the backend URL is baked into the binary at compile time,
there is no way to point a build at a different server without editing source and rebuilding, and
the two URLs shipped in source today aren't `localhost` or any generic placeholder — they're a
specific developer's tunnel and BottleCRM's own production API.
## The compile-time base URL
`mobile/lib/config/api_config.dart` (lines 12–19):
```dart
/// Development API URL
static const String _developmentUrl = 'https://msi-8000.rcdev.in';
/// Production API URL
static const String _productionUrl = 'https://api.bottlecrm.io';
/// Get the current base URL based on build mode
static String get baseUrl => kDebugMode ? _developmentUrl : _productionUrl;
```
Both URLs are `static const` Dart strings, and the choice between them is `kDebugMode` — a
compile-time flag set by which `flutter run`/`flutter build` mode you use, not anything read at
runtime. There is no `--dart-define`, no `String.fromEnvironment` anywhere in `lib/`, and no
runtime settings screen: grepping `lib/` for `SharedPreferences` turns up exactly four files, and
none of them touch the base URL — `auth_service.dart` (auth tokens/user data), list-view
filter/sort state in `screens/tickets/tickets_list_screen.dart` and
`screens/deals/deals_list_screen.dart`, and `providers/tickets_provider.dart`, which only defines
the JSON-serialization helper (`TicketListFilters.toJson()`) that `tickets_list_screen.dart`
persists — it doesn't call the `SharedPreferences` API itself. Whatever `baseUrl` resolves to when
the app is *compiled* is what it talks to for the life of that install.
The practical consequences:
- **A store-distributed release build can only ever talk to `https://api.bottlecrm.io`.** If
you're self-hosting, a generic APK/AAB/IPA built from this source with the constants unchanged
will authenticate against and read/write data on BottleCRM's own hosted API, not yours. There is
no in-app way to redirect it.
- **A debug build (`flutter run`) does not point at `localhost` either.** `_developmentUrl` is
`https://msi-8000.rcdev.in` — an individual developer's personal tunnel/host, not a generic
loopback address, and not one you control. If you want to run the app against a backend on your
own machine, you have to edit this constant regardless of whether you're in debug or release
mode.
- **This means self-hosting the mobile app is source-edit-and-rebuild, not configure-and-run.**
There's no middle ground between "use BottleCRM's hosted API" and "maintain your own fork of
this one file." If that's a blocker for your deployment, treat it as a known limitation rather
than something you missed a flag for.
## Pointing at your backend
1. Edit `mobile/lib/config/api_config.dart` and change `_developmentUrl` and/or `_productionUrl`
to your backend's URL — which one(s) you need depends on which build modes you'll use. If you
don't need a separate dev/prod split, it's simplest to set both to the same value.
2. Rebuild. Because these are compile-time constants, don't rely on hot reload to pick up the
change reliably — stop and re-run (`flutter run`) for a debug build, or do a full `flutter build
apk --release` / `appbundle --release` / iOS build (see [Release builds](release-builds.md)) for
a release one.
A few things to get right about the URL itself:
- **Scheme and reachability depend on where the app runs, not just what your backend serves.**
- An **Android emulator** does not see the host machine as `localhost` — from inside the
emulator, `localhost`/`127.0.0.1` refers to the emulator itself. The documented address for
the host loopback interface is `10.0.2.2`. If your backend is `docker compose up` on your dev
machine (see [Docker](../self-hosting/docker.md), which binds `0.0.0.0:8000`), point the
emulator build at `http://10.0.2.2:8000`, not `http://localhost:8000`.
- An **iOS Simulator** runs natively on the Mac (it isn't a separate virtual network namespace
the way the Android emulator is), so `http://localhost:8000` from inside the simulator does
reach a backend bound on the host Mac.
- A **physical device** (Android or iOS) can't reach "the developer machine" via `localhost` or
`10.0.2.2` at all — it needs your machine's LAN IP address, both devices on the same network,
and no firewall blocking the port.
- **Both platforms default to blocking plain HTTP, and nothing in this project opts back in — but
whether that default actually reaches this app's traffic isn't something this repository can
settle.** The two policies, and what's (not) configured for each:
- **Android** disables cleartext traffic by default for apps whose `targetSdkVersion` is 28 or
higher — that's keyed on `targetSdk`, not `minSdk`. `android/app/build.gradle.kts` pins
`minSdk = 28` (line 40), but `targetSdk = flutter.targetSdkVersion` on the next line is *not*
pinned in this project — it tracks whatever Flutter SDK you have installed (see
[Release builds](release-builds.md)). Either way, there's no `android:usesCleartextTraffic`
attribute and no `android:networkSecurityConfig` in `AndroidManifest.xml`, and no
`network_security_config.xml` resource anywhere under `android/app/src/main/res/` (checked
directly) — nothing here opts back into cleartext.
- **iOS** has the equivalent default: App Transport Security blocks plain HTTP unless the app's
`Info.plist` carries an `NSAppTransportSecurity` exception, and `ios/Runner/Info.plist` has
none.
- On both platforms, though, that default is normally enforced by the platform's own HTTP stack
— Android's `NetworkSecurityPolicy`, consulted by `HttpURLConnection`/OkHttp/WebView; iOS's
App Transport Security, enforced inside `NSURLSession`/CFNetwork. This app's requests don't
obviously go through either: `api_service.dart` builds its client from `package:http`'s
default `Client()` (`final http.Client _client = http.Client();`), which on a non-web platform
resolves to `IOClient`, a thin wrapper around `dart:io`'s own `HttpClient` (`http` package
source, `lib/src/client.dart` / `lib/src/io_client.dart`) — not `cronet_http` or
`cupertino_http`, neither of which is a dependency here. Whether `dart:io`'s socket layer on
either platform consults the OS's cleartext/ATS policy at all isn't something reading source
can settle; it would take an actual build tested against a plain-HTTP host, which is out of
scope for this page.
Treat HTTPS as required regardless of how that ambiguity resolves in practice: this connection
carries JWTs and customer data (see [above](#the-compile-time-base-url)), and it should be
encrypted on its own merits, not because a platform default happens to force it. If you do need
to test against the plain-HTTP [Docker](../self-hosting/docker.md) stack directly, try it — if
the connection fails in a way that looks like a cleartext/ATS block, the Android-side escape
hatch is adding your own `android:usesCleartextTraffic="true"` (or a scoped
`network_security_config.xml`) to `AndroidManifest.xml`; the durable fix on either platform is
putting a TLS-terminating [reverse proxy](../self-hosting/production-deploy.md#reverse-proxy) in
front of your backend, since nothing in the backend itself terminates TLS either.
## Google Sign-In configuration
[Magic-link email sign-in](../getting-started/first-sign-in.md#magic-links) needs no
configuration on either end and works against any backend. Google Sign-In needs configuration on
**both** the backend and the mobile app, and the two have to agree with each other — this is more
involved than the [web app's Google OAuth setup](../self-hosting/google-oauth.md), because the
mobile flow is [ID-token-based, not
redirect-based](../self-hosting/google-oauth.md#mobile-id-tokens).
**Backend side** (prerequisite, covered in full on [Google
OAuth](../self-hosting/google-oauth.md)): `GOOGLE_CLIENT_ID` must be set to a real Google OAuth
Web client ID, because `GoogleIdTokenView` verifies every ID token's audience against exactly that
value (`google.oauth2.id_token.verify_oauth2_token(..., settings.GOOGLE_CLIENT_ID)`). Leave it
blank and Google Sign-In stays off — magic links are the only path in.
**Android side** — three things, all tied to BottleCRM's own Google Cloud project as shipped:
1. `mobile/lib/services/auth_service.dart` (lines 70–73) hardcodes a **Web** OAuth client ID as
`serverClientId`:
```dart
await _googleSignIn.initialize(
serverClientId:
'1072513761792-p59rct7b1c3go7l58e51r3geuqff2tfl.apps.googleusercontent.com',
);
```
The code comment identifies this as "the audience the Django backend's `GOOGLE_CLIENT_ID`
verifies against." For your self-hosted backend to accept tokens from your own mobile build,
this constant must be a Web client ID from **your own** Google Cloud project, and your
backend's `GOOGLE_CLIENT_ID` must be that same value — reusing BottleCRM's own client ID here
will not work against your backend, because your `GOOGLE_CLIENT_ID` won't match it.
2. Android sign-in additionally needs an **Android**-type OAuth client (separate from the Web
client above) registered in Google Cloud, tied to your app's `applicationId` and the SHA-1
fingerprint of whatever certificate signs the build. Get the fingerprint with `cd android &&
./gradlew signingReport` — this needs `android/local.properties` to already exist with a
`flutter.sdk` entry (`settings.gradle.kts` `require`s it and fails with `"flutter.sdk not set in
local.properties"` otherwise); that file is gitignored (`android/.gitignore:6`) and is written
by Flutter's own Gradle-invoking commands (`flutter run`, `flutter build ...`), not by `flutter
pub get` alone, so make sure you've run one of those at least once first. This is also why
`build.gradle.kts`'s `debug` build type signs with the release key when `key.properties` exists
(see [Release builds → Signing](release-builds.md#signing)) rather than the machine-local
Android Gradle Plugin debug keystore — so a locally installed debug build's SHA-1 matches what
you registered.
3. Firebase/Crashlytics config (`android/app/google-services.json`) is required for the Gradle
build to succeed at all once the `com.google.gms.google-services` plugin is applied (it is,
unconditionally, in `android/app/build.gradle.kts`) — only a placeholder
`google-services.json.template` is committed; you supply the real file from your own
Firebase/Google Cloud project. See [Release builds](release-builds.md) for what that plugin
does if the file is missing.
**iOS side — not configured in this repository at all as shipped**, independent of anything you
change on Android. The native SDK the `google_sign_in_ios` plugin wraps (pinned at `6.3.0` in
`pubspec.lock`) can get its client ID from three places, checked in this order by the plugin's own
`configureWithParameters:` handler (`FLTGoogleSignInPlugin.m:180-191`):
1. a runtime `clientId:` argument passed to `GoogleSignIn.instance.initialize(...)`;
2. a bundled `GoogleService-Info.plist`'s `CLIENT_ID` key;
3. `Info.plist`'s own `GIDClientID` key, which `GIDSignIn` reads automatically as its default when
neither of the above is set — the plugin's own source comment calls this "the recommended
configuration method."
All three are absent here: `auth_service.dart`'s `initialize()` call passes only `serverClientId`,
never `clientId`; `ios/Runner/GoogleService-Info.plist` doesn't exist (gitignored, and there's no
committed template the way Android has one); and grepping the whole `ios/` tree for `GIDClientID`,
`CFBundleURLTypes`, and `REVERSED_CLIENT_ID` returns nothing.
That doesn't raise an error at startup, though. When all three sources are absent, the plugin's
internal `configurationWithClientIdentifier:...` returns `nil`, and the handler simply skips
assigning a new configuration (`if (configuration) { self.signIn.configuration = configuration; }`)
rather than failing — `GIDSignIn`'s configuration is just left unset. `auth_service.dart`'s
`initialize()` call is wrapped in a `try`/`catch` that only `debugPrint`s on error regardless, so
the app starts up looking normal on iOS either way. The gap only becomes visible when a user
actually taps "Sign in with Google" and there's no configuration to sign in with.
**The least invasive fix is source #3 above — the `Info.plist` route `GIDSignIn` itself
recommends — since it needs no Dart code changes:**
```xml
<!-- ios/Runner/Info.plist -->
<key>GIDClientID</key>
<string>YOUR-IOS-CLIENT-ID.apps.googleusercontent.com</string>
```
plus the URL-scheme callback handler, which the `google_sign_in_ios` package's own README says is
required regardless of which of the three routes you use for the client ID itself:
```xml
<key>CFBundleURLTypes</key>
<array>
<dict>
<key>CFBundleTypeRole</key>
<string>Editor</string>
<key>CFBundleURLSchemes</key>
<array>
<string>com.googleusercontent.apps.YOUR-REVERSED-CLIENT-ID</string>
</array>
</dict>
</array>
```
Both values come from an iOS-type OAuth client in your Google Cloud project — a
`GoogleService-Info.plist` for that project has them under `CLIENT_ID` and `REVERSED_CLIENT_ID`
respectively, or you can copy them from the Cloud Console directly without ever downloading that
file. The alternative is passing `clientId:` (and optionally `serverClientId:`) as arguments to
`_googleSignIn.initialize(...)` in `auth_service.dart` instead of setting `GIDClientID` (source #1
instead of #3) — but the `CFBundleURLTypes` entry above is still required either way.
This is separate from Firebase Crashlytics, which is deliberately Android-only in this codebase —
`main.dart` explicitly guards Firebase initialization behind `defaultTargetPlatform ==
TargetPlatform.android` with a comment noting iOS has no `GoogleService-Info.plist` "yet." Don't
read the Crashlytics gap as evidence the Google Sign-In gap is equally intentional; the two are
separate config surfaces that happen to share the same missing file.

163
docs/mobile/release-builds.md Normal file
View file

@ -0,0 +1,163 @@
# Release builds
This page covers producing an installable Android/iOS artifact and what signing configuration
already exists in the repository versus what you have to create yourself. It assumes you've
already pointed the app at your own backend — see [Connect to a self-hosted
backend](connect-to-self-hosted.md), since building a release binary bakes in whatever
`lib/config/api_config.dart` says at build time, same as a debug build.
Both platforms read app identity and version from a few shared sources: `applicationId`
`io.bottlecrm` (Android, `android/app/build.gradle.kts`) and bundle identifier `io.bottlecrm` (iOS,
`ios/Runner.xcodeproj/project.pbxproj`) are the same string on both platforms; `pubspec.yaml`'s
`version: 1.2.0+11` supplies both the build name (`1.2.0`) and build number (`11`) via
`flutter.versionName`/`flutter.versionCode` on Android and `$(FLUTTER_BUILD_NAME)`/
`$(FLUTTER_BUILD_NUMBER)` on iOS. `io.bottlecrm` is BottleCRM's own identifier — it's also the
package ID of BottleCRM's own Play Store listing referenced in `mobile/README.md`. If you're
distributing your own build through your own Play Console/App Store Connect listing, you'll want
your own identifier here, which has knock-on effects on the Google Sign-In client registrations
covered on [Connect to a self-hosted backend](connect-to-self-hosted.md#google-sign-in-configuration)
(the Android OAuth client and `google-services.json`'s `package_name` are both tied to whatever
`applicationId` you ship). For a sideloaded APK you build and install yourself, changing it isn't
necessary.
## Android
```bash
flutter build apk --release # single APK, build/app/outputs/flutter-apk/app-release.apk
flutter build appbundle --release # Play Store .aab, build/app/outputs/bundle/release/
```
Real minimum/target/compile SDK levels, from `android/app/build.gradle.kts`:
- `minSdk = 28` (Android 9) — set directly in this file, with the comment `// Android 9 (API 28)`.
`mobile/README.md`'s "Android: API level 31+ (Android 12+)" and `pubspec.yaml`'s
`flutter_launcher_icons.min_sdk_android: 31` (which only controls whether that tool generates
adaptive icons, not the app's own floor) both say something different from the actual Gradle
config — 28 is what the build enforces.
- `compileSdk = flutter.compileSdkVersion`, `targetSdk = flutter.targetSdkVersion` — both come
from your installed Flutter SDK's defaults, not a number pinned in this project.
The build also requires `ndkVersion = "28.2.13676358"` (pinned in the same file, "Bumped to
satisfy jni plugin requirement") and a JDK 17-compatible toolchain (`compileOptions` and the
`kotlin { compilerOptions { jvmTarget } }` block both target Java/Kotlin 17) — see [Build and
configure](build-and-configure.md#prerequisites) for the rest of the local environment needs.
`android/app/build.gradle.kts` applies the `com.google.gms.google-services` and
`com.google.firebase.crashlytics` Gradle plugins unconditionally (classpath versions pinned in
`android/build.gradle.kts`: `com.google.gms:google-services:4.5.0`,
`com.google.firebase:firebase-crashlytics-gradle:3.0.7`). The google-services plugin's documented
behavior is to fail the build outright if `android/app/google-services.json` isn't present. This
repository ships only `android/app/google-services.json.template` — the real file is gitignored
(`**/google-services.json` in `mobile/.gitignore`) and not present in a fresh clone. You need your
own `google-services.json` from a Firebase/Google Cloud project before `flutter build apk` or
`flutter build appbundle` will get past that plugin — see [Connect to a self-hosted
backend](connect-to-self-hosted.md#google-sign-in-configuration) for what that file is for beyond
just satisfying the build.
## iOS
macOS and Xcode only.
```bash
flutter build ios --release # produces build/ios/iphoneos/Runner.app; needs code signing
flutter build ios --release --no-codesign # same, skipping signing (e.g. building without a team configured yet)
```
For an archive ready to upload to App Store Connect, `flutter build ipa --release` is the
Flutter-managed equivalent of `appbundle` for iOS (it drives `xcodebuild -archive` and an export
step, and needs valid signing to succeed — see below).
The real deployment target is `IPHONEOS_DEPLOYMENT_TARGET = 12.0`, set identically in all three
build configurations in `ios/Runner.xcodeproj/project.pbxproj` (Debug, Release, Profile) —
`mobile/README.md`'s "iOS: iOS 11.0+" doesn't match the project file.
Firebase/Crashlytics is deliberately not wired up for iOS in this codebase (`main.dart` skips
`Firebase.initializeApp` outside `TargetPlatform.android`, with a comment noting there's no iOS
`GoogleService-Info.plist` yet), so there's no equivalent of the Android google-services plugin
gate blocking an iOS build — a plain `flutter build ios --release --no-codesign` will get through
without any Firebase config. Getting past code signing is the separate concern covered next.
## Signing
**Android.** `android/app/build.gradle.kts` loads a `key.properties` file from the Android project
root, if one exists:
```kotlin
val keystoreProperties = Properties()
val keystorePropertiesFile = rootProject.file("key.properties")
if (keystorePropertiesFile.exists()) {
keystoreProperties.load(FileInputStream(keystorePropertiesFile))
}
```
It reads four properties from that file — `keyAlias`, `keyPassword`, `storeFile`,
`storePassword` — to build a `release` signing config, and applies that to the `release` build
type. **`key.properties` doesn't ship in this repository** — it's listed in
`android/.gitignore` and there's no committed template for it the way there is for
`google-services.json`. Critically, look at what happens to the `release` build type when the
file is absent:
```kotlin
buildTypes {
release {
signingConfig = if (keystorePropertiesFile.exists())
signingConfigs.getByName("release")
else
signingConfigs.getByName("debug")
}
...
}
```
**On a fresh clone, with no `key.properties` created, `flutter build apk --release` and `flutter
build appbundle --release` fall back to the Android Gradle Plugin's built-in `debug` signing
config** — not a Flutter-provided key, and not one shared across installations. That config is
backed by a `debug.keystore` the Android SDK tooling generates itself, per machine, the first time
one is needed (conventionally at `~/.android/debug.keystore`, with a fixed alias/password pair
`androiddebugkey`/`android`) — Flutter has no involvement in it beyond invoking Gradle, and two
different machines will generate two different debug keystores. That has a concrete consequence
beyond "not a real release key": a debug-signed release build from your laptop and one from your
CI machine carry different signatures, and Android treats differently-signed builds as unrelated
apps for update purposes — one can't silently update an install that came from the other, or from
a build signed with a real upload key later. That's fine for a build you sideload once and don't
expect to update; it is not a build you should distribute publicly, since anyone can reproduce a
"signed" update off their own machine's debug keystore too. Before a release build is meant to go
further than your own device, generate your own keystore — the standard `keytool` invocation is:
```bash
keytool -genkey -v -keystore <path-to-keystore>.jks -keyalg RSA -keysize 2048 \
-validity 10000 -alias <your-key-alias>
```
— and create `android/key.properties` (never commit it) with `storeFile`, `storePassword`,
`keyAlias`, `keyPassword` pointing at it.
One more effect of that same conditional worth knowing about, on the *debug* build type:
```kotlin
debug {
if (keystorePropertiesFile.exists()) {
signingConfig = signingConfigs.getByName("release")
}
}
```
If `key.properties` exists, **debug builds are signed with your release key too**, not the
machine-local Android Gradle Plugin debug keystore described above. The code comment explains why:
so a debug build installed for local testing has the same SHA-1 fingerprint as what you registered
for Google Sign-In (see [Connect to a self-hosted backend → Google Sign-In
configuration](connect-to-self-hosted.md#google-sign-in-configuration)), rather than needing a
second, separate OAuth Android client just for debug installs.
**iOS.** `ios/Runner.xcodeproj/project.pbxproj` sets `CODE_SIGN_IDENTITY[sdk=iphoneos*] = "iPhone
Developer"` for device builds at the project level, inherited by both targets.
`CODE_SIGN_STYLE = Automatic` also appears in the file, but only on the **`RunnerTests`** target's
three build configurations — the `Runner` app target and the project level don't set
`CODE_SIGN_STYLE` at all, and the project's `TargetAttributes` block doesn't set a
`ProvisioningStyle` for either target either. What is consistent across the whole file: there is
no `DEVELOPMENT_TEAM` key anywhere in it (checked directly; it's simply absent), so regardless of
what signing style Xcode shows you when you first open the project, you'll need to open
`ios/Runner.xcworkspace` in Xcode, select the Runner target's Signing & Capabilities tab, and pick
your Apple Developer team before `flutter build ios` (without `--no-codesign`) or `flutter build
ipa` will succeed. There's no keystore-equivalent file to create by hand here — Xcode manages
provisioning profiles itself once a team is selected.

114
docs/reference/environment-variables.md Normal file
View file

@ -0,0 +1,114 @@
# Environment variables
This page is the exhaustive table: every environment variable `backend/crm/settings.py` reads,
its real default, and what it's for. [Self-hosting: Environment variables](../self-hosting/environment-variables.md)
is the narrative companion — how configuration loading actually works (`load_dotenv()`, the
`.env.docker` / `.env.docker.local` layering), and the smaller, curated list of variables an
operator actually has to touch to run a working, non-default install. Read that page first if
you're setting up a deployment; come back here when you need the complete list or an exact default.
## How to set them
Every variable in the table below is read with `os.environ.get("NAME", <default>)` (or the
equivalent `os.getenv`) directly in `crm/settings.py`, at process import time — `manage.py
runserver`, `migrate`, a Gunicorn/Uvicorn worker, a Celery worker, and every management command all
read the same values the same way. A `.env` file next to `manage.py` (for a manual install) or
`.env.docker`/`.env.docker.local` (for the Docker Compose stack) is loaded into the process
environment before any of these calls run, so an unset variable falls back to the default in the
**Default** column rather than crashing immediately. That's true even for `SECRET_KEY`: its own
`os.environ.get` call always succeeds, but the value it produces then feeds a separate check a few
lines later that raises `ValueError` once `ENV_TYPE` is anything other than `dev` (see the
`SECRET_KEY` row below) — a default that satisfies `os.environ.get` doesn't necessarily satisfy
everything downstream of it. The variables with genuinely no default in code — read with plain
`os.environ[...]`, so a missing one is an immediate `KeyError` rather than a fallback — live outside
`crm/settings.py` entirely and are covered in the list below instead of the table.
Three groups of real, project-read environment variables are **not** read by `crm/settings.py`
itself, so they aren't in the table below:
- **`ADMIN_PASSWORD`** is read directly by the `create_default_admin` management command (see
[Management commands](management-commands.md#create_default_admin)) — it still gets its own row
below because it's part of the same Docker bootstrap flow as `ADMIN_EMAIL`, which *is* read in
`settings.py`.
- **`AWS_BUCKET_NAME`, `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `SENTRY_DSN`** are read with
plain `os.environ[...]` (no default) inside `backend/crm/server_settings.py`, which
`crm/settings.py` only imports when `ENV_TYPE=prod`. A missing one is a `KeyError` at import
time — the process refuses to start rather than starting half-configured. `ENV_TYPE` itself
follows the same rule: it's read in `settings.py`, but it's the internal dev/prod switch this
project's own tests exclude from the table by design, not a self-hoster-facing setting on its
own. See [Production deployment](../self-hosting/production-deploy.md#required-settings) and
[Security hardening](../self-hosting/security-hardening.md#secrets) for what `ENV_TYPE=prod`
actually requires and why it shouldn't be set casually.
- **`BCRM_MCP_ENABLED`** and **`BCRM_BASE_URL`** are two different knobs, both read in
`backend/crm/asgi.py`, not `crm/settings.py`. Only `BCRM_MCP_ENABLED` (default `"true"`) gates
whether the optional MCP server is mounted at `/mcp` at all — anything in `0`/`false`/`no`/`off`
disables it, anything else (including unset) leaves it enabled. `BCRM_BASE_URL` (default
`http://127.0.0.1:8000`) has no bearing on whether the mount happens; it's the REST API root the
mounted MCP tools call once it's live. See [MCP server](../integrations/mcp-server.md) for both.
## Reference
### Django core
| Variable | Default | Required | Purpose |
| --- | --- | --- | --- |
| `SECRET_KEY` | `django-insecure-dev-key-please-change-in-production` | Yes, once `ENV_TYPE` is anything other than `dev` (see above) | Django's cryptographic signing key. It also *is* the JWT `SIGNING_KEY` — `SIMPLE_JWT["SIGNING_KEY"]` is set to `SECRET_KEY` directly, so there is no separate JWT signing key setting anywhere in this codebase, and rotating `SECRET_KEY` invalidates every outstanding access and refresh token. See [Architecture: Authentication](../architecture/authentication.md#token-model) and [Security hardening](../self-hosting/security-hardening.md#secrets). |
| `DEBUG` | `False` | No | Django debug mode. Also the sole gate on the `devlogin` management command — see [Management commands](management-commands.md#devlogin). |
| `ALLOWED_HOSTS` | `localhost,127.0.0.1` | Yes, once serving traffic under a real hostname | Comma-separated list, split on `,` into Django's `ALLOWED_HOSTS`. |
### Database
| Variable | Default | Required | Purpose |
| --- | --- | --- | --- |
| `DBNAME` | `crm_db` | No | PostgreSQL database name. |
| `DBUSER` | `postgres` | Yes, before anyone but you can reach the instance | PostgreSQL role. The default is the PostgreSQL superuser account on most installations, and a superuser silently bypasses every RLS policy this project relies on for tenant isolation — see [PostgreSQL and RLS: Creating the application role](../self-hosting/postgresql-and-rls.md#creating-the-application-role). |
| `DBPASSWORD` | `postgres` | Yes, alongside `DBUSER` | PostgreSQL password. |
| `DBHOST` | `localhost` | No | PostgreSQL host. |
| `DBPORT` | `5432` | No | PostgreSQL port. |
### Email
| Variable | Default | Required | Purpose |
| --- | --- | --- | --- |
| `EMAIL_BACKEND` | `django.core.mail.backends.console.EmailBackend` | No | Django email backend. The default prints outbound mail to stdout instead of sending it. Setting `ENV_TYPE=prod` overrides this unconditionally to `django_ses.SESBackend`, regardless of what this variable is set to — see [Email and Celery](../self-hosting/email-and-celery.md#amazon-ses). |
| `DEFAULT_FROM_EMAIL` | `noreply@localhost` | No | `From` address on outbound mail. |
| `ADMIN_EMAIL` | `admin@localhost` | No | Assigned to `settings.ADMIN_EMAIL` but not consumed anywhere else in `settings.py` itself. The one place it's read outside this module and `create_default_admin` (see below) is `common/views/user_views.py`, which returns it as `admin_email` in the user-list API response context. `create_default_admin` also reads `ADMIN_EMAIL` a second time, independently and with the same default, to name the bootstrap Django superuser account. |
| `AWS_SES_REGION_NAME` | `ap-south-1` | No, unless `ENV_TYPE=prod` | Only read when `"django_ses"` appears in `EMAIL_BACKEND` — true if you set `EMAIL_BACKEND=django_ses.SESBackend` directly, or unconditionally true once `ENV_TYPE=prod` forces that backend. Under `ENV_TYPE=prod`, `server_settings.py` requires this variable outright (`os.environ[...]`, no default) before `settings.py`'s own conditional read ever runs, so the default above is effectively unreachable in that mode. |
| `AWS_SES_REGION_ENDPOINT` | `email.<AWS_SES_REGION_NAME>.amazonaws.com` | No, unless `ENV_TYPE=prod` | Same condition and the same `ENV_TYPE=prod`-makes-it-required caveat as `AWS_SES_REGION_NAME` above. |
### Celery and Redis
| Variable | Default | Required | Purpose |
| --- | --- | --- | --- |
| `CELERY_BROKER_URL` | `redis://localhost:6379/0` | Yes, before anyone but you can reach the instance | Redis URL Celery uses as its message broker. |
| `CELERY_RESULT_BACKEND` | `redis://localhost:6379/0` | Yes, alongside `CELERY_BROKER_URL` | Redis URL Celery uses to store task results. |
### CORS and CSRF
| Variable | Default | Required | Purpose |
| --- | --- | --- | --- |
| `CORS_ALLOW_ALL` | `False` | No | Sets Django's `CORS_ORIGIN_ALLOW_ALL` — note the name doesn't match the Django setting it controls. `"true"` (case-insensitive) allows every origin; anything else leaves it off. |
| `CORS_ALLOWED_ORIGINS` | `http://localhost:5173,http://127.0.0.1:5173` | Yes, once the frontend is served from a real domain | Comma-separated list of allowed CORS origins; each entry is stripped of surrounding whitespace and empty entries are dropped. |
| `CSRF_TRUSTED_ORIGINS` | *(empty — no trusted origins)* | Yes, before anyone but you can reach the instance | Comma-separated list, same stripping rule as `CORS_ALLOWED_ORIGINS`. Each entry needs a scheme, e.g. `https://crm.example.com` — a bare hostname is silently useless to Django's CSRF check. |
### URLs
| Variable | Default | Required | Purpose |
| --- | --- | --- | --- |
| `DOMAIN_NAME` | `http://localhost:8000` | Yes, once self-hosted under a real domain | Base URL embedded in outbound emails' links back to the CRM — this is the **backend's** URL, not necessarily the frontend's. Two uses matter more than the rest: it builds the link an **anonymous client** gets in an emailed invoice/estimate (`invoices/api_views.py`, `invoices/tasks.py`, via `send_invoice_to_client(..., domain=...)`) and the link in a **CSAT survey** email (`cases/tasks.py`, `link = f"{domain}/csat/{raw_token}"`) — both go to people outside the organization, so leaving the `localhost` default in a real deployment doesn't just look wrong internally, it emails dead links to customers. The remaining, lower-stakes uses are internal notification emails (comment-mention, account activation/deactivation, and the per-record "assigned to you" emails in `leads/tasks.py`, `cases/tasks.py`, `opportunity/tasks.py`, `contacts/tasks.py`, `accounts/tasks.py`), which only reach people who already have accounts in the org. |
| `FRONTEND_URL` | `http://localhost:5173` | Yes, once self-hosted under a real domain | Base URL of the SvelteKit frontend. Used to build the welcome email's link and the magic-link sign-in URL (`common/tasks.py`) — the links a recipient is actually meant to click into the app. |
| `SWAGGER_ROOT_URL` | `http://localhost:8000` | No | Read into a Django setting in `crm/settings.py`, but not referenced anywhere else in this codebase (backend, frontend, or mobile) as of this writing — setting it currently has no observable effect. |
### Google OAuth
| Variable | Default | Required | Purpose |
| --- | --- | --- | --- |
| `GOOGLE_CLIENT_ID` | *(empty)* | No — leaving it blank disables Google sign-in | OAuth client ID sent to Google's token endpoint and used as the expected audience when verifying a mobile ID token. See [Google OAuth](../self-hosting/google-oauth.md#backend-configuration). |
| `GOOGLE_CLIENT_SECRET` | *(empty)* | No — leaving it blank disables Google sign-in | OAuth client secret, alongside `GOOGLE_CLIENT_ID` above. |
| `GOOGLE_REDIRECT_URI` | *(empty)* | No | Defined in `settings.py` but not read anywhere else in the backend — the redirect URI actually used in the token exchange is the one the frontend sends in the request body on each call, not this setting. Safe to leave unset even in production; see [Google OAuth](../self-hosting/google-oauth.md#backend-configuration) for why. |
### Docker bootstrap (not read by `crm/settings.py`)
| Variable | Default | Required | Purpose |
| --- | --- | --- | --- |
| `ADMIN_PASSWORD` | *(empty string)* | No, but leaving it unset is a real security gap | Read directly by the `create_default_admin` management command. An empty value doesn't mean "no password" — the command falls back to the literal password `admin` and prints a warning. `create_default_admin` runs automatically on every Docker container start (`docker/backend/entrypoint.sh`, right after `migrate`), so an unset `ADMIN_PASSWORD` in a real deployment means the bootstrap Django superuser's password is the literal string `admin` until someone changes it. See [Management commands](management-commands.md#create_default_admin). |

126
docs/reference/glossary.md Normal file
View file

@ -0,0 +1,126 @@
# Glossary
Terms used throughout this documentation that are either specific to BottleCRM's own model, or
that sound generic but resolve to a particular model or field in this codebase. Each entry links to
the page that covers the concept in full; this page is the short definition, not the whole story.
## Terms
### Organization
`Org` (`backend/common/models.py`, db table `organization`) — the tenant boundary. Every org-scoped
record carries an `org` foreign key to this model, either declared directly or inherited from
`BaseOrgModel`, and Row-Level Security enforces that a request in one organization's context never
returns another organization's rows. Carries a `default_currency`/`default_country` locale pair, the
[organization API key](#organization-api-key), and — once a vertical pack has been applied —
`vertical` and `terminology` fields that are **descriptive only**: the model's own comment states
they are never used in a permission check or a queryset filter.
### Profile
`Profile` (`backend/common/models.py`) — a user's membership in one organization: a `(user, org)`
pair, unique together, carrying that membership's role (`ADMIN`/`USER`, from `ROLES`),
`is_organization_admin`, `has_sales_access`/`has_marketing_access`, and `is_active`. One `User` can
hold a separate `Profile` — with a separate role — in each organization they belong to.
`request.profile`, set by the `GetProfileAndOrg` middleware from the JWT on every request, is what
every permission check and org filter in this codebase is written against, never `request.user`
directly. See [Permissions and roles](../architecture/permissions-and-roles.md).
### Tenant
Not a distinct model or field in this codebase; the general term this documentation (and the wider
industry) uses for "one organization's data, isolated from every other organization's." In
BottleCRM, an organization *is* the tenant boundary — see
[Multi-tenancy and RLS](../architecture/multi-tenancy-and-rls.md).
### RLS context
`app.current_org` — the PostgreSQL session variable Row-Level Security policies read to decide
which rows a query is allowed to see. `CONTEXT_VARIABLE` in `backend/common/rls/__init__.py` names
it; the `RequireOrgContext` middleware (`backend/common/middleware/rls_context.py`) sets it once per
request, via `SELECT set_config('app.current_org', <org_id>, false)`, and clears it afterward. The
org id it sets comes from `request.org`, one hop removed from `request.profile.org`: the earlier
`GetProfileAndOrg` middleware (`backend/common/middleware/get_company.py`) is what actually resolves
the org — from the JWT, a personal access token, or an organization API key, depending on which
authentication path the request took — and assigns it to `request.org` directly (as well as setting
`request.profile`); `RequireOrgContext` only reads `request.org`, not `request.profile.org`, when
deciding what to set. A request with no context set sees zero rows from every RLS-protected table —
the fail-safe every isolation policy's `NULLIF(current_setting(...), '')` clause is built around.
See [Multi-tenancy and RLS](../architecture/multi-tenancy-and-rls.md).
### Vertical pack
A small, declarative JSON manifest (`backend/packs/*.json`, validated by
`backend/common/packs/schema.py`) that shapes an *already-existing* organization toward one
industry: a starter pipeline (with stages) for leads, cases, and tasks; custom field definitions;
tags; products; terminology overrides (for example relabeling "Lead" as "Enquiry"); and a short list
of named sample records that reference each other within the manifest. Applying a pack is a
one-time, idempotent action, distinct from `seed_data` — see
[Management commands](management-commands.md#seed_data) — which generates a large randomized
dataset instead of a curated one. See
[Demo data and packs](../getting-started/demo-data.md#vertical-packs).
### Portal token
`PortalAccessToken` (`backend/common/models.py`) — a token minted for an anonymous, unauthenticated
visitor (someone following a shared invoice/estimate link, or answering a CSAT survey) to reach
exactly one record without signing in. Deliberately absent from `ORG_SCOPED_TABLES` and carrying no
RLS policy of its own, because a portal request has no `request.profile` to derive an org context
from in the first place: the table instead maps `sha256(url_token)` to an `org_id` directly, so the
public view can hash the token it was handed, resolve the org, set the RLS context, and only then
query the actual resource. See
[Multi-tenancy and RLS: Portal tokens](../architecture/multi-tenancy-and-rls.md#portal-tokens).
### Personal access token
`bcrm_pat_…` (`PersonalAccessToken`, `backend/common/models.py`) — a token a signed-in user creates
for themselves (`POST /api/profile/tokens/`) to authenticate scripts, the MCP server, or other
programmatic access without a browser session. It authenticates as the profile that created it and
inherits that profile's role and org in full; its `scopes` field is stored but not enforced.
Deactivating the owning profile invalidates the token immediately — `resolve_valid_pat` checks
`profile.is_active` on every use — with no separate revocation step required. See
[Tokens and API keys: Personal access tokens](../api/tokens-and-api-keys.md#personal-access-tokens).
### Organization API key
`Org.api_key` — a single key per organization, not per user, managed through `GET`/`POST
/api/org/api-key/`, admin-only in both directions. A request bearing it in the `Token` header
authenticates as the organization's first active `ADMIN` profile rather than any specific person, so
a leaked key is equivalent to a leaked admin session — not a leaked personal access token, which
always authenticates as one specific individual. See
[Tokens and API keys: Organization API keys](../api/tokens-and-api-keys.md#organization-api-keys).
### Stage versus status
Two different kinds of field that sound alike but aren't. `status` (`Lead.status`, `Case.status`,
`Task.status`) is a plain `CharField` over a fixed set of choices — three separate tuples, not one
shared definition: `LEAD_STATUS` and `STATUS_CHOICE`, defined in `backend/common/utils.py`, back
`Lead.status` and `Case.status` respectively, while `Task.STATUS_CHOICES` is defined inline on the
`Task` model itself in `backend/tasks/models.py`. Each one is still fixed and identical across every
organization — that part of the "status vs. stage" distinction holds for all three. `stage`, on those
same three
models, is a nullable foreign key to a per-org `LeadStage`/`CaseStage`/`TaskStage` row, itself
belonging to a `LeadPipeline`/`CasePipeline`/`TaskPipeline` (see [Pipeline](#pipeline)) — each
organization can define its own Kanban columns, and a stage's `maps_to_status` field is what keeps
the fixed `status` in sync when a record moves between stages. `Opportunity.stage`, despite sharing
the field name, is the *fixed* kind: a plain `CharField` over `common.utils.STAGES` (`PROSPECTING`
… `CLOSED_LOST`), the same six values for every organization, with no configurable pipeline behind
it and no separate `status` field at all. See
[Data model: Conventions](../architecture/data-model.md#conventions) for the full explanation this
entry summarizes.
### Pipeline
`LeadPipeline`, `CasePipeline`, `TaskPipeline` — an org-owned, named container of ordered `Stage`
rows for one entity type (see [Stage versus status](#stage-versus-status)). An organization can
define more than one pipeline for the same entity type — each model's own docstring gives an
example set of names (Inbound/Outbound/Enterprise for leads, Support/Engineering/Billing for cases,
Development/Support/Marketing for tasks) — and *at most* one per organization may be marked
`is_default`, enforced by a `UniqueConstraint` on `org` scoped to `is_default=True`; nothing requires
that one exist. The field's own `help_text` says a default pipeline is "where new [records] without
explicit pipeline go", but that's a statement of intent, not a description of running code: nothing
in this codebase currently reads `is_default` to route a new `Lead`, `Case`, or `Task` onto a
pipeline. A record's `stage` foreign key is simply `NULL` until something sets it explicitly, which
its own `help_text` calls "use status-based kanban" — the fixed `status` field, not a pipeline stage,
is what a newly created record actually falls back to. `Opportunity` has no equivalent pipeline
model: its stage list is the fixed set above, not org-configurable.

152
docs/reference/management-commands.md Normal file
View file

@ -0,0 +1,152 @@
# Management commands
Every command below is BottleCRM's own — the entire `common/management/commands/` package holds
exactly these five files. Run `uv run python manage.py help` from `backend/` and everything
outside the `[common]` section comes from Django itself or an installed third-party package
(`django`, `auth`, `contenttypes`, `sessions`, `staticfiles`, `django_ses`, `drf_spectacular`,
`rest_framework`, `token_blacklist`) — none of it is project-specific, so it isn't repeated here.
Run any command from `backend/` with `uv run python manage.py <command>` (or, against the Docker
stack, `docker compose exec backend python manage.py <command>`).
## manage_rls
Read-only diagnostics for Row-Level Security. None of its three flags modify the database — RLS
policies are enabled and disabled by Django migrations, calling `get_enable_policy_sql()`, never by
this command (the module's own docstring says so explicitly). It checks the tables listed in
`ORG_SCOPED_TABLES` (`common/rls/__init__.py`), which as of this writing has **61 entries**. The
repository's `CLAUDE.md` still says 24; that figure is out of date, and the count is deliberately
not repeated anywhere outside `ORG_SCOPED_TABLES` itself so it cannot drift again.
```bash
uv run python manage.py manage_rls --status
uv run python manage.py manage_rls --test
uv run python manage.py manage_rls --verify-user
```
| Flag | What it does |
| --- | --- |
| `--status` | For every table in `ORG_SCOPED_TABLES`, queries `pg_class.relrowsecurity`/`relforcerowsecurity` and prints `ENABLED (forced)`, `disabled`, or `TABLE NOT FOUND`, plus whether the connected role is a superuser. This is also the **default** — running `manage_rls` with no flags at all runs `--status`. |
| `--test` | Picks two organizations (preferring ones that actually have `Lead` rows), sets the RLS context to each in turn and counts visible leads, then sets an empty context and counts again. Zero rows with no context and nonzero with a real context means isolation is working; nonzero rows with no context is a warning, not a failure, since some policies deliberately allow an empty context. |
| `--verify-user` | Queries `pg_user` for `usesuper`/`usecreatedb` on the connected role and prints an explicit `WARNING: Superusers bypass RLS!` with remediation SQL if the role is a superuser. |
This only covers what the command itself does. For why a superuser role defeats every policy
silently, how to create a correctly-restricted role, and what each flag's output actually proves,
see [PostgreSQL and RLS](../self-hosting/postgresql-and-rls.md#verifying-isolation).
## seed_data
Generates a large, randomized dataset — accounts, contacts, leads, opportunities, cases, tasks,
invoices and more — for one or more organizations, entirely from curated fictional name/company
pools and reserved demo domains (`example.com`/`.example`) so nothing it creates resembles a real
person or business. The first organization created (`--orgs` index 0) is always named
`MicroPyramid`, specifically so local-dev workflows have a stable org name to sign in against with
`devlogin --org MicroPyramid` (see below). For the default, single-org run, re-running the command
with the same `--email` reuses that existing user, org, and profile rather than creating duplicates
— `create_org` does an `Org.objects.get_or_create(name="MicroPyramid", ...)`. That reuse guarantee
is specific to index 0: any organization beyond the first (`--orgs 2` or higher) gets a name drawn
randomly from a company-name pool each run, so re-running with `--orgs 2` typically accumulates
additional organizations rather than reusing the second one from the previous run.
```bash
uv run python manage.py seed_data --email you@example.com
uv run python manage.py seed_data --email you@example.com --orgs 2 --leads 100 --seed 42
uv run python manage.py seed_data --email you@example.com --clear --no-input
```
| Flag | Default | What it does |
| --- | --- | --- |
| `--email` | `aswin.1231@gmail.com` | Email of the admin user to create (or reuse) as the seeded org's admin profile. |
| `--orgs` | `1` | Number of organizations to create. |
| `--users-per-org` | `3` | Additional non-admin users created per org. |
| `--leads` | `20` | Leads per org. |
| `--accounts` | `10` | Accounts per org. |
| `--contacts` | `15` | Contacts per org. |
| `--opportunities` | `10` | Opportunities per org. |
| `--cases` | `5` | Cases per org. |
| `--tasks` | `10` | Tasks per org. |
| `--goals` | `8` | Sales goals per org. |
| `--teams` | `2` | Teams per org. |
| `--tags` | `5` | Tags per org. |
| `--products` | `20` | Products per org. |
| `--invoices` | `50` | Invoices per org. |
| `--estimates` | `15` | Estimates per org. |
| `--recurring-invoices` | `5` | Recurring invoices per org. |
| `--invoice-templates` | `3` | Invoice templates per org. |
| `--currency` | `USD` | Default org currency; restricted to `CURRENCY_CODES` choices. |
| `--country` | `US` | Default org country code; also selects the Faker locale used for generated names/addresses (falls back to `en_US` for an unmapped code). |
| `--seed` | none | Random seed, for a reproducible run. |
| `--password` | `testpass123` | Password set on newly created users. |
| `--clear` | off (flag) | Deletes existing CRM data before seeding — **not** users, orgs, or profiles. Prompts for confirmation unless `--no-input` is also given. This is a separate, broader operation from a vertical pack's own sample-data clearing (see [Demo data and packs](../getting-started/demo-data.md#clearing-sample-data)) and isn't scoped to pack-created records specifically. |
| `--no-input` | off (flag) | Skips the `--clear` confirmation prompt. |
See [Demo data and packs](../getting-started/demo-data.md) for the narrative walkthrough, including
what a full seeded org looks like and how vertical packs layer on top of it.
## devlogin
Mints a JWT access/refresh pair for an existing user directly from the command line — no OAuth
provider, no outbound email, no browser round-trip. **Refuses to run unless `settings.DEBUG` is
`True`**, raising a `CommandError` as the very first thing `handle()` does — that means it prints an
error and exits non-zero in any environment where `DEBUG=False`, not that it silently does nothing.
```bash
uv run python manage.py devlogin <email> --org <name-or-uuid>
uv run python manage.py devlogin <email> --create
```
| Argument | What it does |
| --- | --- |
| `email` (positional, required) | The user to mint a token for. |
| `--org` | Optional org name or UUID. **Resolves id first, then name**: it tries `Org.objects.get(id=org_arg)` first, and only falls back to matching on `Org.name` if that lookup fails (not found, or `org_arg` isn't a valid UUID at all). More than one org sharing that name is refused outright rather than guessed at — pass an id instead. The target user must already have an **active** `Profile` in the resolved org — this command does not create one, and raises a `CommandError` naming the missing user/org pair if none exists. |
| `--create` | Creates the user (with a random, discarded password) if no user with that email exists yet. This only creates the `User` row — it has no effect on the `Profile` requirement above; a freshly created user still needs a `Profile` in the target org before `--org` will succeed for them. |
On success it prints an access token and a refresh token, the org's UUID if `--org` was given (a
token minted this way already carries the `org_id` claim, so no follow-up org-switch call is
needed), and a `localStorage.setItem(...)` snippet for a browser console — that snippet alone does
**not** sign you into the SvelteKit app itself; see
[Using the tokens in the browser](../getting-started/first-sign-in.md#using-the-tokens-in-the-browser)
for the cookies `hooks.server.js` actually checks. See
[First sign-in: Local development](../getting-started/first-sign-in.md#local-development-devlogin)
for the full walkthrough, including what happens with no `--org` at all.
## create_default_admin
Creates a Django superuser, but only if none already exists — `User.objects.filter(is_superuser=
True).exists()` short-circuits the whole command to a no-op if one is found, so it's safe to run on
every startup. It takes no arguments at all (no `add_arguments` override). It reads `ADMIN_EMAIL`
and `ADMIN_PASSWORD` directly with `os.environ.get`, independently of `crm/settings.py` (`ADMIN_EMAIL`
happens to share a name and a default with the copy `settings.py` also reads; `ADMIN_PASSWORD` is
read nowhere else). **If `ADMIN_PASSWORD` is unset or empty, the command falls back to the literal
password `admin`** and prints a warning — it does not refuse to create the account. This command
runs automatically on every Docker container start (`docker/backend/entrypoint.sh`, immediately
after `migrate`), so in the Docker quick start it never needs to be invoked directly, and an unset
`ADMIN_PASSWORD` in a real deployment means the bootstrap superuser's password really is `admin`
until it's changed by hand.
```bash
uv run python manage.py create_default_admin
```
## audit_org_fields
A one-time Phase 1 multi-tenancy hardening tool, not a general or self-updating auditor: it checks
a **hardcoded** pair of Python lists inside the command itself — `models_with_nullable_org`
(currently just `common.Address` and `common.Tags`) and `models_with_required_org` (thirteen named
models) — not every org-scoped model that exists today. A model added to the codebase since this
command was last edited is invisible to it either way.
```bash
uv run python manage.py audit_org_fields --check
uv run python manage.py audit_org_fields --fix
uv run python manage.py audit_org_fields --verbose
```
| Flag | What it does |
| --- | --- |
| `--check` | Accepted by the argument parser but **never read in `handle()`** — passing it or omitting it produces identical output. The audit itself always runs; there's no way to suppress it short of not invoking the command. |
| `--fix` | For `common.Address` rows with a null `org`, attempts to backfill it from a related `Profile`. For `common.Tags`, `--fix` does **not** fix anything — it prints a message asking for manual review instead. For any other model that turns up with a null `org`, it also just prints "Automatic fix not implemented." |
| `--verbose` | Two effects, one per pass: in the nullable-org pass it prints each model's field configuration (`null`, `blank`, `related_query_name`); in the required-org pass it's what makes a "Model not found" warning print at all — without `--verbose`, a `LookupError` there is swallowed silently. |
Output uses ✅/❌/⚠️ markers per model and ends with a summary and a suggested next-steps block
(create a migration, run `makemigrations`/`migrate`) — it never modifies the schema itself, only
data, and only when `--fix` is passed.

96
docs/self-hosting/backups-and-upgrades.md Normal file
View file

@ -0,0 +1,96 @@
# Backups and upgrades
This page covers what actually needs backing up in a self-hosted BottleCRM instance, and what
"upgrading" means for the Docker Compose setup versus a manual install.
## What to back up
Two things hold state that nothing else can regenerate: **the PostgreSQL database** and **the media
directory**. Everything else — the application code, static assets (`collectstatic` output),
installed dependencies, the Redis broker/result store — is either checked into version control,
rebuilt from it, or disposable. Redis in particular is not a backup concern: it's the Celery broker
and result backend (`CELERY_BROKER_URL`/`CELERY_RESULT_BACKEND`), not a store of business data — an
in-flight task queue lost on restart means a retry, not lost records, since the records themselves
live in PostgreSQL.
## Backing up PostgreSQL
Every org, every record, every RLS-protected row lives in the database configured by `DBNAME`/
`DBUSER`/`DBHOST`/`DBPORT` (see [Environment variables](environment-variables.md#database)). A plain
`pg_dump` is enough for a logical backup:
```bash
pg_dump -h <DBHOST> -p <DBPORT> -U <DBUSER> -d <DBNAME> -F c -f backup.dump
```
Under Docker Compose, run it against the `db` service:
```bash
docker compose exec db pg_dump -U postgres -d crm_db -F c -f /tmp/backup.dump
docker compose cp db:/tmp/backup.dump ./backup.dump
```
`docker-compose.yml` also declares `postgres_data` as a named volume mounted at
`/var/lib/postgresql/data` on the `db` service — this is the actual on-disk database, and is what
must survive a `docker compose down` (which removes containers but leaves named volumes intact) or a
rebuild. Only `docker compose down -v`, or removing the volume explicitly, deletes it — see
[Docker](docker.md#persisting-data).
## Media files
`MEDIA_ROOT` depends on which settings module is active. With `ENV_TYPE=dev` (the default),
`crm/settings.py` sets `MEDIA_ROOT` to a local `media/` directory under `BASE_DIR`, and uploaded
files (invoice attachments, case attachments, and similar) land on local disk. Under Docker Compose
this resolves to `/app/media` inside the `backend` container, which — because `/app` is the
bind-mounted `./backend` directory, not a named volume — means uploads land in `backend/media/` on
the host automatically, but are **not** covered by the `postgres_data` volume's guarantees. Back up
`backend/media/` (or your equivalent local media path in a manual install) alongside the database.
With `ENV_TYPE=prod`, `backend/crm/server_settings.py` switches `DEFAULT_FILE_STORAGE` to
`storages.backends.s3boto3.S3Boto3Storage` and media is written to the S3 bucket named by
`AWS_BUCKET_NAME` instead of local disk — in that configuration, back up the bucket using your normal
AWS backup practice (versioning, replication, or a separate backup job) rather than a filesystem
backup.
## Upgrading
Under Docker Compose, `docker/backend/entrypoint.sh` runs `manage.py migrate --noinput`,
`manage.py create_default_admin` and `manage.py collectstatic --noinput` on **every** container
start, not only the first — so pulling new source and rebuilding is the entire upgrade procedure:
```bash
git pull
docker compose up --build
```
There is no separate migration step to remember; the schema and static assets are brought up to
date automatically on restart. Back up `postgres_data` and `backend/media/`, per the sections above,
before upgrading across any change you're not confident is backward compatible with your existing
data — nothing in this compose setup takes an automatic backup for you.
For a manual install (no Docker), the equivalent is: pull the new source, `uv sync` to update
dependencies, then run migrations and `collectstatic` yourself before restarting the process (see
[Migrations](#migrations) below and [Production deployment](production-deploy.md#before-you-start)).
## Migrations
`manage.py migrate` applies schema changes and, for the RLS-related migrations specifically,
re-applies the Row-Level Security policies described in
[PostgreSQL and RLS](postgresql-and-rls.md#enabling-policies) — RLS is enabled and disabled by
migrations, not by any separate command. Those migrations run with `atomic = False` (the comment
next to it, repeated across every migration that calls `get_enable_policy_sql`, is explicit: "RLS
policy creation can't run inside an atomic block"), and `get_enable_policy_sql()` itself issues
`DROP POLICY IF EXISTS` before each `CREATE POLICY`, so re-running `migrate` after an interrupted run
is safe rather than something that fails on a duplicate policy.
This project's own CI enforces that no model change ships without a matching migration:
`.github/workflows/tests.yml` runs
```bash
uv run python manage.py makemigrations --check --dry-run
```
as a dedicated step before the test suite. If you fork this project and add or change a model, run
`manage.py makemigrations` yourself and commit the result. The check names the problem directly;
without it you would find out indirectly, from a test or a request failing against a column the
database does not have.

114
docs/self-hosting/docker.md Normal file
View file

@ -0,0 +1,114 @@
# Docker
`docker-compose.yml`, at the repository root, runs the full stack — database, cache/broker,
API, background workers and frontend — as six services from one file. This page documents that
file service by service. For the fastest path to a running instance with demo data loaded, see
[Docker quick start](../getting-started/docker-quick-start.md); this page instead covers what to
configure, how data is persisted, and how to upgrade an instance you intend to keep running.
**This compose file is a development setup, not a hardened production one.** Its `backend`
service bind-mounts the repository (`./backend:/app`) and its entrypoint
(`docker/backend/entrypoint.sh`) ends by running `python manage.py runserver 0.0.0.0:8000` — the
Django development server, not Gunicorn. `.env.docker` also ships `DEBUG=True`. Running it as-is
gets every service talking to each other correctly; it does not by itself satisfy the
requirements in [Production deployment](production-deploy.md).
## The services
`docker-compose.yml` defines:
| Service | Image / build | Host port | Purpose |
|---|---|---|---|
| `db` | `postgres:16-alpine` | `5432` | The PostgreSQL database, `crm_db`. Row-Level Security policies applied by Django migrations rely on this being real PostgreSQL — see [PostgreSQL and RLS](postgresql-and-rls.md). |
| `redis` | `redis:7-alpine` | `6379` | Celery broker and result backend (`CELERY_BROKER_URL` / `CELERY_RESULT_BACKEND` both point here). |
| `backend` | built from the repo-root `Dockerfile` | `8000` | The Django API, served via `manage.py runserver` in this compose file (see the warning above). |
| `celery-worker` | same image as `backend` | — (no published port) | Runs `celery -A crm worker --loglevel=info`; executes background tasks such as outbound email and RLS-context-aware jobs in `backend/common/tasks.py`. |
| `celery-beat` | same image as `backend` | — (no published port) | Runs `celery -A crm beat --loglevel=info`; fires the periodic tasks registered in `backend/crm/celery.py` (recurring invoices, overdue/expiry checks, SLA scanning, cleanup jobs) on their schedules. |
| `frontend` | built from `frontend/Dockerfile` | `5173` | The SvelteKit app, served via `pnpm dev --host 0.0.0.0` — also a dev server, not a production build. |
`backend`, `celery-worker` and `celery-beat` each declare `depends_on: db (service_healthy), redis
(service_healthy)`, using the `pg_isready` and `redis-cli ping` healthchecks defined on those two
services, so they wait for both to actually accept connections rather than just for their
containers to start.
## Configuration
Every service loads two env files, in order: `.env.docker` (checked into the repository, working
defaults) and then `.env.docker.local` (optional, gitignored, `required: false` — create it next
to `.env.docker` to override specific values without touching the tracked file; values there win).
The variables `.env.docker` sets: `SECRET_KEY` (an insecure placeholder — see
[Production deployment](production-deploy.md) before this instance is anything but local),
`DEBUG=True`, `ENV_TYPE=dev`, `ALLOWED_HOSTS=localhost,127.0.0.1,backend`, `DOMAIN_NAME` and
`SWAGGER_ROOT_URL` (both `http://localhost:8000`), the `DBNAME`/`DBUSER`/`DBPASSWORD`/`DBHOST`/
`DBPORT` the `backend` and Celery containers use to reach `db`, `POSTGRES_DB`/`POSTGRES_USER`/
`POSTGRES_PASSWORD` (used only by the `db` service itself, to create the database and its own
superuser role — see [PostgreSQL and RLS](postgresql-and-rls.md) for why the app never connects
as this role), `CELERY_BROKER_URL`/`CELERY_RESULT_BACKEND` (both pointing at `redis`),
`DEFAULT_FROM_EMAIL`/`ADMIN_EMAIL`/`ADMIN_PASSWORD`, `CORS_ALLOW_ALL=True`,
`GOOGLE_CLIENT_ID`/`GOOGLE_CLIENT_SECRET`/`GOOGLE_REDIRECT_URI` (blank — Google sign-in stays
disabled until set), and `PUBLIC_DJANGO_API_URL=http://localhost:8000` for the frontend build.
## Running it
From the repository root (the directory containing `docker-compose.yml` and `Dockerfile`):
```bash
docker compose up --build
```
This requires the Compose v2 plugin (the `docker compose` subcommand, not the standalone
`docker-compose` binary). On first boot, `docker/postgres/init-rls-user.sql` runs automatically
via PostgreSQL's `/docker-entrypoint-initdb.d/` mechanism and creates the non-superuser `crm_user`
role that `backend` actually connects as. `docker/backend/entrypoint.sh` then waits for
PostgreSQL to accept connections, runs `manage.py migrate --noinput`, runs
`manage.py create_default_admin`, runs `manage.py collectstatic --noinput`, and finally starts the
dev server — every one of those steps runs again on every container restart, not just the first.
Run any one-off management command inside the running `backend` container with `docker compose
exec`, for example:
```bash
docker compose exec backend python manage.py migrate
docker compose exec backend python manage.py manage_rls --status
```
## Persisting data
`docker-compose.yml` declares two named volumes:
| Volume | Mounted at | Holds |
|---|---|---|
| `postgres_data` | `/var/lib/postgresql/data` on `db` | The actual database: every org, record, and RLS-protected row. This is the volume that must survive a rebuild. |
| `frontend_node_modules` | `/app/node_modules` on `frontend` | Installed frontend dependencies — a build cache, not user data. Safe to delete; `pnpm install` recreates it. |
Everything else is a bind mount of source code from the host, not a Docker-managed volume:
`./backend:/app` on `backend`/`celery-worker`/`celery-beat`, and `./frontend:/app` on `frontend`.
One consequence worth knowing: with `ENV_TYPE=dev` (the shipped default), Django's `MEDIA_ROOT` is
`BASE_DIR/media`, which resolves inside the container to `/app/media` — and because `/app` is the
bind-mounted `./backend` directory, uploaded files (invoice attachments, case attachments, and so
on) land in `backend/media/` on the host automatically. They persist across container rebuilds for
the same reason the source code does, without needing a dedicated named volume — but they are
**not** covered by a `postgres_data`-style volume, so back up `backend/media/` alongside the
database if you rely on uploaded files.
`docker compose down` removes containers and the default network but leaves named volumes intact;
only `docker compose down -v`, or removing `postgres_data`/`frontend_node_modules` explicitly,
deletes them.
## Upgrading
To move to a newer version of the application: pull the new source, then rebuild and restart:
```bash
docker compose up --build
```
Because `entrypoint.sh` runs `manage.py migrate --noinput` on every container start (not only the
first), rebuilding and restarting is sufficient to bring the schema up to date — there is no
separate migration step to remember. `collectstatic` also reruns on every start, so updated static
assets are picked up automatically.
Back up the `postgres_data` volume (and `backend/media/`, per the note above) before upgrading
across any change you're not certain is backward compatible with your existing data — nothing in
this compose setup takes an automatic backup for you.

115
docs/self-hosting/email-and-celery.md Normal file
View file

@ -0,0 +1,115 @@
# Email and Celery
BottleCRM sends email (welcome messages, magic-link sign-in, comment mentions) and runs background
and scheduled work (recurring invoices, SLA scanning, token cleanup) through Django's email
framework and Celery, respectively. This page covers how to configure both and the one rule every
background task that touches tenant data has to follow.
## Email backend
`EMAIL_BACKEND` in `backend/crm/settings.py` defaults to
`django.core.mail.backends.console.EmailBackend` — outbound mail is printed to the process's stdout
rather than delivered anywhere. `.env.example` sets `EMAIL_BACKEND` to that console backend
explicitly; `.env.docker` doesn't set `EMAIL_BACKEND` at all (only a comment noting the console
backend is what's active for local development), so it relies on the same default rather than
setting it. To send real email, set `EMAIL_BACKEND` to a working Django email backend, most
commonly `django_ses.SESBackend` (see [Amazon SES](#amazon-ses) below); `django_ses` is always in
`INSTALLED_APPS`, so the package is available regardless of which backend you choose.
There is a second, coarser way `EMAIL_BACKEND` gets set to SES: if you set `ENV_TYPE=prod`,
`backend/crm/server_settings.py` is imported and hardcodes `EMAIL_BACKEND = "django_ses.SESBackend"`
unconditionally, overriding whatever `EMAIL_BACKEND` you set directly. `ENV_TYPE=prod` also requires
`AWS_BUCKET_NAME`, `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_SES_REGION_NAME`,
`AWS_SES_REGION_ENDPOINT` and `SENTRY_DSN` to already be set (a plain `KeyError` at import time if
any is missing) and hardcodes `SESSION_COOKIE_DOMAIN = ".bottlecrm.io"` — see
[Production deployment](production-deploy.md#static-files) for the full consequences. If you only
want SES for email and don't want the S3/Sentry/cookie-domain side effects that come with
`ENV_TYPE=prod`, set `EMAIL_BACKEND=django_ses.SESBackend` directly instead and leave `ENV_TYPE=dev`.
`DEFAULT_FROM_EMAIL` (default `noreply@localhost`) is the sender address used on every outbound
message; `ADMIN_EMAIL` (default `admin@localhost`) is separate and is what
`manage.py create_default_admin` uses for the seeded admin account, not a mail setting.
## Amazon SES
Once `EMAIL_BACKEND` resolves to `django_ses.SESBackend` — whichever of the two paths above got you
there — `crm/settings.py` reads `AWS_SES_REGION_NAME` (default `ap-south-1`) and
`AWS_SES_REGION_ENDPOINT` (default `email.{AWS_SES_REGION_NAME}.amazonaws.com`, computed from
whatever you set for the region). AWS credentials are read implicitly by `django-ses`/`boto3` from
`AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` if you set them; if you don't, `boto3` falls back to
IAM role credentials, which is the intended path when running inside AWS (EC2/ECS instance role)
rather than with long-lived access keys in the environment.
The Celery tasks that actually send mail — `send_welcome_email` and `send_magic_link_email` in
`backend/common/tasks.py` — catch `botocore.exceptions.ClientError` around the `msg.send()` call and
log the failure rather than letting the task raise, specifically because SES can reject a send (an
unverified sender identity, a suppressed recipient address, being outside the SES sandbox) in ways
that are the mail provider's problem, not a bug in the task. Check the worker's logs, not the Celery
task result, if an expected email never arrives.
## Broker and result backend
`CELERY_BROKER_URL` and `CELERY_RESULT_BACKEND` both default to `redis://localhost:6379/0` in
`crm/settings.py`, and both `.env.example` and `.env.docker` leave them at that default (Docker
Compose's value points at the `redis` service hostname instead of `localhost`). Both need a reachable
Redis instance — `backend/crm/celery.py` constructs `Celery("crm")` and calls
`app.config_from_object("django.conf:settings", namespace="CELERY")`, so these two Django settings
are what Celery actually connects with; there's no separate Celery-specific config file.
## Running a worker
```bash
cd backend
uv run celery -A crm worker --loglevel=INFO
```
This process executes tasks queued by the API — sending the emails above, mention notifications,
team-membership propagation, and more. A separate `celery beat` process is required for anything on
a schedule: `backend/crm/celery.py` registers `app.conf.beat_schedule` with ten periodic entries,
including recurring-invoice generation and overdue/expired-estimate checks (daily), SLA breach
scanning for cases (every 5 minutes), stale-timer cleanup (every 30 minutes), and nightly cleanup of
read notifications and expired refresh-token records:
```bash
cd backend
uv run celery -A crm beat --loglevel=INFO
```
Neither process is required for the API itself to answer requests — only for background and
scheduled work to actually run. `docker-compose.yml` runs both as their own services,
`celery-worker` and `celery-beat`, alongside `backend`.
## RLS in background tasks
Celery workers never go through Django's request middleware, so nothing sets the PostgreSQL session
variable `app.current_org` for them the way `RequireOrgContext` does for an HTTP request — see
[PostgreSQL and RLS](postgresql-and-rls.md#verifying-isolation) for the full mechanism. Every task in
`backend/common/tasks.py` that queries an org-scoped table calls `set_rls_context(org_id)`, also
defined in that module, as its first line:
```python
from common.tasks import set_rls_context
@shared_task
def my_task(data_id, org_id):
set_rls_context(org_id) # required before any org-scoped query
...
```
Skip this and the task runs with whatever RLS context the worker's underlying database connection
happens to have — on a fresh connection, that's empty, which (per the fail-safe design described in
[PostgreSQL and RLS](postgresql-and-rls.md#why-rls)) means the query returns zero rows rather than
another org's rows, but it is still a bug: a task that's supposed to act on real data silently does
nothing. `flush_expired_refresh_tokens` is a legitimate case where no context is needed at all: it
operates on simplejwt's `OutstandingToken` table, which isn't in `ORG_SCOPED_TABLES` and carries no
`org_id` column, so there's no per-org filtering to get right in the first place.
`purge_read_notifications` is not the same kind of case, despite its own docstring's claim that "RLS
does not need a per-org context here." `notification` **is** in `ORG_SCOPED_TABLES`
(`backend/common/rls/__init__.py`) and gets the same `FORCE ROW LEVEL SECURITY` policy as every other
org-scoped table, and this task never calls `set_rls_context`. On the non-superuser database role
this project requires, it therefore runs with an empty RLS context, and its `DELETE` matches zero
rows — so instead of purging old read notifications across every org, it silently purges nothing at
all. This is exactly the failure mode described above, not an exception to it: a live bug in this
codebase's own scheduled task, worth knowing if you're relying on it to keep the `notification` table
from growing unbounded.

80
docs/self-hosting/environment-variables.md Normal file
View file

@ -0,0 +1,80 @@
# Environment variables
`backend/crm/settings.py` is read once, at process import time, by every Django entry point —
`manage.py runserver`, `manage.py migrate`, a Gunicorn/Uvicorn worker, a Celery worker, a management
command. This page explains where those values come from and which ones you actually have to set
yourself; the exhaustive per-variable table lives in the Reference section.
## How configuration is loaded
`backend/crm/settings.py` calls `load_dotenv()` (from `python-dotenv`) near the top of the module,
before any setting is read, so a `.env` file next to `manage.py` — created by copying
`backend/.env.example`, per [Manual setup](../getting-started/manual-setup.md) — is loaded into the
process environment automatically. `docker-compose.yml` does the equivalent for containers by
listing `.env.docker` and then the optional, gitignored `.env.docker.local` under each service's
`env_file:`, in that order, so values in `.env.docker.local` win.
Almost every setting in `crm/settings.py` follows the same pattern: `os.environ.get("NAME",
<default>)`, so an unset variable falls back to a working (usually dev-oriented) default rather than
failing. `SECRET_KEY`, `DEBUG`, `ALLOWED_HOSTS`, `DBNAME`/`DBUSER`/`DBPASSWORD`/`DBHOST`/`DBPORT`,
`EMAIL_BACKEND`, `CELERY_BROKER_URL`/`CELERY_RESULT_BACKEND`, `GOOGLE_CLIENT_ID`/`GOOGLE_CLIENT_SECRET`/
`GOOGLE_REDIRECT_URI`, `DOMAIN_NAME`/`FRONTEND_URL`/`SWAGGER_ROOT_URL` are all read this way. A
smaller set of variables has no default at all: `backend/crm/server_settings.py` (imported only when
`ENV_TYPE=prod`, see [Production deployment](production-deploy.md)) reads `AWS_BUCKET_NAME`,
`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_SES_REGION_NAME`, `AWS_SES_REGION_ENDPOINT` and
`SENTRY_DSN` with plain `os.environ[...]` — a missing one is a `KeyError` at import time, which means
the process refuses to start at all rather than starting with something half-configured.
Because all of this runs at import time, not per-request, a changed environment variable only takes
effect the next time the process restarts — editing `.env` while `runserver` is already running does
nothing until you restart it (or, for the Docker Compose setup, until the container restarts, which
happens automatically on `docker compose up --build`).
## The minimum for a working install
The values in `.env.example` and `.env.docker` are enough to bring the stack up and use it locally;
you do not need to touch anything else to follow [Docker quick start](../getting-started/docker-quick-start.md)
or [Manual setup](../getting-started/manual-setup.md). Before anyone other than you can reach the
instance, the variables that actually need a real value are: `SECRET_KEY` (a unique secret, not the
shipped `django-insecure-...` placeholder), `DEBUG=False`, `ALLOWED_HOSTS` (your real hostname(s)),
`CSRF_TRUSTED_ORIGINS` (each entry needs a scheme, e.g. `https://crm.example.com`), `DBUSER` pointed
at a non-superuser PostgreSQL role, and `CELERY_BROKER_URL`/`CELERY_RESULT_BACKEND` pointed at a
Redis instance you control. [Production deployment](production-deploy.md#required-settings) covers
the full list and exactly what each one guards against; [PostgreSQL and RLS](postgresql-and-rls.md)
covers why the database role matters as much as any of the Django settings.
## Database
`DATABASES["default"]` in `crm/settings.py` reads `DBNAME` (default `crm_db`), `DBUSER` (default
`postgres`), `DBPASSWORD` (default `postgres`), `DBHOST` (default `localhost`) and `DBPORT` (default
`5432`). The default `DBUSER` is the PostgreSQL superuser account on most installations, and a
superuser silently bypasses every Row-Level Security policy this project relies on for tenant
isolation — see [PostgreSQL and RLS](postgresql-and-rls.md) for how to create and verify a
non-superuser role before pointing these variables at anything but a local, throwaway database.
## Email
`EMAIL_BACKEND` defaults to `django.core.mail.backends.console.EmailBackend`, so outbound mail
(welcome emails, magic-link sign-in emails, comment-mention notifications) prints to stdout instead
of being sent anywhere. `.env.example` sets `EMAIL_BACKEND` to that same console backend explicitly;
`.env.docker` doesn't set `EMAIL_BACKEND` at all — only a comment noting the console backend is
what's active — so it's relying on the identical default rather than setting it. `DEFAULT_FROM_EMAIL`
and `ADMIN_EMAIL` default to `noreply@localhost`/`admin@localhost`. Switching to a real mail backend,
and the Amazon SES-specific variables that come with it, is covered in full in
[Email and Celery](email-and-celery.md).
## Integrations
`GOOGLE_CLIENT_ID`, `GOOGLE_CLIENT_SECRET` and `GOOGLE_REDIRECT_URI` all default to empty strings in
`crm/settings.py`. `.env.docker` ships all three blank; `.env.example` ships `GOOGLE_CLIENT_ID` and
`GOOGLE_CLIENT_SECRET` blank but omits `GOOGLE_REDIRECT_URI` entirely, relying on the same default.
Google sign-in stays off until `GOOGLE_CLIENT_ID`/`GOOGLE_CLIENT_SECRET` are set, on both the backend
and the SvelteKit frontend. See [Google OAuth](google-oauth.md) for what each one does, where to get
the values, and why `GOOGLE_REDIRECT_URI` specifically doesn't need to be set even in production.
`SENTRY_DSN` is read (with no default) only inside `server_settings.py`, i.e. only when
`ENV_TYPE=prod`; it's not consulted at all otherwise.
## Full reference
Every variable this project reads, its default, and the file that reads it is tabulated in full in
the Reference section.

86
docs/self-hosting/google-oauth.md Normal file
View file

@ -0,0 +1,86 @@
# Google OAuth
BottleCRM supports signing in with Google from both the web app (an authorization-code exchange
with PKCE) and the mobile app (a Google ID token posted directly to the backend). Both are optional:
the deployment works without them, using [magic-link email sign-in](../getting-started/first-sign-in.md)
instead, until you configure a Google OAuth client.
## Registering an OAuth client
Create an OAuth 2.0 Client ID of type "Web application" in the Google Cloud Console. The redirect
URI it needs authorized is built by the frontend's login page
(`frontend/src/routes/(no-layout)/login/+page.server.js`) as `${GOOGLE_LOGIN_DOMAIN}/login` —
`GOOGLE_LOGIN_DOMAIN` is a **frontend** environment variable (`frontend/.env.example` defaults it to
`http://localhost:5173`), so the value to register with Google is `http://localhost:5173/login` for
local development, or `https://<your-frontend-host>/login` in production. There is no separate
redirect registration needed for the mobile flow — it doesn't use a redirect at all (see
[Mobile ID tokens](#mobile-id-tokens) below).
The frontend needs its own copy of the client ID — `frontend/.env.example` sets `GOOGLE_CLIENT_ID`
(client ID only; the comment in that file is explicit that "secret is on backend") — because it
builds the Google authorization URL client-side (`client_id`, `redirect_uri`, PKCE `code_challenge`,
and a CSRF `state` parameter). The client secret only ever lives on the backend.
## Backend configuration
`backend/crm/settings.py` reads three Google-related variables, all defaulting to an empty string:
```python
GOOGLE_CLIENT_ID = os.environ.get("GOOGLE_CLIENT_ID", "")
GOOGLE_CLIENT_SECRET = os.environ.get("GOOGLE_CLIENT_SECRET", "")
GOOGLE_REDIRECT_URI = os.environ.get("GOOGLE_REDIRECT_URI", "")
```
`GOOGLE_CLIENT_ID` and `GOOGLE_CLIENT_SECRET` are what the backend sends to Google's own token
endpoint when exchanging an authorization code (see below) and what it uses as the expected audience
when verifying a mobile ID token — leaving either blank means every such exchange or verification
fails, so sign-in with Google is effectively off until both are set. `.env.docker` ships all three
blank with the comment "leave blank to disable"; `.env.example` doesn't include
`GOOGLE_REDIRECT_URI` at all, only `GOOGLE_CLIENT_ID`/`GOOGLE_CLIENT_SECRET` under an "optional —
leave blank to disable" comment. That's consistent with the code: `GOOGLE_REDIRECT_URI` is defined
in `settings.py` but not read anywhere else in the backend — the redirect URI actually used in the
token exchange is the one the frontend sends in the request body on each call (see
[The PKCE callback flow](#the-pkce-callback-flow)), not this setting.
## The PKCE callback flow
The web flow is the standard OAuth Authorization Code flow with PKCE, split across the frontend and
`GoogleOAuthCallbackView`:
1. `login/+page.server.js` generates a PKCE `code_verifier`/`code_challenge` pair and a random
`state`, stores the verifier and state in short-lived httpOnly cookies, and redirects the browser
to Google's authorization endpoint with the `code_challenge`.
2. Google redirects back to `/login?code=...&state=...`. The frontend checks the returned `state`
against the cookie (CSRF protection) and reads back the stored `code_verifier`, then makes a
server-to-server call — `POST /api/auth/google/callback/`
(`GoogleOAuthCallbackView`, wired at `backend/common/urls.py:67`) — with `code`, `code_verifier`
and `redirect_uri` in the body.
3. `GoogleOAuthCallbackView` exchanges the code for tokens directly with Google
(`https://oauth2.googleapis.com/token`), sending `GOOGLE_CLIENT_ID`, `GOOGLE_CLIENT_SECRET`, the
`redirect_uri` and `code_verifier` it received — this is the step that requires the client secret,
which is why it has to happen on the backend rather than in the browser.
4. The view decodes the `id_token` Google returns (its own code comment notes this is safe without a
signature check specifically because the token came directly from Google over HTTPS in the
previous step, not from an untrusted client) and requires `email_verified` to be true before
trusting the email address at all — an unverified address never provisions or signs in to an
account.
5. It gets-or-creates a `User` by email, rejects with 403 if the account is deactivated
(`user.is_active`), and issues BottleCRM's own JWT pair via
`OrgAwareRefreshToken.for_user_and_org(user, None)`. The `None` org means this token carries no
`org_id` claim — the user still has to pick an organization ([Org
switching](../getting-started/first-sign-in.md#choosing-an-organization)) before any org-scoped
request will work.
## Mobile ID tokens
The mobile app skips the redirect entirely: Google Sign-In on the device returns a Google-issued ID
token, and the app posts it straight to `POST /api/auth/google/`
(`GoogleIdTokenView`, wired at `backend/common/urls.py:69`) as `{"idToken": "..."}`. Unlike the web
callback above, this view does cryptographically verify the token — `google.oauth2.id_token.
verify_oauth2_token(id_token_str, google_requests.Request(), settings.GOOGLE_CLIENT_ID)` — checking
the signature against Google's public keys and the audience against the backend's own
`GOOGLE_CLIENT_ID`, so a token issued for a different client is rejected. It applies the same
`email_verified` check and inactive-account check as the web flow, then returns `JWTtoken`,
`refresh_token` and the list of organizations the user already has a `Profile` in
(`{"id", "name", "role"}` per org) so the mobile app can present an org picker without a second
round trip.

231
docs/self-hosting/postgresql-and-rls.md Normal file
View file

@ -0,0 +1,231 @@
# PostgreSQL and RLS
BottleCRM's multi-tenancy is not just an application-layer convention — every org-scoped table is
protected by PostgreSQL Row-Level Security (RLS), configured centrally in
`backend/common/rls/__init__.py` and enabled by Django migrations. This page explains why that
matters, how to set up the database role RLS depends on, how policies get applied, and how to
verify isolation is actually working rather than assuming it is.
## Why RLS
Every RLS-protected table carries an `org_id` column, and PostgreSQL — not just the Django ORM —
refuses to return or accept rows whose `org_id` doesn't match the current session's org context.
That context is the PostgreSQL session variable `app.current_org`, set by application middleware
(see [Verifying isolation](#verifying-isolation) below) from the caller's authenticated org. The
policies in `get_enable_policy_sql()` apply this filter with `USING (org_id::text = (select
NULLIF(current_setting('app.current_org', true), '')))` — so it's enforced even for a raw SQL
query issued outside the ORM, a query in a Django view that forgot its `.filter(org=...)`, or a
bug in application code. This is defense in depth: `Model.objects.filter(org=request.profile.org)`
in application code is still the contract, RLS is the safety net underneath it.
**The single most important fact in this section: a PostgreSQL superuser role bypasses RLS
entirely, and it does so silently — no error, no log line, nothing.** `manage_rls.py`'s own
`--status` output says as much: `Database user "X" is a SUPERUSER - RLS will be bypassed!`
Every policy still exists and looks correctly configured; a superuser connection simply ignores
them. If your application's database role is a superuser — and the default in
`backend/crm/settings.py` (`DBUSER` defaulting to `postgres`) is exactly that unless you override
it — every RLS policy in your schema is dead weight and tenant isolation depends entirely on
every application code path remembering its own org filter. There is nothing in this codebase
that detects or warns about this at request time; it is only visible if you deliberately run
`manage_rls --verify-user` or `--status`.
## Creating the application role
The application must connect as a **non-superuser** PostgreSQL role. Connected as the `postgres`
superuser (or an equivalent admin account):
```sql
CREATE DATABASE crm_db;
CREATE USER crm_user WITH PASSWORD 'crm_password';
GRANT ALL PRIVILEGES ON DATABASE crm_db TO crm_user;
\connect crm_db;
GRANT ALL ON SCHEMA public TO crm_user;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON TABLES TO crm_user;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON SEQUENCES TO crm_user;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON FUNCTIONS TO crm_user;
```
(This mirrors what `docker/postgres/init-rls-user.sql` runs automatically for the Docker Compose
setup — see [Docker](docker.md) — except that script also runs unconditionally, wrapped in a `DO
$$ ... IF NOT EXISTS ...` block so re-running it on an existing database is safe.)
Granting schema and table privileges (`GRANT ALL ON SCHEMA`, `GRANT ALL ... ON TABLES`) does
**not** weaken RLS — those privileges govern DDL and raw read/write access; only the
`BYPASSRLS` role attribute or superuser status bypasses row-level policies. Do not grant
`BYPASSRLS` to the application role; nothing in this codebase needs it, and there is no
superadmin-style cross-org dashboard in this repository that would justify it.
Point Django at the new role — `DBNAME`, `DBUSER`, `DBPASSWORD`, `DBHOST`, `DBPORT` in your `.env`
(see [Manual setup](../getting-started/manual-setup.md)) or `.env.docker`/`.env.docker.local`
(see [Docker](docker.md)) — then confirm it landed correctly:
```bash
cd backend
uv run python manage.py manage_rls --verify-user
```
This runs `SELECT usename, usesuper, usecreatedb FROM pg_user WHERE usename = current_user` and
prints `Is superuser: False` (and a success line) when configured correctly, or an explicit
`WARNING: Superusers bypass RLS!` with remediation SQL when it isn't.
## Enabling policies
RLS is enabled and disabled **by Django migrations, not by any management command** —
`manage_rls.py`'s own module docstring says so explicitly. Running migrations is what actually
applies policies:
```bash
cd backend
uv run python manage.py migrate
```
The set of tables that get policies is centralized in `ORG_SCOPED_TABLES` in
`backend/common/rls/__init__.py` — at the time of writing this list has 61 entries, spanning core
records (`lead`, `accounts`, `contacts`, `opportunity`, `case`, `task`, `invoice`, ...), supporting
data (`comment`, `attachments`, `document`, `activity`, `tags`, ...), Kanban boards and pipeline
stage tables, invoicing/estimate/recurring-invoice tables, and more. One deliberate exception:
`personal_access_token` is **not** in `ORG_SCOPED_TABLES` — it's looked up by token hash before any
tenant context exists (an "auth-bootstrap" table, the comment in that file calls it, mirroring the
`Org` table itself), and its isolation is enforced instead by explicit `org`/`profile` filters in
`backend/common/views/pat_views.py`.
For each table, `get_enable_policy_sql(table)` runs:
```sql
ALTER TABLE "<table>" ENABLE ROW LEVEL SECURITY;
ALTER TABLE "<table>" FORCE ROW LEVEL SECURITY;
DROP POLICY IF EXISTS org_isolation ON "<table>";
DROP POLICY IF EXISTS org_insert_check ON "<table>";
CREATE POLICY org_isolation ON "<table>"
FOR ALL
USING (org_id::text = (select NULLIF(current_setting('app.current_org', true), '')));
CREATE POLICY org_insert_check ON "<table>"
FOR INSERT
WITH CHECK (org_id::text = (select NULLIF(current_setting('app.current_org', true), '')));
```
The two `DROP POLICY IF EXISTS` statements are why re-running `manage.py migrate` after an
interrupted run is safe rather than something that fails on a duplicate policy — see
[Troubleshooting](troubleshooting.md#migrations-fail) and
[Backups and upgrades](backups-and-upgrades.md#migrations).
`FORCE ROW LEVEL SECURITY` matters specifically because the table owner (typically the role that
ran the migrations) would otherwise be exempt from its own table's RLS policies by default —
forcing it closes that gap without requiring a separate, non-owning application role.
To add RLS to a new org-scoped model: inherit from `BaseOrgModel`, add the table name to
`ORG_SCOPED_TABLES`, and write a migration that calls `get_enable_policy_sql('your_table')` inside
a `RunPython` operation (guarded with `if schema_editor.connection.vendor != 'postgresql': return`,
since SQLite has no concept of row security).
## Verifying isolation
`manage_rls` (`backend/common/management/commands/manage_rls.py`) is a diagnostic command with
three flags, each proving something different — none of them modify the database:
```bash
cd backend
uv run python manage.py manage_rls --status
uv run python manage.py manage_rls --verify-user
uv run python manage.py manage_rls --test
```
- **`--status`** proves *which tables have policies applied, and whether the connected role is a
superuser*. It queries `current_user`/`usesuper` from `pg_user`, then for every table in
`ORG_SCOPED_TABLES` checks `pg_class.relrowsecurity`/`relforcerowsecurity` and prints
`ENABLED (forced)`, `disabled`, or `TABLE NOT FOUND`, ending with an `Enabled: N, Disabled: N`
tally.
- **`--verify-user`** proves *the connected database role cannot bypass RLS*. It's the
`--status` command's superuser check in isolation, with explicit remediation SQL printed if the
role turns out to be a superuser.
- **`--test`** proves *isolation actually holds for real data*, not just that policies exist. It
picks up to two orgs that have `lead` rows (falling back to the first two orgs in the database
if none has leads yet, and skipping entirely if fewer than two orgs exist at all), sets
`app.current_org` to each in turn and counts visible leads, then sets the context to an **empty
string** and counts again. Seeing non-zero counts for the two real orgs and zero for the empty
context is the actual proof of isolation the other two flags can't give you.
**Why the empty-context count matters:** the session variable is `app.current_org`, and it's set
by `RequireOrgContext` (`backend/common/middleware/rls_context.py`, wired into `MIDDLEWARE` in
`backend/crm/settings.py`) from `request.org` — which is itself derived earlier in the same
middleware chain by `GetProfileAndOrg` (`backend/common/middleware/get_company.py`) from the
`org_id` claim inside the caller's signed JWT, validated against an active `Profile` for that
user in that org before it's trusted. (The file also defines a second, older middleware class,
`SetOrgContext`, referenced in that file's own module docstring — but `settings.py` wires
`RequireOrgContext` instead, which additionally rejects unauthenticated/context-less requests
with a 403 rather than silently proceeding without a context; `RequireOrgContext` is the class
actually enforced.) Every policy uses `NULLIF(current_setting('app.current_org', true), '')` —
when no context has been set, `current_setting` returns an empty string, `NULLIF` turns that into
SQL `NULL`, and `org_id::text = NULL` is never true. **No context means zero rows, by design** —
a fail-safe default, not an edge case to special-case around.
`RequireOrgContext` resets the variable to `''` after every request (`SELECT set_config
('app.current_org', '', false)`), specifically to stop it leaking between requests on pooled
connections.
**Celery workers never run this middleware.** Background tasks in `backend/common/tasks.py` that
touch org-scoped tables call `set_rls_context(org_id)` — also defined in `common/tasks.py` — as
their first line, before any ORM query:
```python
from common.tasks import set_rls_context
@shared_task
def my_task(data_id, org_id):
set_rls_context(org_id) # required before any org-scoped query
...
```
`set_rls_context` is a thin wrapper around the same `set_config('app.current_org', ...)` call the
middleware uses, with an early return on non-PostgreSQL connections (so it's a no-op under the
SQLite test settings). A task that skips this and queries an RLS-protected table runs with
whatever context the underlying connection happened to have — on a fresh worker connection, that's
empty, which (per the design above) returns zero rows rather than the wrong org's rows; but it is
still a bug worth catching, since "silently returns nothing" produces its own class of confusing
failures. Not every task in `common/tasks.py` needs this: `flush_expired_refresh_tokens` operates on
simplejwt's `OutstandingToken`, which has no `org` column and is not in `ORG_SCOPED_TABLES`, so no
context applies to it.
`purge_read_notifications` is the counter-example, and it is worth studying because it looks like the
same case and is not. Its docstring asserts that RLS needs no per-org context here, but `notification`
*is* in `ORG_SCOPED_TABLES`, so on the non-superuser role this page tells you to use, that task runs
with an empty context and its delete matches nothing. Read notifications are never purged. Take it as
the practical warning: a docstring claiming a task is exempt is not evidence, and the failure is
silent — the task reports success having done nothing at all.
## Common mistakes
- **Leaving the default database role as `postgres`.** Nothing enforces a non-superuser role for
you; `backend/crm/settings.py`'s `DATABASES` setting defaults `DBUSER` to `postgres` if the
environment variable is unset. That default is a superuser almost everywhere PostgreSQL is
installed. Run `manage_rls --verify-user` after any environment change, not just once at setup.
- **Trusting a green local/dev run as proof RLS works.** If your development database user is a
superuser (very common — it's the path of least resistance to "just get it running"), every
endpoint will appear to isolate tenants correctly purely because application-level `.filter(org=
...)` calls happen to be correct, while RLS itself is doing nothing. Only `--verify-user` (or
`--status`) surfaces this; functional testing cannot.
- **Forgetting `set_rls_context(org_id)` in a new Celery task** that queries an org-scoped model —
see [Verifying isolation](#verifying-isolation) above.
- **Adding a new org-scoped model without registering it.** `BaseOrgModel` gives you the `org`
foreign key, but RLS policies only get created for tables listed in `ORG_SCOPED_TABLES` with a
migration that calls `get_enable_policy_sql`. A table can carry real `org_id` data and simply
have no RLS protection if this step is skipped — `manage_rls --status` only reports on tables
that are in that list in the first place, so an unregistered table won't even show up as
"disabled."
- **Assuming the anonymous client-portal endpoints (invoice/estimate links, CSAT surveys) are
either broken or unprotected — they're a deliberate, narrow exception, and testing them locally
can be misleading either way.** `RequireOrgContext.EXEMPT_PATHS` in
`backend/common/middleware/rls_context.py` exempts exactly `/api/public/invoice/`,
`/api/public/estimate/` and `/api/public/csat/` from requiring org context, because these are
genuinely anonymous requests (a customer clicking an emailed link) authorized by a possession
token rather than a JWT — see `backend/docs/PORTAL_RLS.md`. Exemption from *requiring* context
is not the same as *setting* one: `_set_org_context` returns early when `request.org` is `None`,
so these requests run with `app.current_org` empty — which, per the fail-safe design above,
means the `invoice`/`estimate`/`csat_survey` rows the view is trying to read are invisible under
RLS too, until the view itself resolves the org from the token and calls `set_rls_context`
explicitly. This is exactly the kind of thing a superuser dev database masks: with `DBUSER=
postgres`, RLS does nothing, so a locally-broken or locally-fine portal endpoint looks identical.
Test portal endpoints against a real non-superuser role before trusting them.

118
docs/self-hosting/production-deploy.md Normal file
View file

@ -0,0 +1,118 @@
# Production deployment
Neither `docker-compose.yml` nor the repo-root `Dockerfile` builds a production-hardened
deployment out of the box — see the warning in [Docker](docker.md). This page covers what
changes: the process that serves requests, how static files are handled, what sits in front of
Django, and which environment variables the codebase itself treats as security-relevant. It
assumes you already have a working install per [Manual setup](../getting-started/manual-setup.md)
or [Docker](docker.md) and a PostgreSQL role configured per
[PostgreSQL and RLS](postgresql-and-rls.md).
## Before you start
There is no bundled production `docker-compose` file, systemd unit, or process manager
configuration in this repository — the `Dockerfile` builds an image (`backend/Dockerfile` puts
`uv`-managed dependencies and the venv's `bin/` on `PATH`, so `gunicorn`, `celery`, and `python`
all resolve inside it), but what command that image *runs* is up to you: `docker-compose.yml`'s
`backend` service overrides it with `/entrypoint.sh`, which calls `manage.py runserver` — the
Django development server. For a production container or host process you supply your own start
command (see [Application server](#application-server) below) instead of that entrypoint script.
Whatever you run it as, apply migrations and collect static files before serving traffic:
```bash
cd backend
uv run python manage.py migrate
uv run python manage.py collectstatic --noinput
```
## Application server
`backend/pyproject.toml` lists `gunicorn>=26.0.0` under its dependencies, commented "Production
WSGI server" — Gunicorn is the WSGI server this project ships as a dependency for serving the
Django app itself. `backend/crm/wsgi.py` exposes the standard `application` callable, so from
`backend/`:
```bash
cd backend
uv run gunicorn crm.wsgi:application --bind 0.0.0.0:8000
```
There is one documented exception to WSGI, in the codebase's own words. `backend/crm/asgi.py`
carries this docstring:
> Production deploy must run an ASGI server pointing at this module:
> `uvicorn crm.asgi:application --host 0.0.0.0 --port 8000`
The reason given is the in-app notifications SSE (server-sent events) stream: it's an async view
serving a long-lived connection, and running it under WSGI (Gunicorn) "will hold a worker
hostage" for the life of each connection. `crm/asgi.py` also optionally mounts the BottleCRM MCP
server at `/mcp` when the `mcp` extra is installed and `BCRM_MCP_ENABLED` isn't set to a falsy
value. `uvicorn[standard]>=0.51.0` is a listed dependency alongside Gunicorn for exactly this
path. In short: if you need the SSE notification stream (or the MCP mount) to work correctly
under concurrent load, serve via `uvicorn crm.asgi:application`, not Gunicorn/WSGI; if you don't,
Gunicorn against `crm.wsgi:application` is what this project bundles as its production WSGI
dependency.
## Static files
`whitenoise.middleware.WhiteNoiseMiddleware` is in `MIDDLEWARE` (immediately after
`SecurityMiddleware`), and `STATICFILES_STORAGE` is set to
`whitenoise.storage.CompressedManifestStaticFilesStorage`. This means Django itself serves static
assets — compressed and content-hashed — directly from the WSGI/ASGI process; you do not need a
separate static file server or a reverse-proxy `location /static/` block for correctness. Static
files are collected to `STATIC_ROOT` (`backend/staticfiles/`, i.e. `BASE_DIR/staticfiles`) under
`STATIC_URL = "/static/"` by `manage.py collectstatic`, which you must run yourself — Gunicorn
does not run it for you (the Docker Compose entrypoint does, but only because
`docker/backend/entrypoint.sh` calls it explicitly on every start).
Media (user-uploaded files — invoice attachments, case attachments, and similar) is separate from
static files and is not covered by WhiteNoise. With `ENV_TYPE=dev` (the default),
`MEDIA_ROOT`/`MEDIA_URL` are a local `media/` directory served through Django itself; with
`ENV_TYPE=prod`, `backend/crm/server_settings.py` is imported instead and switches
`DEFAULT_FILE_STORAGE` to `storages.backends.s3boto3.S3Boto3Storage`, reading `AWS_BUCKET_NAME`,
`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_SES_REGION_NAME`, `AWS_SES_REGION_ENDPOINT` and
`SENTRY_DSN` from the environment with `os.environ[...]` — a plain `KeyError` at import time, and
the process refusing to start, if any of them are unset. It also switches `EMAIL_BACKEND` to
`django_ses.SESBackend` and hardcodes `SESSION_COOKIE_DOMAIN = ".bottlecrm.io"`. That domain is
specific to the project's own SaaS hosting, not to your deployment — setting `ENV_TYPE=prod`
without also being ready to supply AWS S3, SES and Sentry credentials, and without meaning to
scope session cookies to `bottlecrm.io`, will not do what you want. See
[Required settings](#required-settings) below for the practical consequence.
## Reverse proxy
Nothing in `backend/crm/settings.py` terminates TLS, and `SECURE_PROXY_SSL_HEADER` is not set
anywhere in this codebase's settings modules. Concretely: if you terminate TLS at a reverse proxy
(nginx, Caddy, or similar) and forward plain HTTP to Gunicorn/Uvicorn, Django's
`request.is_secure()` will report `False` for every request unless you add
`SECURE_PROXY_SSL_HEADER` yourself (typically pointing at the `X-Forwarded-Proto` header your
proxy sets) — this project does not configure that for you. `SECURE_HSTS_SECONDS = 31536000`
(one year), `SECURE_HSTS_INCLUDE_SUBDOMAINS = True`, `SECURE_HSTS_PRELOAD = True` and
`SECURE_CONTENT_TYPE_NOSNIFF = True` are all set unconditionally in `backend/crm/settings.py`, but
Django's `SecurityMiddleware` only emits the `Strict-Transport-Security` header on requests it
considers secure — so without `SECURE_PROXY_SSL_HEADER` configured correctly for your proxy, that
HSTS configuration silently never takes effect.
Point your proxy at whichever port your application server binds (`8000` in the examples above),
and route `/` there. There's no repo-provided proxy config (nginx site file, Caddyfile, or
similar) to point to — the specifics are yours to write.
## Required settings
`backend/crm/settings.py` reads these from the environment at import time (i.e. at process
startup, not per-request):
| Variable | Behavior |
|---|---|
| `SECRET_KEY` | Defaults to the literal string `"django-insecure-dev-key-please-change-in-production"` if unset. The module itself checks this: if `SECRET_KEY` is empty or starts with `"django-insecure"` **and** `ENV_TYPE != "dev"`, it raises `ValueError("SECRET_KEY must be set to a secure value in non-dev environments")` at import time — a hard startup failure, not a warning. This guard only fires when `ENV_TYPE` is set to something other than `dev`; given the `ENV_TYPE=prod` consequences described above (S3/SES/Sentry env vars becoming mandatory, the hardcoded `.bottlecrm.io` cookie domain), don't rely on that check alone — set a real, unique `SECRET_KEY` regardless of `ENV_TYPE`. |
| `DEBUG` | Must be `False` in production. Read as `os.environ.get("DEBUG", "False").lower() == "true"` — defaults to `False` already, so the failure mode to avoid is *explicitly* setting `DEBUG=True` (as `.env.docker` does for local development) and carrying that into production. |
| `ALLOWED_HOSTS` | A comma-separated environment variable: `os.environ.get("ALLOWED_HOSTS", "localhost,127.0.0.1").split(",")`. Set it to the hostname(s) you'll actually be served on, e.g. `ALLOWED_HOSTS=crm.example.com`. |
| `CSRF_TRUSTED_ORIGINS` | Also comma-separated: `os.environ.get("CSRF_TRUSTED_ORIGINS", "").split(",")`, empty by default. Django requires each entry to include a scheme (e.g. `https://crm.example.com`), not a bare hostname. |
A few more the same module reads, worth setting deliberately rather than leaving at their
defaults: `CORS_ALLOWED_ORIGINS` (comma-separated, defaults to the two local dev frontend
origins) and `CORS_ALLOW_ALL` (`CORS_ORIGIN_ALLOW_ALL`, `.env.docker` sets this `True` for local
development — leave it `False` in production and rely on `CORS_ALLOWED_ORIGINS` instead), and
`DOMAIN_NAME`/`FRONTEND_URL`, which are interpolated into emails (magic links, notifications) and
so need to point at your real, public hostnames.

65
docs/self-hosting/requirements.md Normal file
View file

@ -0,0 +1,65 @@
# Requirements
What has to be true about your environment before you follow [Docker](docker.md) or
[Production deployment](production-deploy.md). This page only covers versions, hardware shape and
services — see [Manual setup](../getting-started/manual-setup.md) for the commands that install
them.
## Supported versions
| Component | Version | Source |
|---|---|---|
| Python | 3.12 or later | `backend/pyproject.toml` sets `requires-python = ">=3.12"`; `backend/.python-version` pins `3.12` exactly. |
| Django | 6.0.7 or later | `backend/pyproject.toml` dependency `django>=6.0.7`. |
| PostgreSQL | 16 | `docker-compose.yml`'s `db` service uses image `postgres:16-alpine`. This is also the only version this project's own CI ever runs against — see the note below. |
| Redis | 7 | `docker-compose.yml`'s `redis` service uses image `redis:7-alpine`. |
| Node.js | 24 | `.github/workflows/tests.yml`'s `frontend-checks` job sets up `node-version: '24'`. |
| pnpm | 10 | Same job pins the pnpm action to `version: 10`. |
Backend dependencies (`djangorestframework`, `djangorestframework-simplejwt`, `celery`,
`weasyprint`, `gunicorn`, and the rest) are locked in `backend/uv.lock`; `uv sync` from
`backend/` installs the exact versions CI and this documentation were checked against.
**PostgreSQL is not optional.** BottleCRM's multi-tenancy is enforced by PostgreSQL Row-Level
Security (RLS), keyed on the `app.current_org` session variable — see
[PostgreSQL and RLS](postgresql-and-rls.md) for how that works and why it matters. RLS is a
PostgreSQL feature with no SQLite equivalent. The only place SQLite appears in this codebase is
`backend/crm/test_settings.py`, which swaps in an in-memory SQLite database for the test suite
specifically because "RLS ... is PostgreSQL-only and is skipped on SQLite" (its own docstring).
Running the application itself against SQLite is not supported by any settings module in this
repository.
That gap is also why `.github/workflows/tests.yml`'s `backend-tests` job runs
`pytest -m "not postgres_only"` against SQLite and explicitly excludes every test marked
`postgres_only` — those tests need a real PostgreSQL connection to exercise RLS and don't run in
that job at all. Treat PostgreSQL 16, as pinned in `docker-compose.yml`, as the version this
project actually exercises; nothing in this repository documents support for, or testing against,
any other major version.
## Hardware
There is no benchmarked sizing guide in this repository, and this page won't invent one. What you
do need to plan capacity for is the process list a full self-hosted stack runs concurrently —
`docker-compose.yml` defines six long-running services: `db` (PostgreSQL), `redis`, `backend`
(the Django API), `celery-worker`, `celery-beat`, and `frontend`. All but `frontend` (only needed
if you're serving the SvelteKit dev server rather than a built static bundle) are required for the
API and background jobs to function; running them all on one host means sizing for a database, a
key-value store, an application server and a task worker at once, not just the Django process.
`backend/Dockerfile` also installs Cairo/Pango/gdk-pixbuf (WeasyPrint's PDF-rendering
dependencies) — PDF generation for invoices and estimates runs synchronously in the request or
task that triggers it, so factor that into whatever process handles it.
## Required services
- **PostgreSQL 16** — the only supported database backend (`django.db.backends.postgresql` is the
sole `ENGINE` any non-test settings module in this repository sets). Required for RLS-based
tenant isolation; see [PostgreSQL and RLS](postgresql-and-rls.md).
- **Redis** — `backend/crm/settings.py` reads `CELERY_BROKER_URL` and `CELERY_RESULT_BACKEND`,
both defaulting to `redis://localhost:6379/0`. Redis backs both the Celery message broker and
the result backend; without it, `celery -A crm worker` and `celery -A crm beat` have nothing to
connect to.
Nothing else is required to run the API itself. Email delivery (console backend by default, SES
optionally), Google OAuth, and Sentry are all read from environment variables with working
defaults or an explicit off-state — see [Production deployment](production-deploy.md) for what
each of those needs when you turn it on.

101
docs/self-hosting/security-hardening.md Normal file
View file

@ -0,0 +1,101 @@
# Security hardening
BottleCRM is multi-tenant: every deployment holds more than one organization's data behind the same
database and the same application process. This page collects the settings and surfaces that matter
most once you're running something other people depend on, most of which are covered in more depth
elsewhere in Self-hosting — this page is the checklist, with pointers to the detail.
## Production checklist
- **`DEBUG=False`.** `crm/settings.py` reads it as `os.environ.get("DEBUG", "False").lower() ==
"true"`, so it's already `False` by default — the failure mode to avoid is `.env.docker` (which
ships `DEBUG=True` for local development) carrying into a production environment unchanged.
- **A real `SECRET_KEY`**, not the shipped `django-insecure-...` placeholder — see
[Secrets](#secrets) below.
- **`ALLOWED_HOSTS`** set to your real hostname(s), and **`CSRF_TRUSTED_ORIGINS`** set with a scheme
(`https://crm.example.com`, not a bare hostname) — both detailed in
[Production deployment](production-deploy.md#required-settings).
- **`CORS_ALLOW_ALL=False`** (the default) with **`CORS_ALLOWED_ORIGINS`** scoped to your real
frontend origin(s); `.env.docker` sets `CORS_ALLOW_ALL=True` for local development, which should
not carry into production either.
- **`SECURE_PROXY_SSL_HEADER`** added yourself if you terminate TLS at a reverse proxy — nothing in
`crm/settings.py` sets it, so without it Django never considers a proxied request secure and the
HSTS headers this project does set unconditionally (`SECURE_HSTS_SECONDS`,
`SECURE_HSTS_INCLUDE_SUBDOMAINS`, `SECURE_HSTS_PRELOAD`) silently never get sent — see
[Production deployment](production-deploy.md#reverse-proxy).
- **A non-superuser database role** — see [Database role](#database-role) below.
- **Don't set `ENV_TYPE=prod` casually.** It's not just a flag: it makes `AWS_BUCKET_NAME`,
`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_SES_REGION_NAME`, `AWS_SES_REGION_ENDPOINT` and
`SENTRY_DSN` mandatory (a `KeyError` at startup if any is missing), switches email to Amazon SES,
and hardcodes `SESSION_COOKIE_DOMAIN = ".bottlecrm.io"` — a domain specific to this project's own
SaaS hosting. See [Production deployment](production-deploy.md#static-files) before setting it.
## Database role
The single highest-leverage setting in this list: `DBUSER` must not be a PostgreSQL superuser, or
every Row-Level Security policy in the schema is bypassed silently, with no error and no log line.
The default (`DBUSER` unset, falling back to `postgres`) is a superuser on most installations. Create
a dedicated role and confirm it isn't one:
```bash
cd backend
uv run python manage.py manage_rls --verify-user
```
Full detail — how to create the role, how RLS policies are applied, and how to prove isolation holds
for real data rather than trusting that it does — is in
[PostgreSQL and RLS](postgresql-and-rls.md).
## Secrets
`SECRET_KEY` does more than Django's usual signing duties here: `SIMPLE_JWT["SIGNING_KEY"]` in
`crm/settings.py` is set to `SECRET_KEY` directly, so it is also the key every access and refresh
token is signed and verified with. There is no separate JWT signing-key setting anywhere in this
codebase — rotating `SECRET_KEY` invalidates every outstanding access and refresh token immediately,
which is worth knowing before you rotate it as a routine maintenance step rather than in response to
an actual compromise.
`crm/settings.py` itself guards against the worst case of forgetting to set it: if `SECRET_KEY` is
empty or still starts with `"django-insecure"` **and** `ENV_TYPE` is anything other than `dev`, the
module raises `ValueError("SECRET_KEY must be set to a secure value in non-dev environments")` at
import time — a hard startup failure rather than a silent insecure default. That guard only fires
when `ENV_TYPE` is explicitly non-`dev`, so set a real, unique `SECRET_KEY` regardless of whether you
ever set `ENV_TYPE=prod`.
## Token surfaces
Three different credential types exist, with different blast radii if one leaks:
- **Personal access tokens** (`bcrm_pat_...`, managed at `profile/tokens/`) are scoped to the one
profile that created them. `resolve_valid_pat()` in `backend/common/pat_auth.py` checks
`pat.profile.is_active` (and `pat.org.is_active`) on every use, so deactivating a profile cuts off
its personal access tokens immediately — there's no separate token-revocation step required when
offboarding someone.
- **Admin oversight of tokens** is separate from the self-service surface above: `org/tokens/`
(`OrgAccessTokenListView`/`OrgAccessTokenDetailView`) is admin-only and org-wide — an admin can see
and revoke any token in their org, including one belonging to a colleague who has since been
deactivated. It's deliberately a distinct endpoint from `profile/tokens/` rather than a widened
version of it, so the self-service guard never has to be loosened to support oversight.
- **The org API key** (`GET`/`POST /api/org/api-key/`, `OrgApiKeyView`, admin-only) is the highest-
privilege of the three: `common/middleware/get_company.py`'s `_process_api_key_auth` resolves a
request bearing this key to the org's first active `ADMIN` profile, so a leaked org API key is
equivalent to a leaked admin session for that org, not a leaked user session. It's kept out of
every nested API representation and served only by this endpoint. If it's ever exposed — committed
to a repo, logged, shared over an insecure channel — rotate it with `POST /api/org/api-key/`; the
response is explicit that "the previous key is no longer valid" immediately on rotation.
## What to monitor
`SecurityAuditLog` (`backend/common/audit_log.py`, table `security_audit_log`) is an RLS-protected,
per-org table of security-relevant events, with a matching `security.audit` Python logger that also
writes to `security_audit.log` (configured in `crm/settings.py`'s `LOGGING` dict). It's worth
watching, with one caveat: of the thirteen event types the model defines
(`LOGIN_SUCCESS`, `LOGIN_FAILURE`, `LOGOUT`, `ORG_SWITCH`, `TOKEN_REFRESH`, `TOKEN_REVOKED`,
`PERMISSION_DENIED`, `CROSS_ORG_ATTEMPT`, `API_KEY_USED`, `API_KEY_INVALID`, `MEMBERSHIP_REVOKED`,
`SUSPICIOUS_ACTIVITY`, `SAMPLE_DATA_CLEARED`), only `LOGIN_SUCCESS`, `ORG_SWITCH`, `TOKEN_REFRESH`,
`TOKEN_REVOKED`, `PERMISSION_DENIED` (from one call site) and `SAMPLE_DATA_CLEARED` are actually
logged anywhere in the application code as of this writing. `LOGIN_FAILURE`, `LOGOUT`,
`CROSS_ORG_ATTEMPT`, `API_KEY_USED`, `API_KEY_INVALID`, `MEMBERSHIP_REVOKED` and
`SUSPICIOUS_ACTIVITY` are defined on the model and have corresponding methods on the `AuditLogger`
helper, but nothing in the codebase currently calls them — don't rely on this table to surface a
brute-forced login or a bad API key attempt today; it doesn't yet.

109
docs/self-hosting/troubleshooting.md Normal file
View file

@ -0,0 +1,109 @@
# Troubleshooting
Five symptoms that come up often enough in a multi-tenant, RLS-backed deployment to be worth their
own page, each with the specific cause and a command that confirms it rather than a guess.
## Every list is empty
Two distinct, verifiable causes, depending on where you're looking:
**Through the API**, `HasOrgContext` (`backend/common/permissions.py`) is the permission class most
list/detail views use alongside `IsAuthenticated`. It denies with a `403` and the message
`"Organization context is required. Please login again."` whenever `request.profile` or
`request.org` is unset — which happens when the JWT you're sending has no `org_id` claim (a token
minted without picking an org yet — see [First sign-in](../getting-started/first-sign-in.md#choosing-an-organization))
or belongs to a profile that's no longer active in that org. That response body is the confirmation:
if you're getting `403` with that message rather than an empty `200`, this is it, and the fix is to
obtain a token with the right org bound (via `/api/auth/switch-org/`, or `devlogin --org` in
development).
**Through Django admin** (`/admin/`), the failure looks different and is easy to miss: `/admin/` is
listed in `RequireOrgContext.EXEMPT_PATHS` (`backend/common/middleware/rls_context.py`), which
exempts it from *requiring* org context — but exemption from requiring a context is not the same as
*setting* one, and nothing else sets `app.current_org` for an admin-authenticated request either. On
a correctly configured, non-superuser database role, every RLS-protected model registered in Django
admin (`Lead`, `Task`, `Board`, `Invoice`, and others — see each app's `admin.py`) will show **zero
rows in the admin list view even though the data exists**, because the RLS policy's
`NULLIF(current_setting('app.current_org', true), '')` evaluates to `NULL` with no context set, and
`org_id::text = NULL` is never true. Confirm this by comparing counts: the same model returns real
data through `GET /api/leads/` (which does have org context) but nothing through the `Lead` admin
changelist at `/admin/leads/lead/` (`/admin/leads/` alone is the app index, listing model names, not
rows — it won't show this). This is expected behavior, not a bug to fix — Django admin was never
wired to set RLS context, and doing so would need to resolve which org an admin-site request is
"for," which the admin UI itself has no concept of.
## Tenant data is visible across organizations
This is the fail-safe design inverted, and it has one cause in this codebase: the database role
Django connects as is a PostgreSQL superuser, and superusers bypass Row-Level Security entirely,
silently — no error, no log line, every policy still exists and looks correctly configured. Confirm
it directly:
```bash
cd backend
uv run python manage.py manage_rls --verify-user
```
This prints `Is superuser: True` and explicit remediation SQL if the role is a superuser, or a
success line if it isn't. The default `DBUSER` (unset, falling back to `postgres`) is a superuser on
most PostgreSQL installations — see [PostgreSQL and RLS](postgresql-and-rls.md#creating-the-application-role)
for how to create and switch to a dedicated non-superuser role, and don't trust a green local/dev
environment as proof this is configured correctly: a superuser dev database makes every endpoint
*look* correctly isolated purely because the application's own `.filter(org=...)` calls happen to be
right, while RLS itself does nothing underneath them.
## Celery tasks see no data
A Celery worker never runs Django's request middleware, so nothing sets `app.current_org` for it the
way `RequireOrgContext` does for an HTTP request. Every task in `backend/common/tasks.py` that
queries an org-scoped table is expected to call `set_rls_context(org_id)`, also defined in that
module, as its first line before touching the ORM. A task that queries an RLS-protected model
without calling it first runs with an empty RLS context on a fresh worker connection, which — per
the same fail-safe design as above — means the query returns **zero rows**, not the wrong org's
rows, but it does mean the task silently does nothing useful. Confirm by reading the task itself for
a `set_rls_context(...)` call before its first org-scoped query; there's no runtime flag or log line
that surfaces this automatically. See [Email and Celery](email-and-celery.md#rls-in-background-tasks)
for the pattern and the two tasks that intentionally skip it.
## Sign-in redirects in a loop
The SvelteKit frontend stores JWT tokens in httpOnly cookies whose `secure` flag is gated on
`NODE_ENV`/`env.NODE_ENV === 'production'` — this pattern repeats across
`frontend/src/hooks.server.js` and every `+page.server.js` that sets an auth-related cookie (login,
magic-link verify, org selection). If you set `NODE_ENV=production` on the Node process serving the
frontend while the site is actually reached over plain HTTP (no TLS terminated in front of it, or a
proxy that isn't forwarding correctly), every cookie gets written with the `Secure` attribute, and
the browser silently refuses to store or send a `Secure` cookie over a non-HTTPS connection. The
result: sign-in appears to succeed (the OAuth/magic-link exchange completes and the server tries to
set the cookies), but the next request arrives with no `jwt_access` cookie, `hooks.server.js` treats
it as unauthenticated, and redirects back to `/login` — which the user completes again, with the
same outcome. Confirm it by inspecting the `Set-Cookie` response header in the browser's network
tab: a `Secure` attribute on a response served over `http://` is the signature of this exact failure.
The fix is to serve the frontend over HTTPS if `NODE_ENV=production` is set, or leave `NODE_ENV`
unset/`development` until TLS is actually in front of it.
## Migrations fail
`manage.py migrate` is also what applies and re-applies Row-Level Security policies — see
[PostgreSQL and RLS](postgresql-and-rls.md#enabling-policies). Two things about how those specific
migrations are written are worth knowing if `migrate` fails partway through: they run with
`atomic = False` (every migration that calls `get_enable_policy_sql` carries the comment "RLS policy
creation can't run inside an atomic block"), and the SQL those migrations run always
`DROP POLICY IF EXISTS` before `CREATE POLICY`, so re-running `manage.py migrate` after an
interrupted run is safe rather than something that fails on a duplicate policy.
`ALTER TABLE ... ENABLE ROW LEVEL SECURITY`, which these migrations run, requires the connecting
role to either be the table's owner or a superuser. As long as the same non-superuser role runs
every migration for a given database from the very first one — the setup this project documents in
[PostgreSQL and RLS](postgresql-and-rls.md#creating-the-application-role) — that role owns every
table it creates and this is never an issue. It becomes one if you provision the schema differently:
for example, running the first `migrate` as the `postgres` superuser and only switching `DBUSER` to
a dedicated non-superuser role afterward leaves that role without ownership of the already-created
tables, and a later migration's `ALTER TABLE` fails with a permission error naming the table. If
`migrate` fails this way, check which role owns the affected table (`\dt+ <table>` in `psql`) against
the role in `DBUSER`.
Separately, if you've forked this project and modified a model: this project's own CI runs
`manage.py makemigrations --check --dry-run` before its test suite specifically to catch a model
change that never got its migration generated. Run that same command locally after any model
change.

112
mkdocs.yml Normal file
View file

@ -0,0 +1,112 @@
site_name: BottleCRM
site_description: Self-hosted, multi-tenant CRM — setup, self-hosting, API and development documentation.
site_url: https://bottlecrm.readthedocs.io/
repo_url: https://github.com/django-crm/Django-CRM
repo_name: django-crm/Django-CRM
edit_uri: edit/master/docs/
docs_dir: docs
# docs/ also holds internal specs, plans and design references. MkDocs copies
# every file under docs_dir into the built site regardless of nav, so these
# must be excluded explicitly or they are published.
exclude_docs: |
superpowers/
design/
theme:
name: material
features:
- navigation.sections
- navigation.top
- navigation.instant
- content.code.copy
- content.action.edit
- search.suggest
- toc.follow
palette:
- media: "(prefers-color-scheme: light)"
scheme: default
toggle:
icon: material/weather-night
name: Switch to dark mode
- media: "(prefers-color-scheme: dark)"
scheme: slate
toggle:
icon: material/weather-sunny
name: Switch to light mode
markdown_extensions:
- admonition
- attr_list
- def_list
- tables
- toc:
permalink: true
- pymdownx.details
- pymdownx.superfences
- pymdownx.tabbed:
alternate_style: true
- pymdownx.highlight:
anchor_linenums: true
nav:
- Home: index.md
- Getting started:
- Docker quick start: getting-started/docker-quick-start.md
- Manual setup: getting-started/manual-setup.md
- First sign-in: getting-started/first-sign-in.md
- Demo data and packs: getting-started/demo-data.md
- Self-hosting:
- Requirements: self-hosting/requirements.md
- Docker: self-hosting/docker.md
- Production deployment: self-hosting/production-deploy.md
- PostgreSQL and RLS: self-hosting/postgresql-and-rls.md
- Environment variables: self-hosting/environment-variables.md
- Email and Celery: self-hosting/email-and-celery.md
- Google OAuth: self-hosting/google-oauth.md
- Backups and upgrades: self-hosting/backups-and-upgrades.md
- Security hardening: self-hosting/security-hardening.md
- Troubleshooting: self-hosting/troubleshooting.md
- Architecture:
- Overview: architecture/overview.md
- Multi-tenancy and RLS: architecture/multi-tenancy-and-rls.md
- Authentication: architecture/authentication.md
- Permissions and roles: architecture/permissions-and-roles.md
- Data model: architecture/data-model.md
- Background jobs: architecture/background-jobs.md
- API reference:
- Conventions: api/conventions.md
- Authentication: api/authentication.md
- Errors: api/errors.md
- Tokens and API keys: api/tokens-and-api-keys.md
- Leads: api/leads.md
- Accounts: api/accounts.md
- Contacts: api/contacts.md
- Opportunities: api/opportunities.md
- Cases: api/cases.md
- Tasks: api/tasks.md
- Invoices and estimates: api/invoices.md
- Endpoint index: api/endpoint-index.md
- Integrations:
- MCP server: integrations/mcp-server.md
- Inbound email: integrations/inbound-email.md
- CSV import: integrations/csv-import.md
- Python sample: integrations/python-sample.md
- JavaScript sample: integrations/javascript-sample.md
- Mobile app:
- Build and configure: mobile/build-and-configure.md
- Connect to self-hosted: mobile/connect-to-self-hosted.md
- Release builds: mobile/release-builds.md
- Contributing:
- Development setup: contributing/development-setup.md
- Repository layout: contributing/repo-layout.md
- Testing: contributing/testing.md
- Code style: contributing/code-style.md
- Adding an org-scoped model: contributing/adding-an-org-scoped-model.md
- Security rules: contributing/security-rules.md
- Pull requests: contributing/pull-requests.md
- Reference:
- Environment variables: reference/environment-variables.md
- Management commands: reference/management-commands.md
- Glossary: reference/glossary.md