docs: add AGENTS.md for repo, backend and frontend
This commit is contained in:
@@ -0,0 +1,151 @@
|
||||
# Chitai backend
|
||||
|
||||
Litestar REST API for the eBook library. See the repo-root `AGENTS.md` for the overall picture and
|
||||
dev-environment setup.
|
||||
|
||||
**Stack:** Python 3.13 · Litestar 2 · advanced-alchemy over async SQLAlchemy 2 · asyncpg ·
|
||||
PostgreSQL 17 · Alembic (through advanced-alchemy's `alchemy` CLI) · pydantic-settings · uv.
|
||||
|
||||
## Layering
|
||||
|
||||
```
|
||||
controllers/ HTTP surface only — parse, delegate, serialise. Keep thin.
|
||||
services/ Business logic. One SQLAlchemyAsyncRepositoryService subclass per aggregate.
|
||||
database/models/ SQLAlchemy models.
|
||||
schemas/ Pydantic DTOs for request bodies and responses.
|
||||
```
|
||||
|
||||
Controllers call **service** methods, not the repository. `app.py` assembles the app: route
|
||||
handlers, JWT auth, exception handlers, the SQLAlchemy plugin, and two lifespan context managers.
|
||||
|
||||
## advanced-alchemy idioms
|
||||
|
||||
These are the conventions that are easy to get wrong if you write plain SQLAlchemy here:
|
||||
|
||||
- Models extend `BigIntAuditBase` (adds id/created_at/updated_at); pure link and child rows extend
|
||||
`BigIntBase` (e.g. `Identifier`, `FileMetadata`, `BookAuthorLink`).
|
||||
- A service declares an inner repository and points at it:
|
||||
|
||||
```python
|
||||
class BookService(SQLAlchemyAsyncRepositoryService[Book]):
|
||||
class Repo(SQLAlchemyAsyncRepository[Book]):
|
||||
model_type = Book
|
||||
repository_type = Repo
|
||||
```
|
||||
|
||||
- Transform incoming data with the **`to_model_on_create` / `to_model_on_update` hooks**, not by
|
||||
overriding `create`/`update` wholesale — see `services/book.py:407` onward.
|
||||
- Serialise responses through `service.to_schema(obj, schema_type=s.SomeRead)`; for lists,
|
||||
`to_schema(items, total, filters, schema_type=…)` produces the `OffsetPagination` envelope.
|
||||
- `Author`, `Tag`, `Publisher` and `BookSeries` are deduplicated with **`as_unique_async`**. Never
|
||||
construct them directly when attaching to a book — use
|
||||
`await Author.as_unique_async(session, name=name)` as
|
||||
`BookService._populate_with_unique_relationships` does, or you will create duplicate rows.
|
||||
- Domain methods on services are named for the domain (`create_book`, `update_book`, `add_files`),
|
||||
deliberately distinct from the inherited CRUD names.
|
||||
|
||||
## Dependency injection
|
||||
|
||||
`services/dependencies.py` is the hub; controllers wire providers in their `dependencies` dict.
|
||||
|
||||
- Most providers come from `create_service_provider(SomeService, …)` — one line each.
|
||||
- `provide_book_service` is hand-written because it must inject eager loads *and* scope
|
||||
user-specific rows: `selectinload` for authors/tags/files/etc., plus `with_loader_criteria` so
|
||||
`BookProgress` and `BookListLink` only load rows belonging to `current_user`. If you add a
|
||||
relationship that the API returns, add it to that `load` list.
|
||||
- `create_book_filter_dependencies` intentionally **overrides** advanced-alchemy's stock providers:
|
||||
the search filter becomes a trigram search, and order-by gains a `random` sort order. Do not
|
||||
replace it with the stock `create_filter_dependencies`.
|
||||
- `get_library_by_id` resolves the target library from either a `library_id` query param or the
|
||||
book's own `library_id`, and raises 404 for either miss.
|
||||
|
||||
## Filters
|
||||
|
||||
`services/filters/` holds `StatementFilter` dataclasses that compose into any `list` /
|
||||
`list_and_count` call: `TagFilter`, `AuthorFilter`, `BookshelfFilter`, `ProgressFilter`,
|
||||
`TrigramSearchFilter`, `CustomOrderBy`, `FileHashFilter`, plus the `*LibraryFilter` variants used by
|
||||
OPDS.
|
||||
|
||||
To add list-filtering behaviour: write the dataclass here, add a `provide_*_filter` function in
|
||||
`dependencies.py`, register it in the controller's `dependencies`, and fold it into
|
||||
`provide_book_filters` — the controller handler itself does not change.
|
||||
|
||||
## Authentication — three schemes
|
||||
|
||||
| Scheme | Where | Used by |
|
||||
| --- | --- | --- |
|
||||
| JWT bearer (`OAuth2PasswordBearerAuth`) | `app.py` | The web frontend and the main API. Public paths are listed in its `exclude=[…]`. |
|
||||
| HTTP Basic (`middleware/basic_auth.py`) | `OpdsController` | E-reader / OPDS clients, which only speak Basic. |
|
||||
| `X-AUTH-USER` API key (`middleware/kosync_auth.py`) | `KosyncController` | KOReader devices; the key maps to a `KosyncDevice` row, which maps to a user. |
|
||||
|
||||
Each middleware resolves a `User` onto the connection; the matching
|
||||
`provide_user_via_basic_auth` / `provide_user_via_kosync_auth` dependencies expose it to handlers.
|
||||
|
||||
## Filesystem behaviour
|
||||
|
||||
The backend owns files on disk, not just rows:
|
||||
|
||||
- **Layout** — `services/filesystem_library.py` (`BookPathGenerator`) renders a Jinja2 template
|
||||
against book metadata to decide where a book lives under the library's `root_path`
|
||||
(default: `author/series/position - title/`).
|
||||
- **Metadata extraction** — `services/metadata_extractor.py` reads EPUB (ebooklib) and PDF
|
||||
(pypdfium2) files; extracted values fill only *empty* fields on the incoming payload.
|
||||
- **Covers** — converted to WebP with a UUID filename under `settings.book_cover_path`, served by a
|
||||
static-files router mounted at `/covers`.
|
||||
- **Consume directory** — `services/consume.py` (`ConsumeDirectoryWatcher`) watches
|
||||
`settings.consume_path` with `watchfiles`, creates one subdirectory per library slug, batches
|
||||
additions (3 s debounce) and imports them via `BookService.create_many_from_existing_files`.
|
||||
Started as an asyncio task from the `setup_directory_watcher` lifespan hook.
|
||||
- **Updates move files.** `BookService.update_book` regenerates the path from the new metadata and,
|
||||
if it differs, moves the directory contents and prunes empty parents. Keep that in mind before
|
||||
changing metadata handling.
|
||||
|
||||
## KOReader hashing
|
||||
|
||||
`services/utils.py` reimplements KOReader's partial-MD5 document identifier: 1 KiB samples at
|
||||
offsets produced by LuaJIT's 32-bit `bit.lshift`, including its shift-masking wrap-around
|
||||
(`shift & 0x1F`, so `i=-1` yields offset 0). `_lshift32` looks wrong and is not — the overflow is
|
||||
what makes hashes match real devices. Don't "simplify" it; `tests/integration/test_file_hash.py`
|
||||
guards the behaviour.
|
||||
|
||||
## Database
|
||||
|
||||
- **`pg_trgm` is required.** `Book.__table_args__` declares a GIN trigram index on `title`, which
|
||||
`TrigramSearchFilter` uses for fuzzy title search. The extension is enabled by a migration and, in
|
||||
tests, by `conftest.py`.
|
||||
- Migrations live in `migrations/versions/`. Generate with
|
||||
`alchemy --config chitai.database.config.config make-migrations` (add `--no-autogenerate` for a
|
||||
blank revision), apply with `… upgrade`. `database/config.py` sets `create_all=False`, so nothing
|
||||
is auto-created at runtime; production applies migrations from `entrypoint.sh`.
|
||||
- Sessions use `expire_on_commit=False` and Litestar's `before_send_handler="autocommit"`, so a
|
||||
handler that returns 2xx commits automatically.
|
||||
|
||||
## Testing
|
||||
|
||||
`pytest tests/` — `asyncio_mode = "auto"`, so async tests need no marker.
|
||||
|
||||
- `tests/unit/` exercises services directly against a real session; `tests/integration/` drives the
|
||||
whole app through `AsyncTestClient(app=create_app())`.
|
||||
- The database is a throwaway container from `pytest-databases`, so **Docker must be running**.
|
||||
- `tests/conftest.py` provides the shared fixtures: `client`, `authenticated_client`,
|
||||
`other_authenticated_client` (a second user, for access-control tests), one fixture per service,
|
||||
`test_user` / `test_library`, and an autouse fixture that redirects cover storage into `tmp_path`.
|
||||
- `tests/integration/conftest.py` monkeypatches the module-level alchemy `config` onto the test
|
||||
engine/sessionmaker and drops+recreates+reseeds the schema for every test.
|
||||
- Real EPUB and PDF fixtures live in `tests/data_files/`.
|
||||
|
||||
## Known rough edges
|
||||
|
||||
Observed in the current tree — don't mistake these for intentional patterns to copy:
|
||||
|
||||
- `controllers/book.py` — file-level TODO: `book_id` is a path parameter on some endpoints and a
|
||||
query parameter on others. `set_book_progress_batch` does a documented N+1 (one select + one
|
||||
upsert per book).
|
||||
- `services/filesystem_library.py` — TODO to replace Jinja2 templating with simple placeholders;
|
||||
`generate_filename` accepts a `filename_template` but currently ignores it and returns the
|
||||
original filename.
|
||||
- `app.py` — `watcher_task` is declared as a module-level global but assigned locally inside
|
||||
`setup_directory_watcher`, so the global is never populated (cancellation still works via the
|
||||
closure).
|
||||
- `BookService.get_files` (the multi-book ZIP download) opens `Path(file.path)`, but `file.path` is
|
||||
stored relative to `book.path` — worth verifying before relying on that endpoint.
|
||||
Reference in New Issue
Block a user