Chitai backend

Litestar REST API for the eBook library. See the repo-root AGENTS.md for the overall picture and dev-environment setup.

Stack: Python 3.13 · Litestar 2 · advanced-alchemy over async SQLAlchemy 2 · asyncpg · PostgreSQL 17 · Alembic (through advanced-alchemy's alchemy CLI) · pydantic-settings · uv.

Layering

controllers/      HTTP surface only — parse, delegate, serialise. Keep thin.
services/         Business logic. One SQLAlchemyAsyncRepositoryService subclass per aggregate.
database/models/  SQLAlchemy models.
schemas/          Pydantic DTOs for request bodies and responses.

Controllers call service methods, not the repository. app.py assembles the app: route handlers, JWT auth, exception handlers, the SQLAlchemy plugin, and two lifespan context managers.

advanced-alchemy idioms

These are the conventions that are easy to get wrong if you write plain SQLAlchemy here:

Models extend BigIntAuditBase (adds id/created_at/updated_at); pure link and child rows extend BigIntBase (e.g. Identifier, FileMetadata, BookAuthorLink).

A service declares an inner repository and points at it:

class BookService(SQLAlchemyAsyncRepositoryService[Book]):
    class Repo(SQLAlchemyAsyncRepository[Book]):
        model_type = Book
    repository_type = Repo

Transform incoming data with the to_model_on_create / to_model_on_update hooks, not by overriding create/update wholesale — see services/book.py:407 onward.
Serialise responses through service.to_schema(obj, schema_type=s.SomeRead); for lists, to_schema(items, total, filters, schema_type=…) produces the OffsetPagination envelope.
Author, Tag, Publisher and BookSeries are deduplicated with as_unique_async. Never construct them directly when attaching to a book — use await Author.as_unique_async(session, name=name) as BookService._populate_with_unique_relationships does, or you will create duplicate rows.
Domain methods on services are named for the domain (create_book, update_book, add_files), deliberately distinct from the inherited CRUD names.

Dependency injection

services/dependencies.py is the hub; controllers wire providers in their dependencies dict.

Most providers come from create_service_provider(SomeService, …) — one line each.
provide_book_service is hand-written because it must inject eager loads and scope user-specific rows: selectinload for authors/tags/files/etc., plus with_loader_criteria so BookProgress and BookListLink only load rows belonging to current_user. If you add a relationship that the API returns, add it to that load list.
create_book_filter_dependencies intentionally overrides advanced-alchemy's stock providers: the search filter becomes a trigram search, and order-by gains a random sort order. Do not replace it with the stock create_filter_dependencies.
get_library_by_id resolves the target library from either a library_id query param or the book's own library_id, and raises 404 for either miss.

Filters

services/filters/ holds StatementFilter dataclasses that compose into any list / list_and_count call: TagFilter, AuthorFilter, BookshelfFilter, ProgressFilter, TrigramSearchFilter, CustomOrderBy, FileHashFilter, plus the *LibraryFilter variants used by OPDS.

To add list-filtering behaviour: write the dataclass here, add a provide_*_filter function in dependencies.py, register it in the controller's dependencies, and fold it into provide_book_filters — the controller handler itself does not change.

Authentication — three schemes

Scheme	Where	Used by
JWT bearer (`OAuth2PasswordBearerAuth`)	`app.py`	The web frontend and the main API. Public paths are listed in its `exclude=[…]`.
HTTP Basic (`middleware/basic_auth.py`)	`OpdsController`	E-reader / OPDS clients, which only speak Basic.
`X-AUTH-USER` API key (`middleware/kosync_auth.py`)	`KosyncController`	KOReader devices; the key maps to a `KosyncDevice` row, which maps to a user.

Each middleware resolves a User onto the connection; the matching provide_user_via_basic_auth / provide_user_via_kosync_auth dependencies expose it to handlers.

Filesystem behaviour

The backend owns files on disk, not just rows:

Layout — services/filesystem_library.py (BookPathGenerator) renders a Jinja2 template against book metadata to decide where a book lives under the library's root_path (default: author/series/position - title/).
Metadata extraction — services/metadata_extractor.py reads EPUB (ebooklib) and PDF (pypdfium2) files; extracted values fill only empty fields on the incoming payload.
Covers — converted to WebP with a UUID filename under settings.book_cover_path, served by a static-files router mounted at /covers.
Consume directory — services/consume.py (ConsumeDirectoryWatcher) watches settings.consume_path with watchfiles, creates one subdirectory per library slug, batches additions (3 s debounce) and imports them via BookService.create_many_from_existing_files. Started as an asyncio task from the setup_directory_watcher lifespan hook.
Updates move files. BookService.update_book regenerates the path from the new metadata and, if it differs, moves the directory contents and prunes empty parents. Keep that in mind before changing metadata handling.

KOReader hashing

services/utils.py reimplements KOReader's partial-MD5 document identifier: 1 KiB samples at offsets produced by LuaJIT's 32-bit bit.lshift, including its shift-masking wrap-around (shift & 0x1F, so i=-1 yields offset 0). _lshift32 looks wrong and is not — the overflow is what makes hashes match real devices. Don't "simplify" it; tests/integration/test_file_hash.py guards the behaviour.

Database

pg_trgm is required. Book.__table_args__ declares a GIN trigram index on title, which TrigramSearchFilter uses for fuzzy title search. The extension is enabled by a migration and, in tests, by conftest.py.
Migrations live in migrations/versions/. Generate with alchemy --config chitai.database.config.config make-migrations (add --no-autogenerate for a blank revision), apply with … upgrade. database/config.py sets create_all=False, so nothing is auto-created at runtime; production applies migrations from entrypoint.sh.
Sessions use expire_on_commit=False and Litestar's before_send_handler="autocommit", so a handler that returns 2xx commits automatically.

Testing

pytest tests/ — asyncio_mode = "auto", so async tests need no marker.

tests/unit/ exercises services directly against a real session; tests/integration/ drives the whole app through AsyncTestClient(app=create_app()).
The database is a throwaway container from pytest-databases, so Docker must be running.
tests/conftest.py provides the shared fixtures: client, authenticated_client, other_authenticated_client (a second user, for access-control tests), one fixture per service, test_user / test_library, and an autouse fixture that redirects cover storage into tmp_path.
tests/integration/conftest.py monkeypatches the module-level alchemy config onto the test engine/sessionmaker and drops+recreates+reseeds the schema for every test.
Real EPUB and PDF fixtures live in tests/data_files/.

Known rough edges

Observed in the current tree — don't mistake these for intentional patterns to copy:

controllers/book.py — file-level TODO: book_id is a path parameter on some endpoints and a query parameter on others. set_book_progress_batch does a documented N+1 (one select + one upsert per book).
services/filesystem_library.py — TODO to replace Jinja2 templating with simple placeholders; generate_filename accepts a filename_template but currently ignores it and returns the original filename.
app.py — watcher_task is declared as a module-level global but assigned locally inside setup_directory_watcher, so the global is never populated (cancellation still works via the closure).
BookService.get_files (the multi-book ZIP download) opens Path(file.path), but file.path is stored relative to book.path — worth verifying before relying on that endpoint.

8.3 KiB Raw Blame History