diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..43764cc --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,103 @@ +# Chitai + +Self-hosted eBook library manager. Users organise eBook files into **libraries**, which contain +**books** (one book = one metadata record + one or more files on disk). Books carry authors, +publishers, tags, series and identifiers, can be grouped into per-user **bookshelves**, and are read +in-browser through built-in EPUB and PDF readers with reading-progress tracking. The catalogue is +also exposed as an **OPDS** feed for e-reader apps, and progress syncs with **KOReader** devices via +a KOSync-compatible endpoint. + +## Layout + +| Path | What | +| --- | --- | +| `backend/` | Litestar REST API + PostgreSQL. See `backend/AGENTS.md`. | +| `frontend/` | SvelteKit SSR web app. See `frontend/AGENTS.md`. | +| `docker-compose.yml` | Production stack: `db` (postgres:17), `backend`, `frontend`. | +| `docs/screenshots/` | Images used by `README.md`. | +| `shell.nix` | Root dev shell; composes the two sub-shells. | + +## Development environment + +`nix-shell` from the repo root is the intended entry point. It pulls in both +`backend/shell.nix` and `frontend/shell.nix`, whose `shellHook`s do real work as a side effect: + +- **backend** — `uv venv` + `uv sync`, then `initdb` / `pg_ctl start` into `backend/.postgres/` + (socket dir, not TCP), `createdb chitai`, and finally applies migrations with + `alchemy --config chitai.database.config.config upgrade --no-prompt`. + `exitHook` stops PostgreSQL on shell exit. +- **frontend** — `pnpm install`. + +So entering the shell gives you a running database with an up-to-date schema; you do not need to +start Postgres yourself. Both hooks `cd` around, which can be surprising in scripts. + +Without Nix you need: Python 3.13 + `uv`, Node 24 + `pnpm`, and PostgreSQL 17 with the `pg_trgm` +extension available. + +## Configuration + +All backend settings are read by `backend/src/chitai/config.py` (`Settings`, pydantic-settings) with +the **`CHITAI_` prefix** from the repo-root `.env`. `.env.prod-example` is the template; copy it to +`.env` for a fresh deployment. + +`.env` is gitignored and contains a real `CHITAI_TOKEN_SECRET` — do not print, copy or commit it. + +The frontend reads exactly one variable, `VITE_BACKEND_API_URL` +(`frontend/src/lib/server/config.ts`, defaulting to `http://localhost:8000`). + +## How a request flows + +``` +browser + └─ SvelteKit SSR node server + ├─ hooks.server.ts reads `authToken` cookie, validates via GET /access/me, + │ populates locals.user / locals.api, redirects to /login otherwise + ├─ $lib/api/*.remote.ts remote functions (query/command/form) → locals.api (ApiClient) + └─ routes/api/[...path] catch-all proxy, for browser-direct fetches (reader file streams) + └─ Litestar backend + └─ controllers/ → services/ → SQLAlchemy models → PostgreSQL +``` + +The JWT the frontend holds in the `authToken` cookie is the same bearer token the backend issues +from `POST /access/login`. The frontend never stores credentials beyond that cookie. + +## Commands + +Backend (from `backend/`): + +```bash +uv run litestar --app-dir src/chitai/ run --reload # dev server on :8000 +pytest tests/ # needs Docker (pytest-databases) +ruff format src/ +alchemy --config chitai.database.config.config make-migrations +alchemy --config chitai.database.config.config upgrade +``` + +Frontend (from `frontend/`): + +```bash +pnpm dev # vite dev server on :5173 +pnpm build # adapter-node output in build/ +pnpm check # svelte-check — run before finishing +pnpm lint # prettier --check + eslint +pnpm format # prettier --write +``` + +API docs are served by the running backend at `http://localhost:8000/schema/` (Swagger) and +`/schema/openapi.json`. + +## Cross-cutting rules + +- **Keep the two schema layers in sync.** Backend request/response shapes live in + `backend/src/chitai/schemas/` (Pydantic); the frontend mirrors them as Zod schemas in + `frontend/src/lib/schema/`. Changing one without the other produces runtime validation failures, + not type errors. +- **Regenerate the OpenAPI types** after changing API shapes. + `frontend/src/lib/schema/openapi/schema.d.ts` is generated from the backend's OpenAPI document + with `openapi-typescript` (a devDependency; there is no `package.json` script for it, so it is run + manually against a live backend). +- **Migrations are mandatory.** The app runs with `create_all=False`, so a model change without a + matching Alembic revision will not reach the database. +- **Commit messages** follow `type: summary` — `feat:`, `fix:`, `refactor:`, `chore:`. +- The three `README.md` files are user-facing. Agent-facing knowledge belongs in the `AGENTS.md` + files. diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..43c994c --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1 @@ +@AGENTS.md diff --git a/backend/AGENTS.md b/backend/AGENTS.md new file mode 100644 index 0000000..28d9246 --- /dev/null +++ b/backend/AGENTS.md @@ -0,0 +1,151 @@ +# Chitai backend + +Litestar REST API for the eBook library. See the repo-root `AGENTS.md` for the overall picture and +dev-environment setup. + +**Stack:** Python 3.13 · Litestar 2 · advanced-alchemy over async SQLAlchemy 2 · asyncpg · +PostgreSQL 17 · Alembic (through advanced-alchemy's `alchemy` CLI) · pydantic-settings · uv. + +## Layering + +``` +controllers/ HTTP surface only — parse, delegate, serialise. Keep thin. +services/ Business logic. One SQLAlchemyAsyncRepositoryService subclass per aggregate. +database/models/ SQLAlchemy models. +schemas/ Pydantic DTOs for request bodies and responses. +``` + +Controllers call **service** methods, not the repository. `app.py` assembles the app: route +handlers, JWT auth, exception handlers, the SQLAlchemy plugin, and two lifespan context managers. + +## advanced-alchemy idioms + +These are the conventions that are easy to get wrong if you write plain SQLAlchemy here: + +- Models extend `BigIntAuditBase` (adds id/created_at/updated_at); pure link and child rows extend + `BigIntBase` (e.g. `Identifier`, `FileMetadata`, `BookAuthorLink`). +- A service declares an inner repository and points at it: + + ```python + class BookService(SQLAlchemyAsyncRepositoryService[Book]): + class Repo(SQLAlchemyAsyncRepository[Book]): + model_type = Book + repository_type = Repo + ``` + +- Transform incoming data with the **`to_model_on_create` / `to_model_on_update` hooks**, not by + overriding `create`/`update` wholesale — see `services/book.py:407` onward. +- Serialise responses through `service.to_schema(obj, schema_type=s.SomeRead)`; for lists, + `to_schema(items, total, filters, schema_type=…)` produces the `OffsetPagination` envelope. +- `Author`, `Tag`, `Publisher` and `BookSeries` are deduplicated with **`as_unique_async`**. Never + construct them directly when attaching to a book — use + `await Author.as_unique_async(session, name=name)` as + `BookService._populate_with_unique_relationships` does, or you will create duplicate rows. +- Domain methods on services are named for the domain (`create_book`, `update_book`, `add_files`), + deliberately distinct from the inherited CRUD names. + +## Dependency injection + +`services/dependencies.py` is the hub; controllers wire providers in their `dependencies` dict. + +- Most providers come from `create_service_provider(SomeService, …)` — one line each. +- `provide_book_service` is hand-written because it must inject eager loads *and* scope + user-specific rows: `selectinload` for authors/tags/files/etc., plus `with_loader_criteria` so + `BookProgress` and `BookListLink` only load rows belonging to `current_user`. If you add a + relationship that the API returns, add it to that `load` list. +- `create_book_filter_dependencies` intentionally **overrides** advanced-alchemy's stock providers: + the search filter becomes a trigram search, and order-by gains a `random` sort order. Do not + replace it with the stock `create_filter_dependencies`. +- `get_library_by_id` resolves the target library from either a `library_id` query param or the + book's own `library_id`, and raises 404 for either miss. + +## Filters + +`services/filters/` holds `StatementFilter` dataclasses that compose into any `list` / +`list_and_count` call: `TagFilter`, `AuthorFilter`, `BookshelfFilter`, `ProgressFilter`, +`TrigramSearchFilter`, `CustomOrderBy`, `FileHashFilter`, plus the `*LibraryFilter` variants used by +OPDS. + +To add list-filtering behaviour: write the dataclass here, add a `provide_*_filter` function in +`dependencies.py`, register it in the controller's `dependencies`, and fold it into +`provide_book_filters` — the controller handler itself does not change. + +## Authentication — three schemes + +| Scheme | Where | Used by | +| --- | --- | --- | +| JWT bearer (`OAuth2PasswordBearerAuth`) | `app.py` | The web frontend and the main API. Public paths are listed in its `exclude=[…]`. | +| HTTP Basic (`middleware/basic_auth.py`) | `OpdsController` | E-reader / OPDS clients, which only speak Basic. | +| `X-AUTH-USER` API key (`middleware/kosync_auth.py`) | `KosyncController` | KOReader devices; the key maps to a `KosyncDevice` row, which maps to a user. | + +Each middleware resolves a `User` onto the connection; the matching +`provide_user_via_basic_auth` / `provide_user_via_kosync_auth` dependencies expose it to handlers. + +## Filesystem behaviour + +The backend owns files on disk, not just rows: + +- **Layout** — `services/filesystem_library.py` (`BookPathGenerator`) renders a Jinja2 template + against book metadata to decide where a book lives under the library's `root_path` + (default: `author/series/position - title/`). +- **Metadata extraction** — `services/metadata_extractor.py` reads EPUB (ebooklib) and PDF + (pypdfium2) files; extracted values fill only *empty* fields on the incoming payload. +- **Covers** — converted to WebP with a UUID filename under `settings.book_cover_path`, served by a + static-files router mounted at `/covers`. +- **Consume directory** — `services/consume.py` (`ConsumeDirectoryWatcher`) watches + `settings.consume_path` with `watchfiles`, creates one subdirectory per library slug, batches + additions (3 s debounce) and imports them via `BookService.create_many_from_existing_files`. + Started as an asyncio task from the `setup_directory_watcher` lifespan hook. +- **Updates move files.** `BookService.update_book` regenerates the path from the new metadata and, + if it differs, moves the directory contents and prunes empty parents. Keep that in mind before + changing metadata handling. + +## KOReader hashing + +`services/utils.py` reimplements KOReader's partial-MD5 document identifier: 1 KiB samples at +offsets produced by LuaJIT's 32-bit `bit.lshift`, including its shift-masking wrap-around +(`shift & 0x1F`, so `i=-1` yields offset 0). `_lshift32` looks wrong and is not — the overflow is +what makes hashes match real devices. Don't "simplify" it; `tests/integration/test_file_hash.py` +guards the behaviour. + +## Database + +- **`pg_trgm` is required.** `Book.__table_args__` declares a GIN trigram index on `title`, which + `TrigramSearchFilter` uses for fuzzy title search. The extension is enabled by a migration and, in + tests, by `conftest.py`. +- Migrations live in `migrations/versions/`. Generate with + `alchemy --config chitai.database.config.config make-migrations` (add `--no-autogenerate` for a + blank revision), apply with `… upgrade`. `database/config.py` sets `create_all=False`, so nothing + is auto-created at runtime; production applies migrations from `entrypoint.sh`. +- Sessions use `expire_on_commit=False` and Litestar's `before_send_handler="autocommit"`, so a + handler that returns 2xx commits automatically. + +## Testing + +`pytest tests/` — `asyncio_mode = "auto"`, so async tests need no marker. + +- `tests/unit/` exercises services directly against a real session; `tests/integration/` drives the + whole app through `AsyncTestClient(app=create_app())`. +- The database is a throwaway container from `pytest-databases`, so **Docker must be running**. +- `tests/conftest.py` provides the shared fixtures: `client`, `authenticated_client`, + `other_authenticated_client` (a second user, for access-control tests), one fixture per service, + `test_user` / `test_library`, and an autouse fixture that redirects cover storage into `tmp_path`. +- `tests/integration/conftest.py` monkeypatches the module-level alchemy `config` onto the test + engine/sessionmaker and drops+recreates+reseeds the schema for every test. +- Real EPUB and PDF fixtures live in `tests/data_files/`. + +## Known rough edges + +Observed in the current tree — don't mistake these for intentional patterns to copy: + +- `controllers/book.py` — file-level TODO: `book_id` is a path parameter on some endpoints and a + query parameter on others. `set_book_progress_batch` does a documented N+1 (one select + one + upsert per book). +- `services/filesystem_library.py` — TODO to replace Jinja2 templating with simple placeholders; + `generate_filename` accepts a `filename_template` but currently ignores it and returns the + original filename. +- `app.py` — `watcher_task` is declared as a module-level global but assigned locally inside + `setup_directory_watcher`, so the global is never populated (cancellation still works via the + closure). +- `BookService.get_files` (the multi-book ZIP download) opens `Path(file.path)`, but `file.path` is + stored relative to `book.path` — worth verifying before relying on that endpoint. diff --git a/backend/CLAUDE.md b/backend/CLAUDE.md new file mode 100644 index 0000000..43c994c --- /dev/null +++ b/backend/CLAUDE.md @@ -0,0 +1 @@ +@AGENTS.md diff --git a/frontend/AGENTS.md b/frontend/AGENTS.md new file mode 100644 index 0000000..9a907e9 --- /dev/null +++ b/frontend/AGENTS.md @@ -0,0 +1,115 @@ +# Chitai frontend + +SvelteKit web app for the eBook library. See the repo-root `AGENTS.md` for the overall picture and +dev-environment setup. + +**Stack:** SvelteKit 2 with `adapter-node` · Svelte 5 (runes) · Tailwind v4 · Zod v4 · +`epubjs` · `mode-watcher` (dark mode) · `svelte-sonner` (toasts) · pnpm. + +Two experimental flags are on in `svelte.config.js` and the codebase depends on both: +`kit.experimental.remoteFunctions` and `compilerOptions.experimental.async` (`await` in components). + +Tailwind v4 has **no config file** — the theme, oklch colour tokens and `@custom-variant dark` all +live in `src/app.css`. + +## Talking to the backend + +There are two mechanisms; pick deliberately. + +**1. Remote functions — the default.** `src/lib/api/*.remote.ts` export `query` / `command` / `form` +functions from `$app/server`. Each takes a Zod schema from `$lib/schema` as its validator and runs +on the server, reaching the API through `locals.api` (the `ApiClient` in `src/lib/server/api.ts`): + +```ts +export const getBook = query(stringCoerce, async (id): Promise => { + const { locals } = getRequestEvent(); + const response = await locals.api.get(`/books/${id}`); + if (!response.ok) error(response.status === 404 ? 404 : 500, '…'); + return await response.json(); +}); +``` + +Conventions: build query strings with `createQueryParams` from `$lib/utils`; on a failed response +throw SvelteKit's `error(status, message)`; multipart uploads go through `postMultipart` / +`putMultipart`. Re-export new modules from `src/lib/api/index.ts`. + +**2. The catch-all proxy** at `src/routes/api/[...path]/+server.ts` forwards GET/POST/PATCH/DELETE +to the backend with the auth header attached. Use it only where the **browser itself** must fetch +the backend — e.g. streaming a book file into the EPUB/PDF reader. It is not the general-purpose +path. + +## Auth + +`src/hooks.server.ts` is a `sequence` of two handles: the first reads the `authToken` cookie, +constructs an `ApiClient`, validates it with `GET /access/me` and fills `locals.user` / +`locals.authToken` / `locals.api` (clearing the cookie if invalid); the second redirects any route +outside `/login` to the login page when there is no user. + +The cookie is set in `src/lib/api/auth.remote.ts` (`login`) — httpOnly, secure, sameSite strict, one +week — and deleted by `logout`. The backend JWT never reaches client-side JS. + +## Schemas + +- `src/lib/schema/*.ts` — hand-written Zod schemas, used as remote-function input validators and as + the source of the exported TS types. Mirror `backend/src/chitai/schemas/` when the API changes. +- `src/lib/schema/common.ts` — shared building blocks: `stringCoerce` / `arrayCoerce` coercion + helpers, `PaginatedResponse`, and the pagination / search / order query schemas that most list + endpoints compose from. +- `src/lib/schema/openapi/schema.d.ts` — **generated** from the backend's OpenAPI document with + `openapi-typescript`. Never hand-edit; regenerate after backend API changes. + +## State + +Client state lives in classes in `src/lib/state/*.svelte.ts` using `$state` / `$derived`, shared via +Svelte context with a module-level `Symbol` key and a `setXState` / `getXState` pair: + +```ts +const LIBRARY_KEY = Symbol('LIBRARY'); +export function setLibraryState(libraries: Library[]) { return setContext(LIBRARY_KEY, new LibraryState(libraries)); } +export function getLibraryState() { return getContext>(LIBRARY_KEY); } +``` + +Follow that pattern rather than introducing stores. `library.svelte.ts` is the reference — including +its optimistic-delete-with-rollback and toast handling. `bookCollection` / `bookSelection` / +`bookOperations` split list data, selection and mutations across three cooperating classes. + +## Components + +- `src/lib/components/ui/` — vendored shadcn-svelte (`components.json`) plus jsrepo blocks from + `@ieedan/shadcn-svelte-extras` (`jsrepo.json`). Treat as generated: add components with the CLIs + rather than hand-writing them, and prefer wrapping over editing. +- App components live in `forms/`, `layout/`, `view/` (browser, grid/list/table, filters, sort) and + `reader/` (epub reader + chapter sidebar). +- `cn()` from `$lib/utils` merges Tailwind classes; the `WithElementRef` / `WithoutChild` helpers + there are the shadcn prop-typing conventions. + +## Routing + +Route groups carry the layout structure: + +- `(root)` — the authenticated shell (sidebar, header); `+layout.server.ts` loads the libraries. +- `(root)/(library)` — library-scoped pages: `library/[libraryId]/view`, `book/[bookId]`, edit, and + the readers at `book/[bookId]/read/{epub,pdf}/[fileId]`. +- `+layout@.svelte` breakouts reset to the root layout for the login page and the full-screen reader. + +## Conventions + +Prettier (`.prettierrc`): tabs, single quotes, no trailing commas, 100 columns, with the Svelte and +Tailwind plugins. Run `pnpm check` (svelte-check) and `pnpm lint` before considering work done. + +## Known rough edges + +Observed in the current tree — don't mistake these for intentional patterns to copy: + +- `src/lib/schema/openapi/schema.d.ts` is **stale** — it predates the `BookProgress` rework and has + no `BookProgressRead`, so `book.progress` types as `{}`. This is the source of most of the ~104 + errors `pnpm check` reports on a clean tree; regenerating it should clear them. Get a baseline + before assuming an error is yours. +- `src/routes/api/[...path]/+server.ts` — all four handlers are annotated `RequestHandler` while the + import of that type is commented out at line 4. +- `src/app.d.ts` — `App.Locals["user"]` is typed from `lucide-svelte`'s `User` *icon* component + rather than the `User` interface in `$lib/server/auth`. +- Uncommitted work in progress (as of 2026-08-10): library icons, spanning + `components/ui/icon-picker/`, the newly vendored `components/ui/popover/`, + `forms/library-create-form.svelte`, `layout/library-switcher.svelte` and `schema/library.ts`. + Prefer not to refactor those files mid-flight. diff --git a/frontend/CLAUDE.md b/frontend/CLAUDE.md new file mode 100644 index 0000000..43c994c --- /dev/null +++ b/frontend/CLAUDE.md @@ -0,0 +1 @@ +@AGENTS.md