feat: check for duplicate files when importing books

Incoming files are matched against what is already stored, keyed on the
KOReader hash and the file size. Bulk uploads skip and report them, deliberate
creates are refused with a 409, the consume directory parks them aside, and
allow_duplicates overrides all three.

Also: books whose metadata generates a path another book already owns are moved
aside, so a forced copy cannot overwrite the original's files.

backend/AGENTS.md

@@ -88,6 +88,14 @@ The backend owns files on disk, not just rows:
 - **Layout** — `services/filesystem_library.py` (`BookPathGenerator`) renders a Jinja2 template
   against book metadata to decide where a book lives under the library's `root_path`
   (default: `author/series/position - title/`).
+- **One directory per book, never shared.** The generated path is a pure function of the metadata,
+  so two books with the same author and title produce the same one — two editions, or an
+  `allow_duplicates` copy. `BookService._reserve_book_path` moves the later one to `title (2)`
+  before anything is written, and `update_book` reserves the same way so a rename cannot move a
+  book in on top of another. This matters because `book.path` is what deletes, moves and file
+  lookups act on: books sharing a directory means one overwrites the other's files, and deleting
+  either takes both. `_unused_path` does the same job for filenames within a directory.
+  A book that already has a `path` keeps it — `add_files` must follow the book, not the template.
 - **Metadata extraction** — `services/metadata_extractor.py` reads EPUB (ebooklib) and PDF
   (pypdfium2) files; extracted values fill only *empty* fields on the incoming payload.
 - **Covers** — converted to WebP with a UUID filename under `settings.book_cover_path`, served by a
@@ -100,6 +108,41 @@ The backend owns files on disk, not just rows:
   if it differs, moves the directory contents and prunes empty parents. Keep that in mind before
   changing metadata handling.
 
+## Duplicate detection
+
+Every ingest path screens incoming files against what is already stored, keyed on
+**`(hash, size)`** — never the hash alone, because it samples 12 KiB (see below) and
+EPUBs from one toolchain often share their first window. `FileMetadata.hash` carries a
+plain, deliberately **non-unique** index: a collision must not be able to fail an import,
+and older databases may already hold duplicates.
+
+Scope comes from `CHITAI_DUPLICATE_SCOPE` (`library`, the default | `global` | `off`).
+
+The policy differs by how deliberate the import is:
+
+| Path | Behaviour |
+| --- | --- |
+| `create_many_from_files` (browser bulk) | Skip per file, skip a whole group whose files are all known, report everything skipped in `ImportResult.duplicates`. Re-dropping a folder to pick up what is new is the case this serves. |
+| `create_book` (single, with metadata) | All-or-nothing: raises `DuplicateFilesError`, which `controllers/book.py` renders as a **409** carrying the refused files in `extra`. |
+| `add_files` | A file the book already carries is a no-op; one stored under another book raises `DuplicateFilesError`. |
+| `create_many_from_existing_files` (consume watcher) | Skips, and **moves the file to `CHITAI_DUPLICATE_PATH/<library slug>/`** — nothing is deleted, and it cannot stay put because `watchfiles` only reports additions. That path must stay outside `consume_path` or the watcher re-imports it and tries to read the directory name as a library slug. |
+
+`allow_duplicates=true` overrides all of it, on every endpoint. Keep that working — the
+hash is not proof of identity, so a false positive has to be recoverable.
+
+Two things to preserve when touching this code:
+
+- **Screening runs before anything is written.** `fingerprint_upload` reads the spooled
+  upload and rewinds it; the resulting fingerprints are handed to `_save_book_files`,
+  which skips its own `StreamingHasher` when it already has the answer. Passing them
+  through is what keeps the file from being read twice.
+- **`_screen_for_duplicates` extends the `known` dict as it goes**, so the same bytes
+  submitted twice in one request are caught. Those duplicates report `book_id: None` —
+  there is no row to point at yet.
+
+`POST /books/duplicates` answers the same question from fingerprints alone, for clients
+that want to ask before uploading anything.
+
 ## KOReader hashing
 
 `services/utils.py` reimplements KOReader's partial-MD5 document identifier: 1 KiB samples at