Compare commits
25
Commits
968166c1fd
...
7e33a8fe05
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
7e33a8fe05 | ||
|
|
202cbed30e | ||
|
|
a3443d36f1 | ||
|
|
75fe41e266 | ||
|
|
d16a90f08f | ||
|
|
904d8fc76f | ||
|
|
315149e8a2 | ||
|
|
699a1a7fa2 | ||
|
|
22539644a9 | ||
|
|
a48be517e4 | ||
|
|
2e0d556c33 | ||
|
|
ff75f2c758 | ||
|
|
e967019964 | ||
|
|
f6bb06ac6e | ||
|
|
8ee406533c | ||
|
|
373c96d6d6 | ||
|
|
fc6b97bf38 | ||
|
|
5047277845 | ||
|
|
6d1890ce04 | ||
|
|
523117ec28 | ||
|
|
86e1d096ef | ||
|
|
3f39f1f8ae | ||
|
|
d321315acf | ||
|
|
b124a65d6e | ||
|
|
d78b21c27f |
@@ -5,6 +5,15 @@ CHITAI_TOKEN_SECRET=secret
|
|||||||
CHITAI_DEFAULT_LIBRARY_NAME=Books
|
CHITAI_DEFAULT_LIBRARY_NAME=Books
|
||||||
CHITAI_DEFAULT_LIBRARY_PATH="libraries/books"
|
CHITAI_DEFAULT_LIBRARY_PATH="libraries/books"
|
||||||
|
|
||||||
|
# Duplicate detection when importing files (optional).
|
||||||
|
# Scope: "library" compares against the library being uploaded to, "global" against
|
||||||
|
# every library, "off" disables the check.
|
||||||
|
CHITAI_DUPLICATE_SCOPE=library
|
||||||
|
|
||||||
|
# Where the consume watcher parks files it refused as duplicates. Keep it outside
|
||||||
|
# CHITAI_CONSUME_PATH, or the watcher picks them straight back up.
|
||||||
|
CHITAI_DUPLICATE_PATH="duplicates"
|
||||||
|
|
||||||
# You probably should not change these
|
# You probably should not change these
|
||||||
CHITAI_API_URL="http://backend:8000"
|
CHITAI_API_URL="http://backend:8000"
|
||||||
CHITAI_API_DEBUG=false
|
CHITAI_API_DEBUG=false
|
||||||
|
|||||||
@@ -88,6 +88,14 @@ The backend owns files on disk, not just rows:
|
|||||||
- **Layout** — `services/filesystem_library.py` (`BookPathGenerator`) renders a Jinja2 template
|
- **Layout** — `services/filesystem_library.py` (`BookPathGenerator`) renders a Jinja2 template
|
||||||
against book metadata to decide where a book lives under the library's `root_path`
|
against book metadata to decide where a book lives under the library's `root_path`
|
||||||
(default: `author/series/position - title/`).
|
(default: `author/series/position - title/`).
|
||||||
|
- **One directory per book, never shared.** The generated path is a pure function of the metadata,
|
||||||
|
so two books with the same author and title produce the same one — two editions, or an
|
||||||
|
`allow_duplicates` copy. `BookService._reserve_book_path` moves the later one to `title (2)`
|
||||||
|
before anything is written, and `update_book` reserves the same way so a rename cannot move a
|
||||||
|
book in on top of another. This matters because `book.path` is what deletes, moves and file
|
||||||
|
lookups act on: books sharing a directory means one overwrites the other's files, and deleting
|
||||||
|
either takes both. `_unused_path` does the same job for filenames within a directory.
|
||||||
|
A book that already has a `path` keeps it — `add_files` must follow the book, not the template.
|
||||||
- **Metadata extraction** — `services/metadata_extractor.py` reads EPUB (ebooklib) and PDF
|
- **Metadata extraction** — `services/metadata_extractor.py` reads EPUB (ebooklib) and PDF
|
||||||
(pypdfium2) files; extracted values fill only *empty* fields on the incoming payload.
|
(pypdfium2) files; extracted values fill only *empty* fields on the incoming payload.
|
||||||
- **Covers** — converted to WebP with a UUID filename under `settings.book_cover_path`, served by a
|
- **Covers** — converted to WebP with a UUID filename under `settings.book_cover_path`, served by a
|
||||||
@@ -100,6 +108,140 @@ The backend owns files on disk, not just rows:
|
|||||||
if it differs, moves the directory contents and prunes empty parents. Keep that in mind before
|
if it differs, moves the directory contents and prunes empty parents. Keep that in mind before
|
||||||
changing metadata handling.
|
changing metadata handling.
|
||||||
|
|
||||||
|
## Duplicate detection
|
||||||
|
|
||||||
|
Every ingest path screens incoming files against what is already stored, keyed on
|
||||||
|
**`(hash, size)`** — never the hash alone, because it samples 12 KiB (see below) and
|
||||||
|
EPUBs from one toolchain often share their first window. `FileMetadata.hash` carries a
|
||||||
|
plain, deliberately **non-unique** index: a collision must not be able to fail an import,
|
||||||
|
and older databases may already hold duplicates.
|
||||||
|
|
||||||
|
Scope comes from `CHITAI_DUPLICATE_SCOPE` (`library`, the default | `global` | `off`).
|
||||||
|
|
||||||
|
The policy differs by how deliberate the import is:
|
||||||
|
|
||||||
|
| Path | Behaviour |
|
||||||
|
| --- | --- |
|
||||||
|
| `create_many_from_files` (browser bulk) | Skip per file, skip a whole group whose files are all known, report everything skipped in `ImportResult.duplicates`. Re-dropping a folder to pick up what is new is the case this serves. |
|
||||||
|
| `create_book` (single, with metadata) | All-or-nothing: raises `DuplicateFilesError`, which `controllers/book.py` renders as a **409** carrying the refused files in `extra`. |
|
||||||
|
| `add_files` | A file the book already carries is a no-op; one stored under another book raises `DuplicateFilesError`. |
|
||||||
|
| `create_many_from_existing_files` (consume watcher) | Skips, and **moves the file to `CHITAI_DUPLICATE_PATH/<library slug>/`** — nothing is deleted, and it cannot stay put because `watchfiles` only reports additions. That path must stay outside `consume_path` or the watcher re-imports it and tries to read the directory name as a library slug. |
|
||||||
|
|
||||||
|
`allow_duplicates=true` overrides all of it, on every endpoint. Keep that working — the
|
||||||
|
hash is not proof of identity, so a wrong verdict has to be recoverable, and a scripted
|
||||||
|
import needs a way through. The **web UI deliberately does not offer it**: storing the
|
||||||
|
same bytes twice splits reading progress and shelf membership across two records that
|
||||||
|
can never converge, which is nothing anyone wants on purpose.
|
||||||
|
|
||||||
|
Three things to preserve when touching this code:
|
||||||
|
|
||||||
|
- **A match only counts while the file is on disk.** `find_duplicate_files` stats each
|
||||||
|
candidate, and `add_files` writes a missing file back into the row that already
|
||||||
|
describes it (`_restore_file`) instead of adding a second row beside it. The hash
|
||||||
|
lives in the database and the file does not, so without this a file deleted behind
|
||||||
|
the app's back would go on refusing its own replacement.
|
||||||
|
|
||||||
|
- **Screening runs before anything is written.** `fingerprint_upload` reads the spooled
|
||||||
|
upload and rewinds it; the resulting fingerprints are handed to `_save_book_files`,
|
||||||
|
which skips its own `StreamingHasher` when it already has the answer. Passing them
|
||||||
|
through is what keeps the file from being read twice.
|
||||||
|
- **`_screen_for_duplicates` extends the `known` dict as it goes**, so the same bytes
|
||||||
|
submitted twice in one request are caught. Those duplicates report `book_id: None` —
|
||||||
|
there is no row to point at yet.
|
||||||
|
|
||||||
|
`POST /books/duplicate-files` answers the same question from fingerprints alone, for
|
||||||
|
clients that want to ask before uploading anything.
|
||||||
|
|
||||||
|
### Book-level detection — a different question
|
||||||
|
|
||||||
|
The file check answers "are these the same bytes?". `find_duplicate_books` answers "is
|
||||||
|
this the same book?", which a re-scan, a re-zipped EPUB or another edition cannot be
|
||||||
|
asked with a hash. Two signals, either sufficient: a **shared identifier**, or a
|
||||||
|
**matching normalized title with at least one shared author**.
|
||||||
|
|
||||||
|
It **never blocks**. A metadata match is a guess — a work shares title and author with
|
||||||
|
its own translation, its own second edition and its own audiobook — so the book is
|
||||||
|
created and the candidates are reported alongside it in
|
||||||
|
`ImportResult.possible_duplicates`. File-level dedupe keeps its refuse/skip behaviour;
|
||||||
|
that one is near-certain and this one is not. Do not "improve" this into a refusal.
|
||||||
|
|
||||||
|
Two rules narrow it, both applied in Python over the small candidate set:
|
||||||
|
|
||||||
|
- **A shared author is required for a title match.** Without it every book the
|
||||||
|
extractors gave up on and titled `Unknown` is a duplicate of every other one. A book
|
||||||
|
with no authors can therefore only match on an identifier.
|
||||||
|
- **The same series at a different `series_position` disqualifies a match.** A trilogy
|
||||||
|
shares an author and often most of its title; the position is the library saying
|
||||||
|
outright that these are two books.
|
||||||
|
|
||||||
|
A book is compared under **several title keys, not one** (`_title_keys`). Ebook files
|
||||||
|
are overwhelmingly named `Title - Author.epub`, and wherever nothing inside the file
|
||||||
|
overrode that name the author ended up in the title column — so one copy is stored as
|
||||||
|
`Building Microservices` and another as `Building Microservices Sam Newman`. Both
|
||||||
|
directions are generated, the author stripped off and the author added on, which is why
|
||||||
|
the query is `normalized_title.in_(keys)` rather than `==`. This is still exact matching
|
||||||
|
on an indexed column: no similarity score, nothing to tune. It does not weaken the
|
||||||
|
shared-author requirement, which is a separate condition.
|
||||||
|
|
||||||
|
`find_duplicate_book_groups` is the library-wide pass behind
|
||||||
|
`GET /books/duplicate-books`, since the import-time check says nothing about a
|
||||||
|
collection someone already has. It buckets books by every key they carry and merges the
|
||||||
|
buckets with union-find, so A~B by ISBN and B~C by title land in one group. Pairs in
|
||||||
|
`duplicate_dismissals` are never merged — a reader disagreeing with one pairing must not
|
||||||
|
silently break a group that stands on other evidence.
|
||||||
|
|
||||||
|
### Author names have one stored form
|
||||||
|
|
||||||
|
`Author.name` is always the canonical form, produced by `format_author_name`. Extractors
|
||||||
|
hand over whatever the file said — `Newman, Sam;` from a `DC:creator` list, `Sam Newman`
|
||||||
|
from a PDF, `Sam Newman.epub` from a filename — and storing those verbatim is how one
|
||||||
|
person becomes four rows in the sidebar, four entries in the author filter, and four
|
||||||
|
books that never look like each other.
|
||||||
|
|
||||||
|
This is **display** canonicalization, distinct from `normalize_author`, which throws
|
||||||
|
away case, accents and spacing to build a comparison key nobody sees. Tidying only
|
||||||
|
removes what an extractor added: a trailing separator, a file extension, and the
|
||||||
|
`Surname, Given` ordering. It never touches case or accents — `Michał Płachta` and
|
||||||
|
`Steve McConnell` are the author's own spelling, not something to correct.
|
||||||
|
|
||||||
|
Three places have to agree, and `Author` keeps them together: the `@validates("name")`
|
||||||
|
hook, `unique_hash`, and `unique_filter`. `as_unique_async` looks a row up with the
|
||||||
|
filter and then constructs with the validator, so if the lookup used the raw name and
|
||||||
|
the insert used the tidy one, every variant spelling would miss the existing row and
|
||||||
|
then collide with it on the unique index.
|
||||||
|
|
||||||
|
A form the rule does not recognise is **left exactly as it was found** — `Dave Thomas,
|
||||||
|
Andy Hunt` is two people in one string, and flipping it would invent a third. Leaving a
|
||||||
|
mess visible beats rewriting it wrongly.
|
||||||
|
|
||||||
|
### The normalized columns are written by validators
|
||||||
|
|
||||||
|
`Book.normalized_title`, `Author.normalized_name` and `Identifier.normalized_value` are
|
||||||
|
derived from `services/matching.py` and kept current by **SQLAlchemy `@validates` hooks
|
||||||
|
on the models**, not by any service. Assigning them directly is always wrong.
|
||||||
|
|
||||||
|
This is deliberate and it is invisible at the call sites: `BookService` writes titles
|
||||||
|
through at least three paths (`to_model_on_create`, `to_model_on_update`, and the
|
||||||
|
`setattr` loop in `_populate_with_unique_relationships`), and a validator is the only
|
||||||
|
thing a fourth cannot bypass. The columns carry plain, **non-unique** btree indexes —
|
||||||
|
two spellings collapsing onto one value is the entire point.
|
||||||
|
|
||||||
|
`services/matching.py` is imported *inside* those validators rather than at module
|
||||||
|
scope: reaching it initialises the `chitai.services` package, which imports the
|
||||||
|
services, which import the models. Keep the local import.
|
||||||
|
|
||||||
|
The validators only fire on write, so a migration that adds one of these columns must
|
||||||
|
backfill existing rows through the same helpers — see the `data_upgrades()` hook in
|
||||||
|
`2026-08-15_add_book_matching_keys_and_duplicate__4358e7d4743a.py`.
|
||||||
|
|
||||||
|
**Changing anything in `services/matching.py` needs a revision that recomputes them.**
|
||||||
|
The keys are derived and already written, so a normalization change silently invalidates
|
||||||
|
every stored row: a book written under the old rules just stops matching one written
|
||||||
|
under the new rules, with nothing to show that anything is wrong. Copy
|
||||||
|
`2026-08-15_recompute_book_matching_keys_ed41acf21270.py`, which exists because
|
||||||
|
`normalize_title` learned to strip compact edition markers (`2E`, `5e`). It is
|
||||||
|
idempotent and safe to re-run.
|
||||||
|
|
||||||
## KOReader hashing
|
## KOReader hashing
|
||||||
|
|
||||||
`services/utils.py` reimplements KOReader's partial-MD5 document identifier: 1 KiB samples at
|
`services/utils.py` reimplements KOReader's partial-MD5 document identifier: 1 KiB samples at
|
||||||
|
|||||||
@@ -0,0 +1,80 @@
|
|||||||
|
"""add file hash index
|
||||||
|
|
||||||
|
Revision ID: e9c2c7e875ae
|
||||||
|
Revises: 6d72d1bbc0ee
|
||||||
|
Create Date: 2026-08-13 14:52:09.341906
|
||||||
|
|
||||||
|
"""
|
||||||
|
|
||||||
|
import warnings
|
||||||
|
from typing import TYPE_CHECKING
|
||||||
|
|
||||||
|
import sqlalchemy as sa
|
||||||
|
from alembic import op
|
||||||
|
from advanced_alchemy.types import EncryptedString, EncryptedText, GUID, ORA_JSONB, DateTimeUTC, StoredObject, PasswordHash, FernetBackend
|
||||||
|
from advanced_alchemy.types.encrypted_string import PGCryptoBackend
|
||||||
|
from advanced_alchemy.types.password_hash.argon2 import Argon2Hasher
|
||||||
|
from advanced_alchemy.types.password_hash.passlib import PasslibHasher
|
||||||
|
from advanced_alchemy.types.password_hash.pwdlib import PwdlibHasher
|
||||||
|
from sqlalchemy import Text # noqa: F401
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from collections.abc import Sequence
|
||||||
|
|
||||||
|
__all__ = ["downgrade", "upgrade", "schema_upgrades", "schema_downgrades", "data_upgrades", "data_downgrades"]
|
||||||
|
|
||||||
|
sa.GUID = GUID
|
||||||
|
sa.DateTimeUTC = DateTimeUTC
|
||||||
|
sa.ORA_JSONB = ORA_JSONB
|
||||||
|
sa.EncryptedString = EncryptedString
|
||||||
|
sa.EncryptedText = EncryptedText
|
||||||
|
sa.StoredObject = StoredObject
|
||||||
|
sa.PasswordHash = PasswordHash
|
||||||
|
sa.Argon2Hasher = Argon2Hasher
|
||||||
|
sa.PasslibHasher = PasslibHasher
|
||||||
|
sa.PwdlibHasher = PwdlibHasher
|
||||||
|
sa.FernetBackend = FernetBackend
|
||||||
|
sa.PGCryptoBackend = PGCryptoBackend
|
||||||
|
|
||||||
|
# revision identifiers, used by Alembic.
|
||||||
|
revision = 'e9c2c7e875ae'
|
||||||
|
down_revision = '6d72d1bbc0ee'
|
||||||
|
branch_labels = None
|
||||||
|
depends_on = None
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
with warnings.catch_warnings():
|
||||||
|
warnings.filterwarnings("ignore", category=UserWarning)
|
||||||
|
with op.get_context().autocommit_block():
|
||||||
|
schema_upgrades()
|
||||||
|
data_upgrades()
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
with warnings.catch_warnings():
|
||||||
|
warnings.filterwarnings("ignore", category=UserWarning)
|
||||||
|
with op.get_context().autocommit_block():
|
||||||
|
data_downgrades()
|
||||||
|
schema_downgrades()
|
||||||
|
|
||||||
|
def schema_upgrades() -> None:
|
||||||
|
"""schema upgrade migrations go here."""
|
||||||
|
# ### commands auto generated by Alembic - please adjust! ###
|
||||||
|
with op.batch_alter_table('file_metadata', schema=None) as batch_op:
|
||||||
|
batch_op.create_index('ix_file_metadata_hash', ['hash'], unique=False)
|
||||||
|
|
||||||
|
# ### end Alembic commands ###
|
||||||
|
|
||||||
|
def schema_downgrades() -> None:
|
||||||
|
"""schema downgrade migrations go here."""
|
||||||
|
# ### commands auto generated by Alembic - please adjust! ###
|
||||||
|
with op.batch_alter_table('file_metadata', schema=None) as batch_op:
|
||||||
|
batch_op.drop_index('ix_file_metadata_hash')
|
||||||
|
|
||||||
|
# ### end Alembic commands ###
|
||||||
|
|
||||||
|
def data_upgrades() -> None:
|
||||||
|
"""Add any optional data upgrade migrations here!"""
|
||||||
|
|
||||||
|
def data_downgrades() -> None:
|
||||||
|
"""Add any optional data downgrade migrations here!"""
|
||||||
+163
@@ -0,0 +1,163 @@
|
|||||||
|
"""add book matching keys and duplicate dismissals
|
||||||
|
|
||||||
|
Revision ID: 4358e7d4743a
|
||||||
|
Revises: e9c2c7e875ae
|
||||||
|
Create Date: 2026-08-15 15:09:02.708914
|
||||||
|
|
||||||
|
"""
|
||||||
|
|
||||||
|
import warnings
|
||||||
|
|
||||||
|
import sqlalchemy as sa
|
||||||
|
from alembic import op
|
||||||
|
from advanced_alchemy.types import EncryptedString, EncryptedText, GUID, ORA_JSONB, DateTimeUTC, StoredObject, PasswordHash, FernetBackend
|
||||||
|
from advanced_alchemy.types.encrypted_string import PGCryptoBackend
|
||||||
|
from advanced_alchemy.types.password_hash.argon2 import Argon2Hasher
|
||||||
|
from advanced_alchemy.types.password_hash.passlib import PasslibHasher
|
||||||
|
from advanced_alchemy.types.password_hash.pwdlib import PwdlibHasher
|
||||||
|
from sqlalchemy import Text # noqa: F401
|
||||||
|
|
||||||
|
__all__ = ["downgrade", "upgrade", "schema_upgrades", "schema_downgrades", "data_upgrades", "data_downgrades"]
|
||||||
|
|
||||||
|
sa.GUID = GUID
|
||||||
|
sa.DateTimeUTC = DateTimeUTC
|
||||||
|
sa.ORA_JSONB = ORA_JSONB
|
||||||
|
sa.EncryptedString = EncryptedString
|
||||||
|
sa.EncryptedText = EncryptedText
|
||||||
|
sa.StoredObject = StoredObject
|
||||||
|
sa.PasswordHash = PasswordHash
|
||||||
|
sa.Argon2Hasher = Argon2Hasher
|
||||||
|
sa.PasslibHasher = PasslibHasher
|
||||||
|
sa.PwdlibHasher = PwdlibHasher
|
||||||
|
sa.FernetBackend = FernetBackend
|
||||||
|
sa.PGCryptoBackend = PGCryptoBackend
|
||||||
|
|
||||||
|
# revision identifiers, used by Alembic.
|
||||||
|
revision = '4358e7d4743a'
|
||||||
|
down_revision = 'e9c2c7e875ae'
|
||||||
|
branch_labels = None
|
||||||
|
depends_on = None
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
with warnings.catch_warnings():
|
||||||
|
warnings.filterwarnings("ignore", category=UserWarning)
|
||||||
|
with op.get_context().autocommit_block():
|
||||||
|
schema_upgrades()
|
||||||
|
data_upgrades()
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
with warnings.catch_warnings():
|
||||||
|
warnings.filterwarnings("ignore", category=UserWarning)
|
||||||
|
with op.get_context().autocommit_block():
|
||||||
|
data_downgrades()
|
||||||
|
schema_downgrades()
|
||||||
|
|
||||||
|
def schema_upgrades() -> None:
|
||||||
|
"""schema upgrade migrations go here."""
|
||||||
|
# ### commands auto generated by Alembic - please adjust! ###
|
||||||
|
op.create_table('duplicate_dismissals',
|
||||||
|
sa.Column('id', sa.BigInteger().with_variant(sa.Integer(), 'sqlite'), nullable=False),
|
||||||
|
sa.Column('book_a_id', sa.BigInteger().with_variant(sa.Integer(), 'sqlite'), nullable=False),
|
||||||
|
sa.Column('book_b_id', sa.BigInteger().with_variant(sa.Integer(), 'sqlite'), nullable=False),
|
||||||
|
sa.ForeignKeyConstraint(['book_a_id'], ['books.id'], name=op.f('fk_duplicate_dismissals_book_a_id_books'), ondelete='cascade'),
|
||||||
|
sa.ForeignKeyConstraint(['book_b_id'], ['books.id'], name=op.f('fk_duplicate_dismissals_book_b_id_books'), ondelete='cascade'),
|
||||||
|
sa.PrimaryKeyConstraint('id', name=op.f('pk_duplicate_dismissals')),
|
||||||
|
sa.UniqueConstraint('book_a_id', 'book_b_id', name=op.f('uq_duplicate_dismissals_book_a_id'))
|
||||||
|
)
|
||||||
|
with op.batch_alter_table('duplicate_dismissals', schema=None) as batch_op:
|
||||||
|
batch_op.create_index(batch_op.f('ix_duplicate_dismissals_book_a_id'), ['book_a_id'], unique=False)
|
||||||
|
batch_op.create_index(batch_op.f('ix_duplicate_dismissals_book_b_id'), ['book_b_id'], unique=False)
|
||||||
|
|
||||||
|
# `server_default` so the column can be added to a table that already has rows;
|
||||||
|
# `data_upgrades` fills in the real keys immediately afterwards.
|
||||||
|
with op.batch_alter_table('authors', schema=None) as batch_op:
|
||||||
|
batch_op.add_column(sa.Column('normalized_name', sa.String(), nullable=False, server_default=''))
|
||||||
|
batch_op.create_index(batch_op.f('ix_authors_normalized_name'), ['normalized_name'], unique=False)
|
||||||
|
|
||||||
|
with op.batch_alter_table('books', schema=None) as batch_op:
|
||||||
|
batch_op.add_column(sa.Column('normalized_title', sa.String(), nullable=False, server_default=''))
|
||||||
|
batch_op.create_index(batch_op.f('ix_books_normalized_title'), ['normalized_title'], unique=False)
|
||||||
|
|
||||||
|
with op.batch_alter_table('identifiers', schema=None) as batch_op:
|
||||||
|
batch_op.add_column(sa.Column('normalized_value', sa.String(), nullable=True))
|
||||||
|
batch_op.create_index(batch_op.f('ix_identifiers_normalized_value'), ['normalized_value'], unique=False)
|
||||||
|
|
||||||
|
# ### end Alembic commands ###
|
||||||
|
|
||||||
|
def schema_downgrades() -> None:
|
||||||
|
"""schema downgrade migrations go here."""
|
||||||
|
# ### commands auto generated by Alembic - please adjust! ###
|
||||||
|
with op.batch_alter_table('identifiers', schema=None) as batch_op:
|
||||||
|
batch_op.drop_index(batch_op.f('ix_identifiers_normalized_value'))
|
||||||
|
batch_op.drop_column('normalized_value')
|
||||||
|
|
||||||
|
with op.batch_alter_table('books', schema=None) as batch_op:
|
||||||
|
batch_op.drop_index(batch_op.f('ix_books_normalized_title'))
|
||||||
|
batch_op.drop_column('normalized_title')
|
||||||
|
|
||||||
|
with op.batch_alter_table('authors', schema=None) as batch_op:
|
||||||
|
batch_op.drop_index(batch_op.f('ix_authors_normalized_name'))
|
||||||
|
batch_op.drop_column('normalized_name')
|
||||||
|
|
||||||
|
with op.batch_alter_table('duplicate_dismissals', schema=None) as batch_op:
|
||||||
|
batch_op.drop_index(batch_op.f('ix_duplicate_dismissals_book_b_id'))
|
||||||
|
batch_op.drop_index(batch_op.f('ix_duplicate_dismissals_book_a_id'))
|
||||||
|
|
||||||
|
op.drop_table('duplicate_dismissals')
|
||||||
|
# ### end Alembic commands ###
|
||||||
|
|
||||||
|
def data_upgrades() -> None:
|
||||||
|
"""
|
||||||
|
Fill the matching keys in for rows that already exist.
|
||||||
|
|
||||||
|
The validators on the models only fire when something is written, so without this
|
||||||
|
every book imported before today is invisible to duplicate detection. Run through
|
||||||
|
the same helpers the validators use, so a backfilled row and a freshly written one
|
||||||
|
are guaranteed to agree.
|
||||||
|
"""
|
||||||
|
from chitai.services.matching import (
|
||||||
|
normalize_author,
|
||||||
|
normalize_identifier,
|
||||||
|
normalize_title,
|
||||||
|
)
|
||||||
|
|
||||||
|
connection = op.get_bind()
|
||||||
|
|
||||||
|
books = connection.execute(sa.text("SELECT id, title FROM books")).fetchall()
|
||||||
|
_apply(
|
||||||
|
connection,
|
||||||
|
"UPDATE books SET normalized_title = :key WHERE id = :id",
|
||||||
|
[{"id": id, "key": normalize_title(title)} for id, title in books],
|
||||||
|
)
|
||||||
|
|
||||||
|
authors = connection.execute(sa.text("SELECT id, name FROM authors")).fetchall()
|
||||||
|
_apply(
|
||||||
|
connection,
|
||||||
|
"UPDATE authors SET normalized_name = :key WHERE id = :id",
|
||||||
|
[{"id": id, "key": normalize_author(name)} for id, name in authors],
|
||||||
|
)
|
||||||
|
|
||||||
|
identifiers = connection.execute(
|
||||||
|
sa.text("SELECT id, name, value FROM identifiers")
|
||||||
|
).fetchall()
|
||||||
|
_apply(
|
||||||
|
connection,
|
||||||
|
"UPDATE identifiers SET normalized_value = :key WHERE id = :id",
|
||||||
|
[
|
||||||
|
{"id": id, "key": normalize_identifier(name, value)}
|
||||||
|
for id, name, value in identifiers
|
||||||
|
],
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _apply(connection, statement: str, parameters: list[dict]) -> None:
|
||||||
|
"""Run one update per row, in batches, skipping the work when there are none."""
|
||||||
|
batch_size = 1000
|
||||||
|
|
||||||
|
for start in range(0, len(parameters), batch_size):
|
||||||
|
connection.execute(sa.text(statement), parameters[start : start + batch_size])
|
||||||
|
|
||||||
|
|
||||||
|
def data_downgrades() -> None:
|
||||||
|
"""Add any optional data downgrade migrations here!"""
|
||||||
@@ -0,0 +1,130 @@
|
|||||||
|
"""canonicalize author names
|
||||||
|
|
||||||
|
Revision ID: 49a9e85a0ffc
|
||||||
|
Revises: ed41acf21270
|
||||||
|
Create Date: 2026-08-15 15:59:47.331545
|
||||||
|
|
||||||
|
"""
|
||||||
|
|
||||||
|
import warnings
|
||||||
|
|
||||||
|
import sqlalchemy as sa
|
||||||
|
from alembic import op
|
||||||
|
from advanced_alchemy.types import EncryptedString, EncryptedText, GUID, ORA_JSONB, DateTimeUTC, StoredObject, PasswordHash, FernetBackend
|
||||||
|
from advanced_alchemy.types.encrypted_string import PGCryptoBackend
|
||||||
|
from advanced_alchemy.types.password_hash.argon2 import Argon2Hasher
|
||||||
|
from advanced_alchemy.types.password_hash.passlib import PasslibHasher
|
||||||
|
from advanced_alchemy.types.password_hash.pwdlib import PwdlibHasher
|
||||||
|
from sqlalchemy import Text # noqa: F401
|
||||||
|
|
||||||
|
__all__ = ["downgrade", "upgrade", "schema_upgrades", "schema_downgrades", "data_upgrades", "data_downgrades"]
|
||||||
|
|
||||||
|
sa.GUID = GUID
|
||||||
|
sa.DateTimeUTC = DateTimeUTC
|
||||||
|
sa.ORA_JSONB = ORA_JSONB
|
||||||
|
sa.EncryptedString = EncryptedString
|
||||||
|
sa.EncryptedText = EncryptedText
|
||||||
|
sa.StoredObject = StoredObject
|
||||||
|
sa.PasswordHash = PasswordHash
|
||||||
|
sa.Argon2Hasher = Argon2Hasher
|
||||||
|
sa.PasslibHasher = PasslibHasher
|
||||||
|
sa.PwdlibHasher = PwdlibHasher
|
||||||
|
sa.FernetBackend = FernetBackend
|
||||||
|
sa.PGCryptoBackend = PGCryptoBackend
|
||||||
|
|
||||||
|
# revision identifiers, used by Alembic.
|
||||||
|
revision = '49a9e85a0ffc'
|
||||||
|
down_revision = 'ed41acf21270'
|
||||||
|
branch_labels = None
|
||||||
|
depends_on = None
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
with warnings.catch_warnings():
|
||||||
|
warnings.filterwarnings("ignore", category=UserWarning)
|
||||||
|
with op.get_context().autocommit_block():
|
||||||
|
schema_upgrades()
|
||||||
|
data_upgrades()
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
with warnings.catch_warnings():
|
||||||
|
warnings.filterwarnings("ignore", category=UserWarning)
|
||||||
|
with op.get_context().autocommit_block():
|
||||||
|
data_downgrades()
|
||||||
|
schema_downgrades()
|
||||||
|
|
||||||
|
def schema_upgrades() -> None:
|
||||||
|
"""schema upgrade migrations go here."""
|
||||||
|
pass
|
||||||
|
|
||||||
|
def schema_downgrades() -> None:
|
||||||
|
"""schema downgrade migrations go here."""
|
||||||
|
pass
|
||||||
|
|
||||||
|
def data_upgrades() -> None:
|
||||||
|
"""
|
||||||
|
Rewrite every author into the canonical form, merging the rows that collide.
|
||||||
|
|
||||||
|
`Author.name` is only now guaranteed tidy — until this revision extractors wrote
|
||||||
|
whatever the file said, so one person could hold several rows: "Sam Newman" beside
|
||||||
|
"Newman, Sam;" beside "Sam Newman.epub" (the last from a filename whose extension
|
||||||
|
was never stripped). Each showed up as its own author in the sidebar and its own
|
||||||
|
filter, and no amount of fixing the extractors repairs a row already written.
|
||||||
|
|
||||||
|
Rows that canonicalize onto one name are merged into the lowest id, which keeps
|
||||||
|
whichever row the library has been referring to longest. `book.path` is stored, not
|
||||||
|
derived, so renaming an author moves nothing on disk.
|
||||||
|
"""
|
||||||
|
from chitai.services.matching import format_author_name, normalize_author
|
||||||
|
|
||||||
|
connection = op.get_bind()
|
||||||
|
authors = connection.execute(sa.text("SELECT id, name FROM authors")).fetchall()
|
||||||
|
|
||||||
|
groups: dict[str, list[int]] = {}
|
||||||
|
for id, name in sorted(authors):
|
||||||
|
# A name with nothing left of it after tidying is left exactly as it was:
|
||||||
|
# merging those together would invent one author out of several unrelated
|
||||||
|
# broken rows, which is worse than leaving the mess visible.
|
||||||
|
if canonical := format_author_name(name):
|
||||||
|
groups.setdefault(canonical, []).append(id)
|
||||||
|
|
||||||
|
for canonical, ids in groups.items():
|
||||||
|
winner, losers = ids[0], ids[1:]
|
||||||
|
|
||||||
|
for loser in losers:
|
||||||
|
# A book credited to both rows would otherwise breach the
|
||||||
|
# (book_id, author_id) unique constraint the moment the link is repointed.
|
||||||
|
connection.execute(
|
||||||
|
sa.text(
|
||||||
|
"DELETE FROM book_author_links WHERE author_id = :loser AND book_id IN"
|
||||||
|
" (SELECT book_id FROM book_author_links WHERE author_id = :winner)"
|
||||||
|
),
|
||||||
|
{"loser": loser, "winner": winner},
|
||||||
|
)
|
||||||
|
connection.execute(
|
||||||
|
sa.text(
|
||||||
|
"UPDATE book_author_links SET author_id = :winner"
|
||||||
|
" WHERE author_id = :loser"
|
||||||
|
),
|
||||||
|
{"loser": loser, "winner": winner},
|
||||||
|
)
|
||||||
|
connection.execute(
|
||||||
|
sa.text("DELETE FROM authors WHERE id = :loser"), {"loser": loser}
|
||||||
|
)
|
||||||
|
|
||||||
|
# Only after the losers are gone, or this collides with the unique index.
|
||||||
|
connection.execute(
|
||||||
|
sa.text(
|
||||||
|
"UPDATE authors SET name = :name, normalized_name = :key WHERE id = :id"
|
||||||
|
),
|
||||||
|
{"id": winner, "name": canonical, "key": normalize_author(canonical)},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def data_downgrades() -> None:
|
||||||
|
"""
|
||||||
|
Nothing to undo.
|
||||||
|
|
||||||
|
The rows a merge removed are gone, and the spellings it replaced were never
|
||||||
|
recorded anywhere else — there is nothing to restore them from.
|
||||||
|
"""
|
||||||
@@ -0,0 +1,122 @@
|
|||||||
|
"""recompute book matching keys
|
||||||
|
|
||||||
|
Revision ID: ed41acf21270
|
||||||
|
Revises: 4358e7d4743a
|
||||||
|
Create Date: 2026-08-15 15:44:28.341020
|
||||||
|
|
||||||
|
"""
|
||||||
|
|
||||||
|
import warnings
|
||||||
|
|
||||||
|
import sqlalchemy as sa
|
||||||
|
from alembic import op
|
||||||
|
from advanced_alchemy.types import EncryptedString, EncryptedText, GUID, ORA_JSONB, DateTimeUTC, StoredObject, PasswordHash, FernetBackend
|
||||||
|
from advanced_alchemy.types.encrypted_string import PGCryptoBackend
|
||||||
|
from advanced_alchemy.types.password_hash.argon2 import Argon2Hasher
|
||||||
|
from advanced_alchemy.types.password_hash.passlib import PasslibHasher
|
||||||
|
from advanced_alchemy.types.password_hash.pwdlib import PwdlibHasher
|
||||||
|
from sqlalchemy import Text # noqa: F401
|
||||||
|
|
||||||
|
__all__ = ["downgrade", "upgrade", "schema_upgrades", "schema_downgrades", "data_upgrades", "data_downgrades"]
|
||||||
|
|
||||||
|
sa.GUID = GUID
|
||||||
|
sa.DateTimeUTC = DateTimeUTC
|
||||||
|
sa.ORA_JSONB = ORA_JSONB
|
||||||
|
sa.EncryptedString = EncryptedString
|
||||||
|
sa.EncryptedText = EncryptedText
|
||||||
|
sa.StoredObject = StoredObject
|
||||||
|
sa.PasswordHash = PasswordHash
|
||||||
|
sa.Argon2Hasher = Argon2Hasher
|
||||||
|
sa.PasslibHasher = PasslibHasher
|
||||||
|
sa.PwdlibHasher = PwdlibHasher
|
||||||
|
sa.FernetBackend = FernetBackend
|
||||||
|
sa.PGCryptoBackend = PGCryptoBackend
|
||||||
|
|
||||||
|
# revision identifiers, used by Alembic.
|
||||||
|
revision = 'ed41acf21270'
|
||||||
|
down_revision = '4358e7d4743a'
|
||||||
|
branch_labels = None
|
||||||
|
depends_on = None
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
with warnings.catch_warnings():
|
||||||
|
warnings.filterwarnings("ignore", category=UserWarning)
|
||||||
|
with op.get_context().autocommit_block():
|
||||||
|
schema_upgrades()
|
||||||
|
data_upgrades()
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
with warnings.catch_warnings():
|
||||||
|
warnings.filterwarnings("ignore", category=UserWarning)
|
||||||
|
with op.get_context().autocommit_block():
|
||||||
|
data_downgrades()
|
||||||
|
schema_downgrades()
|
||||||
|
|
||||||
|
def schema_upgrades() -> None:
|
||||||
|
"""schema upgrade migrations go here."""
|
||||||
|
pass
|
||||||
|
|
||||||
|
def schema_downgrades() -> None:
|
||||||
|
"""schema downgrade migrations go here."""
|
||||||
|
pass
|
||||||
|
|
||||||
|
def data_upgrades() -> None:
|
||||||
|
"""
|
||||||
|
Recompute every matching key against the current normalization.
|
||||||
|
|
||||||
|
The keys are derived, so changing a helper in `services/matching.py` silently
|
||||||
|
invalidates every row already written — a book stored under the old rules simply
|
||||||
|
stops matching one stored under the new ones, with nothing to show that anything
|
||||||
|
is wrong. `normalize_title` learned to strip the compact edition markers a cover
|
||||||
|
actually carries ("2E", "5e"), which moved `Building Microservices, 2E` onto the
|
||||||
|
same key as `Building Microservices`.
|
||||||
|
|
||||||
|
Any later change to those helpers wants a revision that looks exactly like this
|
||||||
|
one. It is idempotent and safe to re-run.
|
||||||
|
"""
|
||||||
|
from chitai.services.matching import (
|
||||||
|
normalize_author,
|
||||||
|
normalize_identifier,
|
||||||
|
normalize_title,
|
||||||
|
)
|
||||||
|
|
||||||
|
connection = op.get_bind()
|
||||||
|
|
||||||
|
books = connection.execute(sa.text("SELECT id, title FROM books")).fetchall()
|
||||||
|
_apply(
|
||||||
|
connection,
|
||||||
|
"UPDATE books SET normalized_title = :key WHERE id = :id",
|
||||||
|
[{"id": id, "key": normalize_title(title)} for id, title in books],
|
||||||
|
)
|
||||||
|
|
||||||
|
authors = connection.execute(sa.text("SELECT id, name FROM authors")).fetchall()
|
||||||
|
_apply(
|
||||||
|
connection,
|
||||||
|
"UPDATE authors SET normalized_name = :key WHERE id = :id",
|
||||||
|
[{"id": id, "key": normalize_author(name)} for id, name in authors],
|
||||||
|
)
|
||||||
|
|
||||||
|
identifiers = connection.execute(
|
||||||
|
sa.text("SELECT id, name, value FROM identifiers")
|
||||||
|
).fetchall()
|
||||||
|
_apply(
|
||||||
|
connection,
|
||||||
|
"UPDATE identifiers SET normalized_value = :key WHERE id = :id",
|
||||||
|
[
|
||||||
|
{"id": id, "key": normalize_identifier(name, value)}
|
||||||
|
for id, name, value in identifiers
|
||||||
|
],
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _apply(connection, statement: str, parameters: list[dict]) -> None:
|
||||||
|
"""Run one update per row, in batches, skipping the work when there are none."""
|
||||||
|
batch_size = 1000
|
||||||
|
|
||||||
|
for start in range(0, len(parameters), batch_size):
|
||||||
|
connection.execute(sa.text(statement), parameters[start : start + batch_size])
|
||||||
|
|
||||||
|
|
||||||
|
def data_downgrades() -> None:
|
||||||
|
"""Add any optional data downgrade migrations here!"""
|
||||||
@@ -1,9 +1,25 @@
|
|||||||
|
from enum import StrEnum
|
||||||
|
|
||||||
from pydantic import Field, PostgresDsn, computed_field
|
from pydantic import Field, PostgresDsn, computed_field
|
||||||
from pydantic_settings import BaseSettings, SettingsConfigDict
|
from pydantic_settings import BaseSettings, SettingsConfigDict
|
||||||
from advanced_alchemy.extensions.litestar import (
|
from advanced_alchemy.extensions.litestar import (
|
||||||
SQLAlchemyAsyncConfig,
|
SQLAlchemyAsyncConfig,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class DuplicateScope(StrEnum):
|
||||||
|
"""How widely an incoming file is compared against what is already stored."""
|
||||||
|
|
||||||
|
LIBRARY = "library"
|
||||||
|
"""Only files in the library being uploaded to count as duplicates."""
|
||||||
|
|
||||||
|
GLOBAL = "global"
|
||||||
|
"""A file already held by any library counts as a duplicate."""
|
||||||
|
|
||||||
|
OFF = "off"
|
||||||
|
"""No duplicate detection at all."""
|
||||||
|
|
||||||
|
|
||||||
class Settings(BaseSettings):
|
class Settings(BaseSettings):
|
||||||
version: str = Field("0.0.1")
|
version: str = Field("0.0.1")
|
||||||
project_name: str = Field("chitai")
|
project_name: str = Field("chitai")
|
||||||
@@ -33,6 +49,14 @@ class Settings(BaseSettings):
|
|||||||
# Path to consume directory
|
# Path to consume directory
|
||||||
consume_path: str = Field("./consume")
|
consume_path: str = Field("./consume")
|
||||||
|
|
||||||
|
# Duplicate detection
|
||||||
|
duplicate_scope: DuplicateScope = Field(DuplicateScope.LIBRARY)
|
||||||
|
|
||||||
|
# Where the consume watcher parks files it refused as duplicates. Must sit
|
||||||
|
# outside `consume_path`, or the watcher picks them straight back up and
|
||||||
|
# tries to resolve the directory name as a library slug.
|
||||||
|
duplicate_path: str = Field("./duplicates")
|
||||||
|
|
||||||
@computed_field
|
@computed_field
|
||||||
@property
|
@property
|
||||||
def postgres_uri(self) -> PostgresDsn:
|
def postgres_uri(self) -> PostgresDsn:
|
||||||
|
|||||||
@@ -12,7 +12,12 @@ from litestar.params import Dependency, Body
|
|||||||
from litestar.enums import RequestEncodingType
|
from litestar.enums import RequestEncodingType
|
||||||
from litestar.response import File, Stream
|
from litestar.response import File, Stream
|
||||||
from litestar.exceptions import HTTPException
|
from litestar.exceptions import HTTPException
|
||||||
from litestar.status_codes import HTTP_400_BAD_REQUEST
|
from litestar.status_codes import (
|
||||||
|
HTTP_200_OK,
|
||||||
|
HTTP_204_NO_CONTENT,
|
||||||
|
HTTP_400_BAD_REQUEST,
|
||||||
|
HTTP_409_CONFLICT,
|
||||||
|
)
|
||||||
from litestar.datastructures import UploadFile
|
from litestar.datastructures import UploadFile
|
||||||
from advanced_alchemy.service.pagination import OffsetPagination
|
from advanced_alchemy.service.pagination import OffsetPagination
|
||||||
from advanced_alchemy.filters import CollectionFilter
|
from advanced_alchemy.filters import CollectionFilter
|
||||||
@@ -23,6 +28,24 @@ from chitai.services import dependencies as deps
|
|||||||
from chitai import schemas as s
|
from chitai import schemas as s
|
||||||
from chitai.database import models as m
|
from chitai.database import models as m
|
||||||
from chitai.services import BookService, BookProgressService
|
from chitai.services import BookService, BookProgressService
|
||||||
|
from chitai.services.book import DuplicateFilesError
|
||||||
|
|
||||||
|
|
||||||
|
def _duplicate_conflict(exc: DuplicateFilesError) -> HTTPException:
|
||||||
|
"""
|
||||||
|
Turn refused files into a 409 the caller can act on.
|
||||||
|
|
||||||
|
The files ride along in `extra` so the client can name them and offer to send them
|
||||||
|
again with `allow_duplicates`, rather than being told only that something clashed.
|
||||||
|
"""
|
||||||
|
return HTTPException(
|
||||||
|
status_code=HTTP_409_CONFLICT,
|
||||||
|
detail="These files are already in the library",
|
||||||
|
extra=[
|
||||||
|
s.DuplicateFileRead.model_validate(duplicate).model_dump(mode="json")
|
||||||
|
for duplicate in exc.duplicates
|
||||||
|
],
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
class BookController(Controller):
|
class BookController(Controller):
|
||||||
@@ -63,6 +86,7 @@ class BookController(Controller):
|
|||||||
books_service: BookService,
|
books_service: BookService,
|
||||||
library: m.Library,
|
library: m.Library,
|
||||||
data: Annotated[s.BookCreate, Body(media_type=RequestEncodingType.MULTI_PART)],
|
data: Annotated[s.BookCreate, Body(media_type=RequestEncodingType.MULTI_PART)],
|
||||||
|
allow_duplicates: bool = False,
|
||||||
) -> s.BookRead:
|
) -> s.BookRead:
|
||||||
"""
|
"""
|
||||||
Create a new book with metadata and files.
|
Create a new book with metadata and files.
|
||||||
@@ -73,6 +97,10 @@ class BookController(Controller):
|
|||||||
Path Parameters:
|
Path Parameters:
|
||||||
library_id: The ID of the library the book belongs to.
|
library_id: The ID of the library the book belongs to.
|
||||||
|
|
||||||
|
Query Parameters:
|
||||||
|
allow_duplicates: If True, store the files even if the library already
|
||||||
|
holds them.
|
||||||
|
|
||||||
Request Body:
|
Request Body:
|
||||||
data: Book creation data including metadata and files.
|
data: Book creation data including metadata and files.
|
||||||
|
|
||||||
@@ -83,9 +111,17 @@ class BookController(Controller):
|
|||||||
Returns:
|
Returns:
|
||||||
The created book as a BookRead schema.
|
The created book as a BookRead schema.
|
||||||
|
|
||||||
|
Raises:
|
||||||
|
HTTPException: 409 if any of the files is already in the library.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
result = await books_service.create_book(data, library)
|
try:
|
||||||
|
result = await books_service.create_book(
|
||||||
|
data, library, screen_duplicates=not allow_duplicates
|
||||||
|
)
|
||||||
|
except DuplicateFilesError as exc:
|
||||||
|
raise _duplicate_conflict(exc)
|
||||||
|
|
||||||
book = await books_service.get(result.id)
|
book = await books_service.get(result.id)
|
||||||
return books_service.to_schema(book, schema_type=s.BookRead)
|
return books_service.to_schema(book, schema_type=s.BookRead)
|
||||||
|
|
||||||
@@ -97,13 +133,20 @@ class BookController(Controller):
|
|||||||
data: Annotated[
|
data: Annotated[
|
||||||
s.BooksCreateFromFiles, Body(media_type=RequestEncodingType.MULTI_PART)
|
s.BooksCreateFromFiles, Body(media_type=RequestEncodingType.MULTI_PART)
|
||||||
],
|
],
|
||||||
) -> OffsetPagination[s.BookRead]:
|
allow_duplicates: bool = False,
|
||||||
|
) -> s.BooksUploadResult:
|
||||||
"""
|
"""
|
||||||
Create multiple books from uploaded files.
|
Create multiple books from uploaded files.
|
||||||
|
|
||||||
Groups files by directory and creates separate books for each group.
|
Groups files by directory and creates separate books for each group.
|
||||||
Metadata is automatically extracted from the files.
|
Metadata is automatically extracted from the files.
|
||||||
|
|
||||||
|
Files the library already holds are skipped rather than refused, and reported
|
||||||
|
back so the caller can say which ones did not make it in and why.
|
||||||
|
|
||||||
|
Query Parameters:
|
||||||
|
allow_duplicates: If True, store every file, even one already held.
|
||||||
|
|
||||||
Request Body:
|
Request Body:
|
||||||
data: Container with list of uploaded files.
|
data: Container with list of uploaded files.
|
||||||
|
|
||||||
@@ -112,19 +155,200 @@ class BookController(Controller):
|
|||||||
library: The library the books belong to.
|
library: The library the books belong to.
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
Paginated list of created books.
|
The books created, and the files skipped as duplicates.
|
||||||
"""
|
"""
|
||||||
try:
|
try:
|
||||||
results = await books_service.create_many_from_files(data, library)
|
result = await books_service.create_many_from_files(
|
||||||
|
data, library, allow_duplicates=allow_duplicates
|
||||||
|
)
|
||||||
except ValueError:
|
except ValueError:
|
||||||
raise HTTPException(
|
raise HTTPException(
|
||||||
status_code=HTTP_400_BAD_REQUEST, detail="Must upload at least one file"
|
status_code=HTTP_400_BAD_REQUEST, detail="Must upload at least one file"
|
||||||
)
|
)
|
||||||
|
|
||||||
books = await books_service.list(
|
books = (
|
||||||
CollectionFilter("id", [result.id for result in results])
|
await books_service.list(
|
||||||
|
CollectionFilter("id", [book.id for book in result.books])
|
||||||
|
)
|
||||||
|
if result.books
|
||||||
|
else []
|
||||||
)
|
)
|
||||||
return books_service.to_schema(books, schema_type=s.BookRead)
|
|
||||||
|
return s.BooksUploadResult(
|
||||||
|
created=[
|
||||||
|
books_service.to_schema(book, schema_type=s.BookRead) for book in books
|
||||||
|
],
|
||||||
|
skipped=[
|
||||||
|
s.DuplicateFileRead.model_validate(duplicate)
|
||||||
|
for duplicate in result.duplicates
|
||||||
|
],
|
||||||
|
possible_duplicates=[
|
||||||
|
s.PossibleDuplicateRead.model_validate(possible)
|
||||||
|
for possible in result.possible_duplicates
|
||||||
|
],
|
||||||
|
)
|
||||||
|
|
||||||
|
@get(path="duplicate-books")
|
||||||
|
async def list_duplicate_books(
|
||||||
|
self, books_service: BookService, library: m.Library
|
||||||
|
) -> list[s.DuplicateBookGroupRead]:
|
||||||
|
"""
|
||||||
|
Report books already in the library that look like copies of one another.
|
||||||
|
|
||||||
|
The import-time check only ever sees what is arriving, so this is what covers
|
||||||
|
a collection someone already has. Matching is on metadata and therefore a
|
||||||
|
guess: a group is a question for the reader, not a verdict.
|
||||||
|
|
||||||
|
Query Parameters:
|
||||||
|
library_id: The library to review.
|
||||||
|
|
||||||
|
Injected Dependencies:
|
||||||
|
books_service: The book service for database operations.
|
||||||
|
library: The library to review.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
One entry per group of two or more books. Empty when there is nothing to
|
||||||
|
review, or when duplicate detection is switched off.
|
||||||
|
"""
|
||||||
|
groups = await books_service.find_duplicate_book_groups(library)
|
||||||
|
|
||||||
|
return [
|
||||||
|
s.DuplicateBookGroupRead(
|
||||||
|
books=[s.DuplicateBookRead.model_validate(book) for book in group]
|
||||||
|
)
|
||||||
|
for group in groups
|
||||||
|
]
|
||||||
|
|
||||||
|
@post(path="merge")
|
||||||
|
async def merge_books(
|
||||||
|
self, books_service: BookService, library: m.Library, data: s.BookMerge
|
||||||
|
) -> s.BookRead:
|
||||||
|
"""
|
||||||
|
Fold several books into one and delete the records folded in.
|
||||||
|
|
||||||
|
The survivor keeps its id, so links and bookmarks still resolve. Files, reading
|
||||||
|
progress, shelves, tags and unheld identifiers move onto it; metadata is only
|
||||||
|
changed by what `metadata` names, because choosing between two titles is the
|
||||||
|
reader's judgement rather than this endpoint's.
|
||||||
|
|
||||||
|
Nothing is removed from disk — a wrong merge should cost metadata that can be
|
||||||
|
retyped, not a book.
|
||||||
|
|
||||||
|
Query Parameters:
|
||||||
|
library_id: The library the books belong to.
|
||||||
|
|
||||||
|
Request Body:
|
||||||
|
data: The survivor, the books to fold in, and the resolved metadata.
|
||||||
|
|
||||||
|
Injected Dependencies:
|
||||||
|
books_service: The book service for database operations.
|
||||||
|
library: The library the books belong to.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
The surviving book.
|
||||||
|
|
||||||
|
Raises:
|
||||||
|
HTTPException: 400 if fewer than two distinct books were named, one is
|
||||||
|
unknown, or they do not all belong to one library.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
book = await books_service.merge_books(
|
||||||
|
data.survivor_id,
|
||||||
|
data.merged_ids,
|
||||||
|
library,
|
||||||
|
metadata=data.metadata.model_dump(exclude_unset=True)
|
||||||
|
if data.metadata
|
||||||
|
else None,
|
||||||
|
)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=HTTP_400_BAD_REQUEST, detail=str(exc))
|
||||||
|
|
||||||
|
return books_service.to_schema(book, schema_type=s.BookRead)
|
||||||
|
|
||||||
|
@post(path="duplicate-books/dismissals", status_code=HTTP_204_NO_CONTENT)
|
||||||
|
async def dismiss_duplicate_books(
|
||||||
|
self, books_service: BookService, data: s.DuplicateDismissal
|
||||||
|
) -> None:
|
||||||
|
"""
|
||||||
|
Record that two books are not the same book.
|
||||||
|
|
||||||
|
Without this the review screen proposes the same wrong pair forever, which is
|
||||||
|
how a reader learns to stop looking at it.
|
||||||
|
|
||||||
|
Request Body:
|
||||||
|
data: The two book IDs. Order does not matter.
|
||||||
|
|
||||||
|
Injected Dependencies:
|
||||||
|
books_service: The book service for database operations.
|
||||||
|
|
||||||
|
Raises:
|
||||||
|
HTTPException: 400 if the two IDs are the same or either book is unknown.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
await books_service.dismiss_duplicates(data.book_a_id, data.book_b_id)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=HTTP_400_BAD_REQUEST, detail=str(exc))
|
||||||
|
|
||||||
|
@delete(path="duplicate-books/dismissals")
|
||||||
|
async def restore_duplicate_books(
|
||||||
|
self, books_service: BookService, book_a_id: int, book_b_id: int
|
||||||
|
) -> None:
|
||||||
|
"""
|
||||||
|
Undo a dismissal, so the pair is proposed again.
|
||||||
|
|
||||||
|
Query Parameters:
|
||||||
|
book_a_id: One of the two books.
|
||||||
|
book_b_id: The other. Order does not matter.
|
||||||
|
|
||||||
|
Injected Dependencies:
|
||||||
|
books_service: The book service for database operations.
|
||||||
|
"""
|
||||||
|
await books_service.restore_duplicates(book_a_id, book_b_id)
|
||||||
|
|
||||||
|
# A question, not a change: 200 rather than the 201 a POST would default to.
|
||||||
|
@post(path="duplicate-files", status_code=HTTP_200_OK)
|
||||||
|
async def check_duplicate_files(
|
||||||
|
self,
|
||||||
|
books_service: BookService,
|
||||||
|
library: m.Library,
|
||||||
|
data: list[s.FileFingerprint],
|
||||||
|
) -> list[s.DuplicateFileRead]:
|
||||||
|
"""
|
||||||
|
Report which of the given files the library already holds.
|
||||||
|
|
||||||
|
Lets a client ask before it uploads anything, which is the difference between
|
||||||
|
re-sending a folder of books and re-sending twelve kilobytes of hashes.
|
||||||
|
|
||||||
|
Query Parameters:
|
||||||
|
library_id: The library to check against.
|
||||||
|
|
||||||
|
Request Body:
|
||||||
|
data: Hash and size for each file, optionally with the name to echo back.
|
||||||
|
|
||||||
|
Injected Dependencies:
|
||||||
|
books_service: The book service for database operations.
|
||||||
|
library: The library to check against.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
One entry per submitted file that is already stored. Files that are not
|
||||||
|
are absent.
|
||||||
|
"""
|
||||||
|
matches = await books_service.find_duplicate_files(
|
||||||
|
((item.hash, item.size) for item in data), library
|
||||||
|
)
|
||||||
|
|
||||||
|
return [
|
||||||
|
s.DuplicateFileRead(
|
||||||
|
filename=item.filename or match.filename,
|
||||||
|
hash=match.hash,
|
||||||
|
size=match.size,
|
||||||
|
library_id=match.library_id,
|
||||||
|
book_id=match.book_id,
|
||||||
|
book_title=match.book_title,
|
||||||
|
)
|
||||||
|
for item in data
|
||||||
|
if (match := matches.get((item.hash, item.size))) is not None
|
||||||
|
]
|
||||||
|
|
||||||
@get(path="/{book_id:int}")
|
@get(path="/{book_id:int}")
|
||||||
async def get_book_by_id(
|
async def get_book_by_id(
|
||||||
@@ -303,13 +527,20 @@ class BookController(Controller):
|
|||||||
],
|
],
|
||||||
library: m.Library,
|
library: m.Library,
|
||||||
books_service: BookService,
|
books_service: BookService,
|
||||||
|
allow_duplicates: bool = False,
|
||||||
) -> s.BookRead:
|
) -> s.BookRead:
|
||||||
"""
|
"""
|
||||||
Add files to an existing book.
|
Add files to an existing book.
|
||||||
|
|
||||||
|
A file the book already carries is ignored, so re-sending one is harmless.
|
||||||
|
|
||||||
Path Parameters:
|
Path Parameters:
|
||||||
book_id: The ID of the book to modify
|
book_id: The ID of the book to modify
|
||||||
|
|
||||||
|
Query Parameters:
|
||||||
|
allow_duplicates: If True, store the files even if the library already
|
||||||
|
holds them.
|
||||||
|
|
||||||
Request Body:
|
Request Body:
|
||||||
files: The files to add to the book
|
files: The files to add to the book
|
||||||
|
|
||||||
@@ -320,9 +551,17 @@ class BookController(Controller):
|
|||||||
Returns:
|
Returns:
|
||||||
The modified book
|
The modified book
|
||||||
|
|
||||||
|
Raises:
|
||||||
|
HTTPException: 409 if a file is already stored under a different book.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
await books_service.add_files(book_id, data, library)
|
try:
|
||||||
|
await books_service.add_files(
|
||||||
|
book_id, data, library, allow_duplicates=allow_duplicates
|
||||||
|
)
|
||||||
|
except DuplicateFilesError as exc:
|
||||||
|
raise _duplicate_conflict(exc)
|
||||||
|
|
||||||
book = await books_service.get(book_id)
|
book = await books_service.get(book_id)
|
||||||
return books_service.to_schema(book, schema_type=s.BookRead)
|
return books_service.to_schema(book, schema_type=s.BookRead)
|
||||||
|
|
||||||
|
|||||||
@@ -3,6 +3,7 @@ from .book import Book, Identifier, FileMetadata
|
|||||||
from .book_list import BookList, BookListLink
|
from .book_list import BookList, BookListLink
|
||||||
from .book_progress import BookProgress
|
from .book_progress import BookProgress
|
||||||
from .book_series import BookSeries
|
from .book_series import BookSeries
|
||||||
|
from .duplicate_dismissal import DuplicateDismissal
|
||||||
from .kosync_device import KosyncDevice
|
from .kosync_device import KosyncDevice
|
||||||
from .kosync_progress import KosyncProgress
|
from .kosync_progress import KosyncProgress
|
||||||
from .library import Library
|
from .library import Library
|
||||||
|
|||||||
@@ -5,6 +5,7 @@ from sqlalchemy import ColumnElement, ForeignKey, UniqueConstraint
|
|||||||
from sqlalchemy.orm import Mapped
|
from sqlalchemy.orm import Mapped
|
||||||
from sqlalchemy.orm import mapped_column
|
from sqlalchemy.orm import mapped_column
|
||||||
from sqlalchemy.orm import relationship
|
from sqlalchemy.orm import relationship
|
||||||
|
from sqlalchemy.orm import validates
|
||||||
|
|
||||||
from advanced_alchemy.base import BigIntAuditBase, BigIntBase
|
from advanced_alchemy.base import BigIntAuditBase, BigIntBase
|
||||||
from advanced_alchemy.mixins import UniqueMixin
|
from advanced_alchemy.mixins import UniqueMixin
|
||||||
@@ -16,18 +17,55 @@ if TYPE_CHECKING:
|
|||||||
class Author(BigIntAuditBase, UniqueMixin):
|
class Author(BigIntAuditBase, UniqueMixin):
|
||||||
__tablename__ = "authors"
|
__tablename__ = "authors"
|
||||||
|
|
||||||
|
# Always the canonical form — see `_canonicalize`. Extractors hand over whatever
|
||||||
|
# the file happened to say: "Newman, Sam;" from a `DC:creator` list, or
|
||||||
|
# "Sam Newman.epub" from a filename. Storing those verbatim is how one person ends
|
||||||
|
# up as several rows in the sidebar.
|
||||||
name: Mapped[str] = mapped_column(unique=True, index=True)
|
name: Mapped[str] = mapped_column(unique=True, index=True)
|
||||||
|
|
||||||
|
# Kept current by `_canonicalize` too — never assign it directly. Not unique: two
|
||||||
|
# spellings that survive canonicalization, "Steve Mcconnell" and "Steve McConnell",
|
||||||
|
# are still one person to a reader, which is what this column exists to express.
|
||||||
|
normalized_name: Mapped[str] = mapped_column(default="", index=True)
|
||||||
|
|
||||||
description: Mapped[Optional[str]]
|
description: Mapped[Optional[str]]
|
||||||
|
|
||||||
|
@validates("name")
|
||||||
|
def _canonicalize(self, _key: str, name: str) -> str:
|
||||||
|
"""
|
||||||
|
Store the tidied name, and derive the matching key from it.
|
||||||
|
|
||||||
|
A validator so no write can get around it, and `unique_hash` / `unique_filter`
|
||||||
|
below tidy the same way so `as_unique_async` looks the row up under the name it
|
||||||
|
would actually be stored as. All three have to agree: if the lookup used the
|
||||||
|
raw name and the insert used the tidy one, every variant spelling would miss
|
||||||
|
the existing row and then collide with it on the unique index.
|
||||||
|
"""
|
||||||
|
# Imported here rather than at module scope: `chitai.services.matching` cannot
|
||||||
|
# be reached without initialising the `chitai.services` package, which imports
|
||||||
|
# the services, which import this module.
|
||||||
|
from chitai.services.matching import format_author_name, normalize_author
|
||||||
|
|
||||||
|
name = format_author_name(name)
|
||||||
|
self.normalized_name = normalize_author(name)
|
||||||
|
return name
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def _tidy(cls, name: str) -> str:
|
||||||
|
"""The name as `_canonicalize` would store it."""
|
||||||
|
from chitai.services.matching import format_author_name
|
||||||
|
|
||||||
|
return format_author_name(name)
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
def unique_hash(cls, name: str) -> Hashable:
|
def unique_hash(cls, name: str) -> Hashable:
|
||||||
"""Generate a unique hash for deduplication."""
|
"""Generate a unique hash for deduplication."""
|
||||||
return name
|
return cls._tidy(name)
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
def unique_filter(cls, name: str) -> ColumnElement[bool]:
|
def unique_filter(cls, name: str) -> ColumnElement[bool]:
|
||||||
"""SQL filter for finding existing records."""
|
"""SQL filter for finding existing records."""
|
||||||
return cls.name == name
|
return cls.name == cls._tidy(name)
|
||||||
|
|
||||||
def __repr__(self) -> str:
|
def __repr__(self) -> str:
|
||||||
return f"Author({self.name!r})"
|
return f"Author({self.name!r})"
|
||||||
|
|||||||
@@ -2,13 +2,10 @@ from datetime import date
|
|||||||
from typing import TYPE_CHECKING, Any, Optional
|
from typing import TYPE_CHECKING, Any, Optional
|
||||||
|
|
||||||
from sqlalchemy import Index, ForeignKey, UniqueConstraint
|
from sqlalchemy import Index, ForeignKey, UniqueConstraint
|
||||||
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
from sqlalchemy.orm import Mapped, mapped_column, relationship, validates
|
||||||
from sqlalchemy.orm import mapped_column
|
|
||||||
from sqlalchemy.orm import relationship
|
|
||||||
from sqlalchemy.ext.orderinglist import ordering_list
|
from sqlalchemy.ext.orderinglist import ordering_list
|
||||||
from sqlalchemy.ext.associationproxy import association_proxy
|
from sqlalchemy.ext.associationproxy import association_proxy
|
||||||
from sqlalchemy.ext.associationproxy import AssociationProxy
|
from sqlalchemy.ext.associationproxy import AssociationProxy
|
||||||
from sqlalchemy.orm.collections import attribute_keyed_dict
|
|
||||||
|
|
||||||
from advanced_alchemy.base import BigIntAuditBase, BigIntBase
|
from advanced_alchemy.base import BigIntAuditBase, BigIntBase
|
||||||
|
|
||||||
@@ -44,6 +41,13 @@ class Book(BigIntAuditBase):
|
|||||||
library: Mapped["Library"] = relationship(back_populates="books")
|
library: Mapped["Library"] = relationship(back_populates="books")
|
||||||
|
|
||||||
title: Mapped[str]
|
title: Mapped[str]
|
||||||
|
|
||||||
|
# Kept current by `_normalize_title` below — never assign it directly.
|
||||||
|
#
|
||||||
|
# Deliberately not unique: two spellings collapsing onto one value is the whole
|
||||||
|
# point of the column, and a second edition is allowed to exist.
|
||||||
|
normalized_title: Mapped[str] = mapped_column(default="", index=True)
|
||||||
|
|
||||||
subtitle: Mapped[Optional[str]]
|
subtitle: Mapped[Optional[str]]
|
||||||
description: Mapped[Optional[str]]
|
description: Mapped[Optional[str]]
|
||||||
published_date: Mapped[Optional[date]]
|
published_date: Mapped[Optional[date]]
|
||||||
@@ -111,6 +115,24 @@ class Book(BigIntAuditBase):
|
|||||||
def progress(self) -> Optional["BookProgress"]:
|
def progress(self) -> Optional["BookProgress"]:
|
||||||
return self.progress_records[0] if self.progress_records else None
|
return self.progress_records[0] if self.progress_records else None
|
||||||
|
|
||||||
|
@validates("title")
|
||||||
|
def _normalize_title(self, _key: str, title: str) -> str:
|
||||||
|
"""
|
||||||
|
Derive `normalized_title` from whatever writes the title.
|
||||||
|
|
||||||
|
A validator rather than a service call because `BookService` sets titles from
|
||||||
|
at least three places — `to_model_on_create`, `to_model_on_update` and the
|
||||||
|
`setattr` loop in `_populate_with_unique_relationships` — and a fourth would
|
||||||
|
otherwise leave the key silently stale.
|
||||||
|
"""
|
||||||
|
# Imported here rather than at module scope: `chitai.services.matching` cannot
|
||||||
|
# be reached without initialising the `chitai.services` package, which imports
|
||||||
|
# the services, which import this module.
|
||||||
|
from chitai.services.matching import normalize_title
|
||||||
|
|
||||||
|
self.normalized_title = normalize_title(title)
|
||||||
|
return title
|
||||||
|
|
||||||
def __repr__(self) -> str:
|
def __repr__(self) -> str:
|
||||||
return f"Book({self.title=!r})"
|
return f"Book({self.title=!r})"
|
||||||
|
|
||||||
@@ -132,6 +154,24 @@ class Identifier(BigIntBase):
|
|||||||
book_id: Mapped[int] = mapped_column(ForeignKey("books.id", ondelete="cascade"))
|
book_id: Mapped[int] = mapped_column(ForeignKey("books.id", ondelete="cascade"))
|
||||||
value: Mapped[str]
|
value: Mapped[str]
|
||||||
|
|
||||||
|
# Kept current by `_normalize` below — never assign it directly. Null for an
|
||||||
|
# identifier that cannot carry a match: a per-build UUID, or an ISBN that fails
|
||||||
|
# its own checksum.
|
||||||
|
normalized_value: Mapped[Optional[str]] = mapped_column(index=True)
|
||||||
|
|
||||||
|
@validates("name", "value")
|
||||||
|
def _normalize(self, key: str, value: str) -> str:
|
||||||
|
"""Recompute `normalized_value` whenever either half of the pair changes."""
|
||||||
|
from chitai.services.matching import normalize_identifier
|
||||||
|
|
||||||
|
name = value if key == "name" else self.name
|
||||||
|
raw = value if key == "value" else self.value
|
||||||
|
|
||||||
|
self.normalized_value = (
|
||||||
|
normalize_identifier(name, raw) if name and raw else None
|
||||||
|
)
|
||||||
|
return value
|
||||||
|
|
||||||
def __repr__(self):
|
def __repr__(self):
|
||||||
return f"Identifier({self.name!r} : {self.value!r})"
|
return f"Identifier({self.name!r} : {self.value!r})"
|
||||||
|
|
||||||
@@ -139,6 +179,15 @@ class Identifier(BigIntBase):
|
|||||||
class FileMetadata(BigIntBase):
|
class FileMetadata(BigIntBase):
|
||||||
__tablename__ = "file_metadata"
|
__tablename__ = "file_metadata"
|
||||||
|
|
||||||
|
__table_args__ = (
|
||||||
|
# Deliberately not unique. The hash is KOReader's partial MD5, which samples
|
||||||
|
# 12 KiB of the file, so two genuinely different files can collide — and an
|
||||||
|
# existing database may already hold duplicates, which a unique index would
|
||||||
|
# refuse to build over. Duplicate detection pairs it with `size` and treats a
|
||||||
|
# match as advisory, so this only has to make the lookup cheap.
|
||||||
|
Index("ix_file_metadata_hash", "hash"),
|
||||||
|
)
|
||||||
|
|
||||||
book_id: Mapped[int] = mapped_column(ForeignKey("books.id", ondelete="cascade"))
|
book_id: Mapped[int] = mapped_column(ForeignKey("books.id", ondelete="cascade"))
|
||||||
book: Mapped[Book] = relationship(back_populates="files")
|
book: Mapped[Book] = relationship(back_populates="files")
|
||||||
hash: Mapped[str]
|
hash: Mapped[str]
|
||||||
|
|||||||
@@ -0,0 +1,37 @@
|
|||||||
|
from sqlalchemy import ForeignKey, UniqueConstraint
|
||||||
|
from sqlalchemy.orm import Mapped, mapped_column
|
||||||
|
|
||||||
|
from advanced_alchemy.base import BigIntBase
|
||||||
|
|
||||||
|
|
||||||
|
class DuplicateDismissal(BigIntBase):
|
||||||
|
"""
|
||||||
|
Two books a reader has said are not the same book.
|
||||||
|
|
||||||
|
Title and author matching is probabilistic, so it will keep proposing a second
|
||||||
|
edition, a translation and a sequel that shares its predecessor's name. A review
|
||||||
|
screen with no way to disagree with it nags forever, which is how people learn to
|
||||||
|
ignore a screen.
|
||||||
|
|
||||||
|
The pair is stored ordered — `book_a_id` is always the lower id — so "A and B" and
|
||||||
|
"B and A" are one row and the unique constraint can do its job. Use `pair()` rather
|
||||||
|
than assigning the columns directly.
|
||||||
|
"""
|
||||||
|
|
||||||
|
__tablename__ = "duplicate_dismissals"
|
||||||
|
__table_args__ = (UniqueConstraint("book_a_id", "book_b_id"),)
|
||||||
|
|
||||||
|
book_a_id: Mapped[int] = mapped_column(
|
||||||
|
ForeignKey("books.id", ondelete="cascade"), index=True
|
||||||
|
)
|
||||||
|
book_b_id: Mapped[int] = mapped_column(
|
||||||
|
ForeignKey("books.id", ondelete="cascade"), index=True
|
||||||
|
)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def pair(first: int, second: int) -> tuple[int, int]:
|
||||||
|
"""The two book ids in the order this table stores them."""
|
||||||
|
return (first, second) if first <= second else (second, first)
|
||||||
|
|
||||||
|
def __repr__(self) -> str:
|
||||||
|
return f"DuplicateDismissal({self.book_a_id!r}, {self.book_b_id!r})"
|
||||||
@@ -4,7 +4,15 @@ from .book import (
|
|||||||
BookProgressCreate,
|
BookProgressCreate,
|
||||||
BookProgressRead,
|
BookProgressRead,
|
||||||
BooksCreateFromFiles,
|
BooksCreateFromFiles,
|
||||||
|
BooksUploadResult,
|
||||||
|
BookMerge,
|
||||||
BookMetadataUpdate,
|
BookMetadataUpdate,
|
||||||
|
DuplicateBookGroupRead,
|
||||||
|
DuplicateBookRead,
|
||||||
|
DuplicateDismissal,
|
||||||
|
DuplicateFileRead,
|
||||||
|
FileFingerprint,
|
||||||
|
PossibleDuplicateRead,
|
||||||
FileMetadataRead,
|
FileMetadataRead,
|
||||||
BookSeriesRead,
|
BookSeriesRead,
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -129,6 +129,83 @@ class BooksCreateFromFiles(BaseModel):
|
|||||||
model_config = ConfigDict(arbitrary_types_allowed=True)
|
model_config = ConfigDict(arbitrary_types_allowed=True)
|
||||||
|
|
||||||
|
|
||||||
|
class FileFingerprint(BaseModel):
|
||||||
|
"""What a client can say about a file it has not uploaded yet."""
|
||||||
|
|
||||||
|
hash: str
|
||||||
|
size: int
|
||||||
|
filename: str = ""
|
||||||
|
|
||||||
|
|
||||||
|
class DuplicateFileRead(BaseModel):
|
||||||
|
"""A file that was not stored because the library already holds its bytes."""
|
||||||
|
|
||||||
|
model_config = ConfigDict(from_attributes=True)
|
||||||
|
|
||||||
|
filename: str
|
||||||
|
hash: str
|
||||||
|
size: int
|
||||||
|
library_id: int
|
||||||
|
|
||||||
|
# Null when the match was another file in the same upload, which has no row yet.
|
||||||
|
book_id: int | None = None
|
||||||
|
book_title: str | None = None
|
||||||
|
|
||||||
|
|
||||||
|
class DuplicateBookRead(BaseModel):
|
||||||
|
"""
|
||||||
|
A stored book that may be the same book as another one.
|
||||||
|
|
||||||
|
Unlike `DuplicateFileRead` this is a guess: the evidence is metadata two editions
|
||||||
|
of one work legitimately share. Nothing was refused on the strength of it.
|
||||||
|
"""
|
||||||
|
|
||||||
|
model_config = ConfigDict(from_attributes=True)
|
||||||
|
|
||||||
|
book_id: int
|
||||||
|
title: str
|
||||||
|
authors: list[str]
|
||||||
|
library_id: int
|
||||||
|
cover_image: Path | None = None
|
||||||
|
|
||||||
|
# Why it matched: "identifier" and/or "title-author".
|
||||||
|
matched_on: list[str]
|
||||||
|
|
||||||
|
|
||||||
|
class PossibleDuplicateRead(BaseModel):
|
||||||
|
"""A book that was imported, together with what it might be a second copy of."""
|
||||||
|
|
||||||
|
model_config = ConfigDict(from_attributes=True)
|
||||||
|
|
||||||
|
book_id: int
|
||||||
|
title: str
|
||||||
|
candidates: list[DuplicateBookRead]
|
||||||
|
|
||||||
|
|
||||||
|
class DuplicateBookGroupRead(BaseModel):
|
||||||
|
"""Books the library holds that all look like copies of one book."""
|
||||||
|
|
||||||
|
books: list[DuplicateBookRead]
|
||||||
|
|
||||||
|
|
||||||
|
class DuplicateDismissal(BaseModel):
|
||||||
|
"""Two books a reader is saying are not the same book."""
|
||||||
|
|
||||||
|
book_a_id: int
|
||||||
|
book_b_id: int
|
||||||
|
|
||||||
|
|
||||||
|
class BooksUploadResult(BaseModel):
|
||||||
|
"""The outcome of a multi-file upload: what was created, and what was skipped."""
|
||||||
|
|
||||||
|
created: list["BookRead"]
|
||||||
|
skipped: list[DuplicateFileRead]
|
||||||
|
|
||||||
|
# Created, not skipped — these are books that went in and look like something the
|
||||||
|
# library already had. The reader decides what to do about it.
|
||||||
|
possible_duplicates: list[PossibleDuplicateRead] = Field(default_factory=list)
|
||||||
|
|
||||||
|
|
||||||
class BookMetadataUpdate(BaseModel):
|
class BookMetadataUpdate(BaseModel):
|
||||||
title: str | None = None
|
title: str | None = None
|
||||||
subtitle: str | None = None
|
subtitle: str | None = None
|
||||||
@@ -170,6 +247,20 @@ class BookMetadataUpdate(BaseModel):
|
|||||||
return v
|
return v
|
||||||
|
|
||||||
|
|
||||||
|
class BookMerge(BaseModel):
|
||||||
|
"""
|
||||||
|
Fold several books into one.
|
||||||
|
|
||||||
|
`metadata` is the reader's resolution of the fields the records disagreed on.
|
||||||
|
Anything it does not name keeps the survivor's value — merging metadata is a
|
||||||
|
judgement, so nothing is guessed on the caller's behalf.
|
||||||
|
"""
|
||||||
|
|
||||||
|
survivor_id: int
|
||||||
|
merged_ids: list[int]
|
||||||
|
metadata: Optional["BookMetadataUpdate"] = None
|
||||||
|
|
||||||
|
|
||||||
class BookProgressCreate(BaseModel):
|
class BookProgressCreate(BaseModel):
|
||||||
percentage: float
|
percentage: float
|
||||||
epub_cfi: str | None = None
|
epub_cfi: str | None = None
|
||||||
|
|||||||
+1298
-46
File diff suppressed because it is too large
Load Diff
@@ -1,6 +1,7 @@
|
|||||||
import asyncio
|
import asyncio
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from collections import defaultdict
|
from collections import defaultdict
|
||||||
|
from chitai.config import settings
|
||||||
from chitai.database.models.library import Library
|
from chitai.database.models.library import Library
|
||||||
from chitai.services import BookService, LibraryService
|
from chitai.services import BookService, LibraryService
|
||||||
from chitai.services.metadata_extractor import Extractor
|
from chitai.services.metadata_extractor import Extractor
|
||||||
@@ -111,14 +112,33 @@ class ConsumeDirectoryWatcher:
|
|||||||
"""Process a batch of files."""
|
"""Process a batch of files."""
|
||||||
try:
|
try:
|
||||||
|
|
||||||
books = await self.book_service.create_many_from_existing_files(
|
result = await self.book_service.create_many_from_existing_files(
|
||||||
list(file_paths),
|
list(file_paths),
|
||||||
self.watch_path / Path(library_slug),
|
self.watch_path / Path(library_slug),
|
||||||
library=await self._get_library(library_slug),
|
library=await self._get_library(library_slug),
|
||||||
)
|
)
|
||||||
|
|
||||||
print(f"Created {len(books)} books!")
|
print(f"Created {len(result.books)} books!")
|
||||||
|
|
||||||
|
if result.duplicates:
|
||||||
|
print(
|
||||||
|
f"Moved {len(result.duplicates)} already-stored file(s) "
|
||||||
|
f"to {settings.duplicate_path}"
|
||||||
|
)
|
||||||
|
|
||||||
|
# Imported all the same — a metadata match is a guess, and there is nobody
|
||||||
|
# here to ask. The library's duplicates screen is where these get decided.
|
||||||
|
for possible in result.possible_duplicates:
|
||||||
|
names = ", ".join(
|
||||||
|
f"{candidate.title} (#{candidate.book_id})"
|
||||||
|
for candidate in possible.candidates
|
||||||
|
)
|
||||||
|
print(
|
||||||
|
f"Imported {possible.title!r} (#{possible.book_id}), which may "
|
||||||
|
f"already be in the library as: {names}"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
print(f"Error processing batch: {e}")
|
print(f"Error processing batch: {e}")
|
||||||
raise e
|
raise e
|
||||||
|
|||||||
@@ -127,6 +127,31 @@ def create_book_filter_dependencies(
|
|||||||
# Get base filters first
|
# Get base filters first
|
||||||
filters = create_filter_dependencies(config, dep_defaults)
|
filters = create_filter_dependencies(config, dep_defaults)
|
||||||
|
|
||||||
|
# OVERRIDE: id filter typed by the configured id type, not always `str`
|
||||||
|
#
|
||||||
|
# advanced_alchemy's `provide_id_filter` annotates `ids` as `list[str]` and
|
||||||
|
# ignores `config["id_filter"]` entirely, so `?ids=12` reaches the database as
|
||||||
|
# the string "12" and Postgres refuses `bigint = character varying`. Nothing
|
||||||
|
# called `?ids=` until the duplicates screen needed to fetch a handful of books
|
||||||
|
# by id, which is why it went unnoticed.
|
||||||
|
if id_type := config.get("id_filter"):
|
||||||
|
id_field = config.get("id_field", "id")
|
||||||
|
|
||||||
|
def provide_typed_id_filter(
|
||||||
|
ids=Parameter(query="ids", default=None, required=False),
|
||||||
|
) -> CollectionFilter:
|
||||||
|
return CollectionFilter(field_name=id_field, values=ids)
|
||||||
|
|
||||||
|
# Attached as a type object rather than written as an annotation: this module
|
||||||
|
# has `from __future__ import annotations`, so a written one is stored as the
|
||||||
|
# string "Optional[list[id_type]]" and resolved against module globals, where
|
||||||
|
# a local named `id_type` does not exist.
|
||||||
|
provide_typed_id_filter.__annotations__["ids"] = Optional[list[id_type]]
|
||||||
|
|
||||||
|
filters[dep_defaults.ID_FILTER_DEPENDENCY_KEY] = Provide(
|
||||||
|
provide_typed_id_filter, sync_to_thread=False
|
||||||
|
)
|
||||||
|
|
||||||
# OVERRIDE: Custom search filter with trigram search
|
# OVERRIDE: Custom search filter with trigram search
|
||||||
if config.get("search"):
|
if config.get("search"):
|
||||||
search_fields = config.get("search")
|
search_fields = config.get("search")
|
||||||
|
|||||||
@@ -0,0 +1,207 @@
|
|||||||
|
# src/chitai/services/matching.py
|
||||||
|
|
||||||
|
"""
|
||||||
|
Normalisation for book-level duplicate detection.
|
||||||
|
|
||||||
|
Two copies of one book rarely agree on how it is written down. One says
|
||||||
|
`The Metamorphosis`, the other `Metamorphosis`; one credits `Kafka, Franz`, the other
|
||||||
|
`Franz Kafka`; one carries the ISBN-10 and the other the ISBN-13 of the same edition.
|
||||||
|
These functions reduce each of those to a single key, so the comparison is an equality
|
||||||
|
test the database can index rather than a similarity score nobody can explain.
|
||||||
|
|
||||||
|
Everything here is pure: the keys are computed once and stored on the row (see
|
||||||
|
`Book.normalized_title`, `Author.normalized_name`, `Identifier.normalized_value`), so
|
||||||
|
no Postgres extension is needed at query time.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import re
|
||||||
|
import unicodedata
|
||||||
|
|
||||||
|
from chitai.services.utils import is_valid_isbn, isbn10_to_isbn13
|
||||||
|
|
||||||
|
|
||||||
|
# Asides a title carries that say nothing about which book it is:
|
||||||
|
# "Frankenstein (Illustrated)", "Dune [Deluxe]".
|
||||||
|
_BRACKETED = re.compile(r"[(\[{][^)\]}]*[)\]}]")
|
||||||
|
|
||||||
|
# Edition and format qualifiers, matched only as a *trailing* run of words. Anchoring
|
||||||
|
# to the end is what keeps "The Illustrated Man" a book and "Moby Dick Illustrated" a
|
||||||
|
# format note — a qualifier trails the title, it is never the thing the title is about.
|
||||||
|
_EDITION_NOISE = re.compile(
|
||||||
|
r"\s+(?:"
|
||||||
|
# "2nd edition", but also the compact forms publishers actually print on a
|
||||||
|
# cover: "2E", "3 Ed", "5e". The number alone is never enough — "Catch 22" is
|
||||||
|
# a title and must survive.
|
||||||
|
r"\d+(?:st|nd|rd|th)?\s*(?:edition|edn|ed|e)"
|
||||||
|
r"|(?:first|second|third|fourth|fifth|sixth|new|revised|expanded|updated|"
|
||||||
|
r"annotated|illustrated|unabridged|abridged|complete|definitive|deluxe|"
|
||||||
|
r"anniversary|collectors|international|kindle|paperback|hardcover|hardback|"
|
||||||
|
r"ebook|audiobook)"
|
||||||
|
r"(?:\s+(?:and|&)\s+\w+)*"
|
||||||
|
r"(?:\s+ed(?:ition|n)?)?"
|
||||||
|
r")$"
|
||||||
|
)
|
||||||
|
|
||||||
|
_LEADING_ARTICLE = re.compile(r"^(?:the|a|an)\s+")
|
||||||
|
|
||||||
|
# Anything that is not a letter, a digit or a space, once accents are gone.
|
||||||
|
_PUNCTUATION = re.compile(r"[^0-9a-z ]+")
|
||||||
|
|
||||||
|
_WHITESPACE = re.compile(r"\s+")
|
||||||
|
|
||||||
|
# Junk an extractor leaves on the end of a name: the `;` from a `DC:creator` list, the
|
||||||
|
# `.epub` from a filename the author's name was read out of.
|
||||||
|
_TRAILING_SEPARATORS = " ;,&/"
|
||||||
|
_FILE_EXTENSION = re.compile(
|
||||||
|
r"\.(?:epub|pdf|mobi|azw3?|djvu|fb2|txt|cbz|cbr)$", re.IGNORECASE
|
||||||
|
)
|
||||||
|
|
||||||
|
# `J. R. R.` survives punctuation stripping as three one-letter words; `J.R.R.` as one.
|
||||||
|
# Joining any run of them makes both `jrr`.
|
||||||
|
_INITIAL_RUN = re.compile(r"\b(?:[a-z] )+[a-z]\b")
|
||||||
|
|
||||||
|
# ISBNs are the same number under several names; everything else keeps its own.
|
||||||
|
_ISBN_NAMES = {"isbn", "isbn-10", "isbn10", "isbn-13", "isbn13"}
|
||||||
|
|
||||||
|
# Generated fresh for every build of a file, so two copies of one book never share one.
|
||||||
|
# Matching on them would only re-find files the hash check already catches.
|
||||||
|
_PER_BUILD_NAMES = {"uuid", "urn:uuid"}
|
||||||
|
|
||||||
|
# Below this an identifier is not specific enough to be evidence: a Calibre `id` of
|
||||||
|
# "42" would otherwise pair two unrelated books.
|
||||||
|
_MIN_IDENTIFIER_LENGTH = 4
|
||||||
|
|
||||||
|
|
||||||
|
def _fold(text: str) -> str:
|
||||||
|
"""Casefolded, accent-free, punctuation-free, single-spaced."""
|
||||||
|
decomposed = unicodedata.normalize("NFKD", text.casefold())
|
||||||
|
unaccented = "".join(c for c in decomposed if not unicodedata.combining(c))
|
||||||
|
|
||||||
|
return _WHITESPACE.sub(" ", _PUNCTUATION.sub(" ", unaccented)).strip()
|
||||||
|
|
||||||
|
|
||||||
|
def normalize_title(title: str | None) -> str:
|
||||||
|
"""
|
||||||
|
Reduce a title to the key two copies of one book should share.
|
||||||
|
|
||||||
|
`Book.subtitle` is already split off by `Extractor.format_book_title`, so only what
|
||||||
|
is left in the title column is considered here.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
title: The title as it was stored.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
The comparison key, or an empty string if nothing survives normalisation —
|
||||||
|
which is the signal not to match on the title at all.
|
||||||
|
"""
|
||||||
|
if not title:
|
||||||
|
return ""
|
||||||
|
|
||||||
|
folded = _fold(_BRACKETED.sub(" ", title).replace("&", " and "))
|
||||||
|
|
||||||
|
# Repeated because qualifiers stack: "Dune Deluxe Edition Illustrated".
|
||||||
|
while (trimmed := _EDITION_NOISE.sub("", folded)) != folded:
|
||||||
|
folded = trimmed
|
||||||
|
|
||||||
|
# An article says nothing, but a title that is only an article is not improved by
|
||||||
|
# having none, and neither is one that noise removal emptied out.
|
||||||
|
return _LEADING_ARTICLE.sub("", folded, count=1) or folded
|
||||||
|
|
||||||
|
|
||||||
|
def format_author_name(name: str | None) -> str:
|
||||||
|
"""
|
||||||
|
Tidy an author's name into the one form the library writes them in.
|
||||||
|
|
||||||
|
Distinct from `normalize_author`, which throws away case, accents and spacing to
|
||||||
|
build a comparison key. This one is what a reader sees, so it keeps everything
|
||||||
|
that belongs to the name and only removes what an extractor added: a trailing
|
||||||
|
separator left over from a creator list, a file extension carried in from a
|
||||||
|
filename, and the `Surname, Given` ordering that EPUBs file names under.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
name: The name as the file or filename gave it.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
The name to store, or an empty string if there is nothing left of it.
|
||||||
|
"""
|
||||||
|
if not name:
|
||||||
|
return ""
|
||||||
|
|
||||||
|
tidied = _FILE_EXTENSION.sub("", name.strip().strip(_TRAILING_SEPARATORS).strip())
|
||||||
|
|
||||||
|
if tidied.count(",") == 1:
|
||||||
|
surname, given = (part.strip() for part in tidied.split(","))
|
||||||
|
|
||||||
|
# Only when the part before the comma is a single word. "Dave Thomas, Andy
|
||||||
|
# Hunt" is two people in one string, and flipping it would invent a third
|
||||||
|
# person who does not exist. Leaving an unrecognised form alone is the safe
|
||||||
|
# failure; rewriting it wrongly is not.
|
||||||
|
if surname and given and " " not in surname:
|
||||||
|
tidied = f"{given} {surname}"
|
||||||
|
|
||||||
|
return _WHITESPACE.sub(" ", tidied).strip()
|
||||||
|
|
||||||
|
|
||||||
|
def normalize_author(name: str | None) -> str:
|
||||||
|
"""
|
||||||
|
Reduce an author's name to the key their other books should share.
|
||||||
|
|
||||||
|
Deliberately not reduced to surname plus initial: that collides unrelated people,
|
||||||
|
and a wrong match here is a book pointed at a stranger's shelf.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
name: The name as it was stored, in either `Franz Kafka` or `Kafka, Franz` form.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
The comparison key, or an empty string if nothing survives normalisation.
|
||||||
|
"""
|
||||||
|
if not name:
|
||||||
|
return ""
|
||||||
|
|
||||||
|
# `Kafka, Franz` is one name written backwards. More than one comma is a list, or a
|
||||||
|
# suffix, and guessing at either does more harm than leaving it alone.
|
||||||
|
if name.count(",") == 1:
|
||||||
|
surname, forename = name.split(",")
|
||||||
|
name = f"{forename.strip()} {surname.strip()}"
|
||||||
|
|
||||||
|
folded = _fold(name)
|
||||||
|
|
||||||
|
return _INITIAL_RUN.sub(lambda run: run.group().replace(" ", ""), folded)
|
||||||
|
|
||||||
|
|
||||||
|
def normalize_identifier(name: str, value: str) -> str | None:
|
||||||
|
"""
|
||||||
|
Reduce one identifier to a `scheme:value` key, if it can carry a match at all.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
name: What kind of identifier it is, as stored.
|
||||||
|
value: The identifier itself.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
The key, or None when the identifier is no use for matching: a per-build UUID,
|
||||||
|
something too short to be evidence, or an ISBN that fails its own checksum.
|
||||||
|
"""
|
||||||
|
name = (name or "").strip().casefold()
|
||||||
|
value = (value or "").strip()
|
||||||
|
|
||||||
|
if not name or not value or name in _PER_BUILD_NAMES:
|
||||||
|
return None
|
||||||
|
|
||||||
|
if name in _ISBN_NAMES:
|
||||||
|
digits = re.sub(r"[^0-9Xx]", "", value).upper()
|
||||||
|
|
||||||
|
if not is_valid_isbn(digits):
|
||||||
|
return None
|
||||||
|
|
||||||
|
# One scheme for both forms: a publisher prints whichever it likes, and the
|
||||||
|
# ISBN-10 and ISBN-13 of an edition are the same number written twice.
|
||||||
|
isbn = digits if len(digits) == 13 else isbn10_to_isbn13(digits)
|
||||||
|
return f"isbn:{isbn}" if isbn else None
|
||||||
|
|
||||||
|
folded = _fold(value) or value.casefold()
|
||||||
|
if len(folded) < _MIN_IDENTIFIER_LENGTH:
|
||||||
|
return None
|
||||||
|
|
||||||
|
return f"{name}:{folded}"
|
||||||
@@ -3,7 +3,6 @@
|
|||||||
# TODO: Code is a mess. Clean it up and add docstrings
|
# TODO: Code is a mess. Clean it up and add docstrings
|
||||||
|
|
||||||
# Standard library
|
# Standard library
|
||||||
from abc import ABC, abstractmethod
|
|
||||||
import datetime
|
import datetime
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from io import BytesIO
|
from io import BytesIO
|
||||||
@@ -31,6 +30,147 @@ from chitai.services.utils import (
|
|||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
# Identifier schemes an EPUB can declare, mapped onto the names the rest of the app
|
||||||
|
# uses. A scheme arrives either as an `opf:scheme` attribute or as a prefix on the
|
||||||
|
# value itself (`urn:isbn:…`, `calibre:…`), and the two say the same thing.
|
||||||
|
_IDENTIFIER_SCHEMES = {
|
||||||
|
"isbn": "isbn",
|
||||||
|
"isbn10": "isbn-10",
|
||||||
|
"isbn-10": "isbn-10",
|
||||||
|
"isbn13": "isbn-13",
|
||||||
|
"isbn-13": "isbn-13",
|
||||||
|
"uuid": "uuid",
|
||||||
|
"calibre": "calibre",
|
||||||
|
"doi": "doi",
|
||||||
|
"asin": "asin",
|
||||||
|
"amazon": "asin",
|
||||||
|
"mobi-asin": "asin",
|
||||||
|
"google": "google",
|
||||||
|
"goodreads": "goodreads",
|
||||||
|
}
|
||||||
|
|
||||||
|
# `scheme:rest`, with an optional `urn:` in front of it.
|
||||||
|
_SCHEME_PREFIX = re.compile(r"^(?:urn:)?([A-Za-z][A-Za-z0-9.-]*):(.+)$")
|
||||||
|
|
||||||
|
_UUID = re.compile(r"^[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12}$", re.IGNORECASE)
|
||||||
|
|
||||||
|
|
||||||
|
def parse_identifier(value: str, scheme: str | None = None) -> tuple[str, str] | None:
|
||||||
|
"""
|
||||||
|
Work out what one raw identifier is, and what it is worth storing as.
|
||||||
|
|
||||||
|
EPUBs write the same ISBN as `9780486282114`, `978-0-486-28211-4` and
|
||||||
|
`urn:isbn:978-0-486-28211-4`, and carry plenty of identifiers that are not ISBNs
|
||||||
|
at all. Validating the string verbatim keeps only the first form and throws the
|
||||||
|
rest away, so normalise first and name whatever survives.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
value: The identifier as the file wrote it.
|
||||||
|
scheme: What the file said it is, if it said anything — an `opf:scheme`
|
||||||
|
attribute. A prefix on the value takes precedence over this.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
The `(name, value)` to store, or `None` when there is nothing usable: an
|
||||||
|
empty value, or one declared to be an ISBN that fails its own checksum.
|
||||||
|
"""
|
||||||
|
value = (value or "").strip()
|
||||||
|
if not value:
|
||||||
|
return None
|
||||||
|
|
||||||
|
name = _IDENTIFIER_SCHEMES.get((scheme or "").strip().casefold())
|
||||||
|
|
||||||
|
# An unrecognised prefix is part of the value rather than a scheme —
|
||||||
|
# "http://example.com/book" is not an identifier called "http".
|
||||||
|
if (match := _SCHEME_PREFIX.match(value)) and (
|
||||||
|
prefixed := _IDENTIFIER_SCHEMES.get(match.group(1).casefold())
|
||||||
|
):
|
||||||
|
name = prefixed
|
||||||
|
value = match.group(2).strip()
|
||||||
|
|
||||||
|
if name is None or name.startswith("isbn"):
|
||||||
|
digits = re.sub(r"[^0-9Xx]", "", value).upper()
|
||||||
|
if is_valid_isbn(digits):
|
||||||
|
return f"isbn-{len(digits)}", digits
|
||||||
|
|
||||||
|
# Something that announced itself as an ISBN and is not one carries no
|
||||||
|
# information: storing it would link the reader to a page that does not exist.
|
||||||
|
if name is not None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
return name or ("uuid" if _UUID.match(value) else "id"), value
|
||||||
|
|
||||||
|
|
||||||
|
# Numbered editions, in the forms covers and catalogue records actually use:
|
||||||
|
# "3rd Edition", "2E", "4e", "8_e", "/6e", "(2nd edition)", "Third International Edition".
|
||||||
|
_ORDINAL_WORDS = {
|
||||||
|
"first": 1, "second": 2, "third": 3, "fourth": 4, "fifth": 5, "sixth": 6,
|
||||||
|
"seventh": 7, "eighth": 8, "ninth": 9, "tenth": 10, "eleventh": 11, "twelfth": 12,
|
||||||
|
}
|
||||||
|
|
||||||
|
# Words that sit between the number and "Edition" and belong to the same statement.
|
||||||
|
_EDITION_QUALIFIER = (
|
||||||
|
r"(?:international|global|revised|updated|expanded|anniversary|deluxe|student|"
|
||||||
|
r"instructors?|annotated|illustrated|reprint)"
|
||||||
|
)
|
||||||
|
|
||||||
|
_EDITION = re.compile(
|
||||||
|
rf"""
|
||||||
|
[\s,;:/\-–—(\[]+ # the separator the statement hangs off
|
||||||
|
(?:
|
||||||
|
(?P<num>\d{{1,2}})\s*(?:st|nd|rd|th)?[\s_]*
|
||||||
|
(?:{_EDITION_QUALIFIER}\s+)*(?:edition\b|edn\b|ed\b\.?|e\b)
|
||||||
|
| (?P<word>{"|".join(_ORDINAL_WORDS)})\s+
|
||||||
|
(?:{_EDITION_QUALIFIER}\s+)*(?:edition\b|edn\b|ed\b\.?)
|
||||||
|
)
|
||||||
|
[\s)\]]* # and its closing bracket, if it had one
|
||||||
|
""",
|
||||||
|
re.IGNORECASE | re.VERBOSE,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def split_edition(title: str | None) -> tuple[str | None, int | None]:
|
||||||
|
"""
|
||||||
|
Separate a numbered edition statement from the title it is written into.
|
||||||
|
|
||||||
|
"Fluent Python, 2nd Edition" is one book with a field for the edition, not a
|
||||||
|
title. Left in place it also splits the library: the second edition never looks
|
||||||
|
like the first, and neither matches the copy whose file simply did not mention it.
|
||||||
|
|
||||||
|
The number is what makes this safe. Nothing is stripped without one, so
|
||||||
|
"Catch 22" and "Blade Runner 2049" keep their numbers and "Global Edition" —
|
||||||
|
which is a variant, not a numbered edition, and has nowhere to go in an
|
||||||
|
integer column — is left in the title where it can still be read.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
title: The title as the file or filename gave it.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
The title without the edition statement, and the edition number. The title
|
||||||
|
unchanged and None when there is no numbered edition in it, or when removing
|
||||||
|
it would leave nothing behind.
|
||||||
|
"""
|
||||||
|
if not title:
|
||||||
|
return title, None
|
||||||
|
|
||||||
|
if (match := _EDITION.search(title)) is None:
|
||||||
|
return title, None
|
||||||
|
|
||||||
|
edition = (
|
||||||
|
int(match["num"]) if match["num"] else _ORDINAL_WORDS[match["word"].casefold()]
|
||||||
|
)
|
||||||
|
|
||||||
|
stripped = _EDITION.sub(" ", title)
|
||||||
|
stripped = re.sub(r"\s{2,}", " ", stripped)
|
||||||
|
stripped = re.sub(r"\s+([,;:.!?])", r"\1", stripped) # "Works : What" → "Works: What"
|
||||||
|
stripped = stripped.strip(" ,;:-–—/")
|
||||||
|
|
||||||
|
# A title that is only an edition statement is not improved by having none.
|
||||||
|
if not stripped:
|
||||||
|
return title, None
|
||||||
|
|
||||||
|
return stripped, edition
|
||||||
|
|
||||||
|
|
||||||
class FileExtractor(Protocol):
|
class FileExtractor(Protocol):
|
||||||
@classmethod
|
@classmethod
|
||||||
async def extract_metadata(
|
async def extract_metadata(
|
||||||
@@ -54,15 +194,36 @@ class Extractor:
|
|||||||
# EPUB tends to give better metadata results over pdf
|
# EPUB tends to give better metadata results over pdf
|
||||||
sorted_files = sorted(files, key=lambda f: Extractor._get_file_priority(f))
|
sorted_files = sorted(files, key=lambda f: Extractor._get_file_priority(f))
|
||||||
|
|
||||||
|
# Identifiers accumulate across formats instead of replacing each other. Every
|
||||||
|
# other field is a single value where the later, better-trusted format simply
|
||||||
|
# wins, but identifiers are a *collection*: a book holding an EPUB and a PDF
|
||||||
|
# genuinely carries what both of them declare, and merging the dict wholesale
|
||||||
|
# threw away everything the earlier format found. An EPUB that declares an
|
||||||
|
# ASIN, a Google volume id and a Calibre id kept none of them once a PDF
|
||||||
|
# contributed a single ISBN.
|
||||||
|
identifiers: dict[str, str] = {}
|
||||||
|
|
||||||
for file in sorted_files:
|
for file in sorted_files:
|
||||||
match get_file_extension(file):
|
match get_file_extension(file):
|
||||||
case "epub":
|
case "epub":
|
||||||
metadata = metadata | await EpubExtractor.extract_metadata(file)
|
extracted = await EpubExtractor.extract_metadata(file)
|
||||||
case "pdf":
|
case "pdf":
|
||||||
metadata = metadata | await PdfExtractor.extract_metadata(file)
|
extracted = await PdfExtractor.extract_metadata(file)
|
||||||
case _:
|
case _:
|
||||||
break
|
break
|
||||||
|
|
||||||
|
# First writer wins per name, and the files are already ordered by how
|
||||||
|
# far their metadata can be trusted. A `dc:identifier` the publisher
|
||||||
|
# declared outranks an ISBN scraped out of a PDF's copyright page, which
|
||||||
|
# routinely prints the ISBNs of other formats and older editions too.
|
||||||
|
for name, value in (extracted.pop("identifiers", None) or {}).items():
|
||||||
|
identifiers.setdefault(name, value)
|
||||||
|
|
||||||
|
metadata = metadata | extracted
|
||||||
|
|
||||||
|
if identifiers:
|
||||||
|
metadata["identifiers"] = identifiers
|
||||||
|
|
||||||
# Get metadata from file names
|
# Get metadata from file names
|
||||||
for file in files:
|
for file in files:
|
||||||
metadata = FilenameExtractor.extract_metadata(file) | metadata
|
metadata = FilenameExtractor.extract_metadata(file) | metadata
|
||||||
@@ -75,7 +236,14 @@ class Extractor:
|
|||||||
|
|
||||||
# format the title
|
# format the title
|
||||||
if metadata.get('title', None):
|
if metadata.get('title', None):
|
||||||
title, subtitle = Extractor.format_book_title(metadata["title"])
|
# Before the subtitle split, so the edition cannot be mistaken for one:
|
||||||
|
# "How Linux Works, 3rd Edition: What Every Superuser Should Know" has to
|
||||||
|
# lose the edition first for the colon count to mean anything.
|
||||||
|
title, edition = split_edition(metadata["title"])
|
||||||
|
if edition is not None:
|
||||||
|
metadata.setdefault("edition", edition)
|
||||||
|
|
||||||
|
title, subtitle = Extractor.format_book_title(title)
|
||||||
metadata["title"] = title
|
metadata["title"] = title
|
||||||
metadata["subtitle"] = subtitle
|
metadata["subtitle"] = subtitle
|
||||||
|
|
||||||
@@ -236,7 +404,7 @@ class PdfExtractor(FileExtractor):
|
|||||||
try:
|
try:
|
||||||
return datetime.datetime.strptime(date_portion, "%Y%m%d").date()
|
return datetime.datetime.strptime(date_portion, "%Y%m%d").date()
|
||||||
|
|
||||||
except Exception as e:
|
except Exception:
|
||||||
return None
|
return None
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
@@ -363,15 +531,23 @@ class EpubExtractor(FileExtractor):
|
|||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
def _extract_identifiers(cls, epub: epub.EpubBook) -> dict[str, str]:
|
def _extract_identifiers(cls, epub: epub.EpubBook) -> dict[str, str]:
|
||||||
|
"""
|
||||||
|
Every `DC:identifier` the file carries, keyed by what kind of thing it is.
|
||||||
|
|
||||||
|
Non-ISBN identifiers are kept: `Identifier` is a free-form name/value pair, so
|
||||||
|
a Calibre id or an ASIN costs nothing to store and is one more thing two copies
|
||||||
|
of a book can be recognised by.
|
||||||
|
"""
|
||||||
identifiers = {}
|
identifiers = {}
|
||||||
|
|
||||||
for id in epub.get_metadata("DC", "identifier"):
|
for value, attributes in epub.get_metadata("DC", "identifier"):
|
||||||
if is_valid_isbn(id[0]):
|
scheme = None
|
||||||
if len(id[0]) == 13:
|
if isinstance(attributes, dict):
|
||||||
identifiers.update({"isbn-13": id[0]})
|
scheme = attributes.get("opf:scheme") or attributes.get("scheme")
|
||||||
|
|
||||||
elif len(id[0]) == 10:
|
if (parsed := parse_identifier(value, scheme)) is not None:
|
||||||
identifiers.update({"isbn-10": id[0]})
|
name, parsed_value = parsed
|
||||||
|
identifiers[name] = parsed_value
|
||||||
|
|
||||||
return identifiers
|
return identifiers
|
||||||
|
|
||||||
@@ -380,7 +556,7 @@ class EpubExtractor(FileExtractor):
|
|||||||
try:
|
try:
|
||||||
return epub.get_metadata("DC", "description")[0][0]
|
return epub.get_metadata("DC", "description")[0][0]
|
||||||
|
|
||||||
except:
|
except Exception:
|
||||||
return None
|
return None
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
@@ -389,15 +565,15 @@ class EpubExtractor(FileExtractor):
|
|||||||
date_str = epub.get_metadata("DC", "date")[0][0].split("T")[0]
|
date_str = epub.get_metadata("DC", "date")[0][0].split("T")[0]
|
||||||
return datetime.date.fromisoformat(date_str)
|
return datetime.date.fromisoformat(date_str)
|
||||||
|
|
||||||
except:
|
except Exception:
|
||||||
return None
|
return None
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
def _extract_publisher(cls, epub: epub.EpubBook) -> str | None:
|
def _extract_publisher(cls, epub: epub.EpubBook) -> str | None:
|
||||||
try:
|
try:
|
||||||
epub.get_metadata("DC", "publisher")[0][0]
|
return epub.get_metadata("DC", "publisher")[0][0]
|
||||||
|
|
||||||
except:
|
except Exception:
|
||||||
return None
|
return None
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
@@ -421,7 +597,7 @@ class EpubExtractor(FileExtractor):
|
|||||||
cover_item = epub.get_item_with_id(cover_id)
|
cover_item = epub.get_item_with_id(cover_id)
|
||||||
if cover_item:
|
if cover_item:
|
||||||
return PIL.Image.open(BytesIO(cover_item.content))
|
return PIL.Image.open(BytesIO(cover_item.content))
|
||||||
except Exception as e:
|
except Exception:
|
||||||
pass # Fallback to next strategy
|
pass # Fallback to next strategy
|
||||||
|
|
||||||
# Strategy 2: Search image filenames for "cover" keyword
|
# Strategy 2: Search image filenames for "cover" keyword
|
||||||
@@ -489,7 +665,11 @@ class FilenameExtractor(FileExtractor):
|
|||||||
elif isinstance(input, Path):
|
elif isinstance(input, Path):
|
||||||
filename = get_filename(input, ext=False)
|
filename = get_filename(input, ext=False)
|
||||||
elif isinstance(input, UploadFile):
|
elif isinstance(input, UploadFile):
|
||||||
filename = Path(input.filename).name
|
# `.stem`, not `.name`: the extension is not part of the metadata, and
|
||||||
|
# this is the browser upload path, so keeping it is how a library fills
|
||||||
|
# up with authors called "Sam Newman.epub". The other two branches have
|
||||||
|
# always stripped it.
|
||||||
|
filename = Path(input.filename).stem
|
||||||
else:
|
else:
|
||||||
raise ValueError("Input type not supported")
|
raise ValueError("Input type not supported")
|
||||||
|
|
||||||
|
|||||||
@@ -3,13 +3,11 @@
|
|||||||
# Standard library
|
# Standard library
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import errno
|
||||||
import hashlib
|
import hashlib
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
import shutil
|
import shutil
|
||||||
from typing import TYPE_CHECKING, BinaryIO
|
from typing import BinaryIO
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
|
||||||
from hashlib import _Hash
|
|
||||||
|
|
||||||
# Third-party libraries
|
# Third-party libraries
|
||||||
import PIL
|
import PIL
|
||||||
@@ -32,6 +30,9 @@ KO_STEP = 1024
|
|||||||
KO_SAMPLE_SIZE = 1024
|
KO_SAMPLE_SIZE = 1024
|
||||||
KO_INDICES = range(-1, 11) # -1 to 10 inclusive
|
KO_INDICES = range(-1, 11) # -1 to 10 inclusive
|
||||||
|
|
||||||
|
# How much is read at a time while hashing.
|
||||||
|
HASH_CHUNK_SIZE = 262144 # 256 KiB
|
||||||
|
|
||||||
|
|
||||||
def _lshift32(val: int, shift: int) -> int:
|
def _lshift32(val: int, shift: int) -> int:
|
||||||
"""
|
"""
|
||||||
@@ -100,10 +101,9 @@ async def calculate_koreader_hash(file_path: Path) -> str:
|
|||||||
offsets = _get_koreader_offsets()
|
offsets = _get_koreader_offsets()
|
||||||
|
|
||||||
file_pos = 0
|
file_pos = 0
|
||||||
chunk_size = 262144 # 256 KiB
|
|
||||||
|
|
||||||
async with aiofiles.open(file_path, "rb") as f:
|
async with aiofiles.open(file_path, "rb") as f:
|
||||||
while chunk := await f.read(chunk_size):
|
while chunk := await f.read(HASH_CHUNK_SIZE):
|
||||||
_partial_md5_from_chunk(chunk, hasher, offsets, file_pos)
|
_partial_md5_from_chunk(chunk, hasher, offsets, file_pos)
|
||||||
file_pos += len(chunk)
|
file_pos += len(chunk)
|
||||||
|
|
||||||
@@ -132,6 +132,49 @@ class StreamingHasher:
|
|||||||
"""Return the final hash."""
|
"""Return the final hash."""
|
||||||
return self.hasher.hexdigest()
|
return self.hasher.hexdigest()
|
||||||
|
|
||||||
|
@property
|
||||||
|
def size(self) -> int:
|
||||||
|
"""Total number of bytes fed in so far."""
|
||||||
|
return self.position
|
||||||
|
|
||||||
|
|
||||||
|
async def fingerprint_upload(file: UploadFile) -> tuple[str, int]:
|
||||||
|
"""
|
||||||
|
Calculate the hash and byte size of an uploaded file without storing it.
|
||||||
|
|
||||||
|
Duplicate detection has to answer before anything is written to the library, so
|
||||||
|
the file is read here and rewound for whoever writes it afterwards.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
file: The uploaded file to read.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
The file's `(hash, size)` pair.
|
||||||
|
"""
|
||||||
|
hasher = StreamingHasher()
|
||||||
|
|
||||||
|
await file.seek(0)
|
||||||
|
while chunk := await file.read(HASH_CHUNK_SIZE):
|
||||||
|
hasher.update(chunk)
|
||||||
|
await file.seek(0)
|
||||||
|
|
||||||
|
return hasher.hexdigest(), hasher.size
|
||||||
|
|
||||||
|
|
||||||
|
async def fingerprint_file(file_path: Path) -> tuple[str, int]:
|
||||||
|
"""
|
||||||
|
Calculate the hash and byte size of a file already on disk.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
file_path: Path to the file to read.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
The file's `(hash, size)` pair.
|
||||||
|
"""
|
||||||
|
stats = await aios.stat(file_path)
|
||||||
|
return await calculate_koreader_hash(file_path), stats.st_size
|
||||||
|
|
||||||
|
|
||||||
##################################
|
##################################
|
||||||
# Filesystem related utilities #
|
# Filesystem related utilities #
|
||||||
##################################
|
##################################
|
||||||
@@ -172,7 +215,7 @@ async def create_directory(dir_path: Path | str) -> None:
|
|||||||
async def move_file(src_path: Path, dest_path: Path, create_dirs=True) -> None:
|
async def move_file(src_path: Path, dest_path: Path, create_dirs=True) -> None:
|
||||||
"""
|
"""
|
||||||
Move a file from source to destination asynchronously.
|
Move a file from source to destination asynchronously.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
source_path: Path to the source file
|
source_path: Path to the source file
|
||||||
destination_path: Path to the destination file
|
destination_path: Path to the destination file
|
||||||
@@ -184,7 +227,16 @@ async def move_file(src_path: Path, dest_path: Path, create_dirs=True) -> None:
|
|||||||
if dest_dir: # Only create if there's a directory path
|
if dest_dir: # Only create if there's a directory path
|
||||||
await aios.makedirs(dest_dir, exist_ok=True)
|
await aios.makedirs(dest_dir, exist_ok=True)
|
||||||
|
|
||||||
await aios.rename(src_path, dest_path)
|
try:
|
||||||
|
await aios.rename(src_path, dest_path)
|
||||||
|
except OSError as exc:
|
||||||
|
if exc.errno != errno.EXDEV:
|
||||||
|
raise
|
||||||
|
|
||||||
|
# Source and destination are on different filesystems, which rename cannot
|
||||||
|
# cross. Libraries, the consume directory and the duplicates directory are
|
||||||
|
# all configured separately, so they can easily be separate mounts.
|
||||||
|
shutil.move(str(src_path), str(dest_path))
|
||||||
|
|
||||||
async def move_dir_contents(source_dir: Path | str, target_dir: Path | str) -> None:
|
async def move_dir_contents(source_dir: Path | str, target_dir: Path | str) -> None:
|
||||||
"""
|
"""
|
||||||
@@ -432,7 +484,7 @@ def is_valid_isbn(isbn: str) -> bool:
|
|||||||
return is_valid_isbn13(isbn)
|
return is_valid_isbn13(isbn)
|
||||||
else:
|
else:
|
||||||
return False
|
return False
|
||||||
except:
|
except Exception:
|
||||||
return False
|
return False
|
||||||
|
|
||||||
|
|
||||||
@@ -457,6 +509,29 @@ def is_valid_isbn10(isbn: str) -> bool:
|
|||||||
return str(check_digit) == isbn[-1] or (check_digit == 10 and isbn[-1] in "Xx")
|
return str(check_digit) == isbn[-1] or (check_digit == 10 and isbn[-1] in "Xx")
|
||||||
|
|
||||||
|
|
||||||
|
def isbn10_to_isbn13(isbn: str) -> str | None:
|
||||||
|
"""
|
||||||
|
Convert an ISBN-10 to the ISBN-13 naming the same edition.
|
||||||
|
|
||||||
|
The two are the same number written twice: prefix `978`, drop the ISBN-10 check
|
||||||
|
digit, recompute the check digit under the ISBN-13 rule. Matching only works if
|
||||||
|
both forms collapse onto one, since a publisher may print either.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
isbn: A 10-character ISBN, digits and an optional trailing `X` only.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
The equivalent ISBN-13, or None if the input is not a valid ISBN-10.
|
||||||
|
"""
|
||||||
|
if not is_valid_isbn(isbn) or len(isbn) != 10:
|
||||||
|
return None
|
||||||
|
|
||||||
|
digits = f"978{isbn[:9]}"
|
||||||
|
total = sum(int(digit) * (1 if i % 2 == 0 else 3) for i, digit in enumerate(digits))
|
||||||
|
|
||||||
|
return f"{digits}{(10 - total % 10) % 10}"
|
||||||
|
|
||||||
|
|
||||||
def is_valid_isbn13(isbn: str) -> bool:
|
def is_valid_isbn13(isbn: str) -> bool:
|
||||||
"""
|
"""
|
||||||
Validate an ISBN-13 number using its check digit.
|
Validate an ISBN-13 number using its check digit.
|
||||||
|
|||||||
@@ -40,7 +40,13 @@ pytest_plugins = [
|
|||||||
|
|
||||||
@pytest.fixture(autouse=True)
|
@pytest.fixture(autouse=True)
|
||||||
def _patch_settings(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
|
def _patch_settings(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
monkeypatch.setattr(settings, "book_cover_path", f"{tmp_path}/covers")
|
# PIL will not create the directory it is asked to save into, so anything that
|
||||||
|
# imports a file carrying a cover needs it to exist first.
|
||||||
|
covers = tmp_path / "covers"
|
||||||
|
covers.mkdir()
|
||||||
|
|
||||||
|
monkeypatch.setattr(settings, "book_cover_path", str(covers))
|
||||||
|
monkeypatch.setattr(settings, "duplicate_path", str(tmp_path / "duplicates"))
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture(name="engine")
|
@pytest.fixture(name="engine")
|
||||||
|
|||||||
@@ -37,7 +37,8 @@ from pathlib import Path
|
|||||||
(
|
(
|
||||||
Path("tests/data_files/Calculus Made Easy - Silvanus Thompson.pdf"),
|
Path("tests/data_files/Calculus Made Easy - Silvanus Thompson.pdf"),
|
||||||
2,
|
2,
|
||||||
"The Project Gutenberg eBook #33283: Calculus Made Easy, 2nd Edition",
|
# The ", 2nd Edition" is split off into `edition`, not kept in the title.
|
||||||
|
"The Project Gutenberg eBook #33283: Calculus Made Easy",
|
||||||
["Silvanus Phillips Thompson"],
|
["Silvanus Phillips Thompson"],
|
||||||
),
|
),
|
||||||
],
|
],
|
||||||
@@ -208,6 +209,21 @@ async def test_get_book_file(
|
|||||||
assert downloaded_content == file_content
|
assert downloaded_content == file_content
|
||||||
|
|
||||||
|
|
||||||
|
async def test_list_books_by_id(populated_authenticated_client: AsyncClient) -> None:
|
||||||
|
"""
|
||||||
|
`?ids=` has to reach the database as integers.
|
||||||
|
|
||||||
|
advanced_alchemy's stock id filter annotates the parameter as `list[str]` whatever
|
||||||
|
the configured id type, so the ids arrived as strings and Postgres refused to
|
||||||
|
compare a bigint primary key against them. Nothing called it until a screen needed
|
||||||
|
to fetch a handful of books by id.
|
||||||
|
"""
|
||||||
|
response = await populated_authenticated_client.get("/books?ids=1&ids=2&pageSize=10")
|
||||||
|
|
||||||
|
assert response.status_code == 200
|
||||||
|
assert sorted(book["id"] for book in response.json()["items"]) == [1, 2]
|
||||||
|
|
||||||
|
|
||||||
async def test_get_book_by_id(populated_authenticated_client: AsyncClient) -> None:
|
async def test_get_book_by_id(populated_authenticated_client: AsyncClient) -> None:
|
||||||
"""Test retrieving a specific book by ID."""
|
"""Test retrieving a specific book by ID."""
|
||||||
|
|
||||||
@@ -366,7 +382,8 @@ async def test_create_multiple_books_from_directory(
|
|||||||
|
|
||||||
assert response.status_code == 201
|
assert response.status_code == 201
|
||||||
data = response.json()
|
data = response.json()
|
||||||
assert len(data.get("items") or data.get("data")) >= 1
|
assert len(data["created"]) == 2
|
||||||
|
assert data["skipped"] == []
|
||||||
|
|
||||||
|
|
||||||
async def test_create_books_from_parent_directory_keeps_embedded_title(
|
async def test_create_books_from_parent_directory_keeps_embedded_title(
|
||||||
@@ -396,7 +413,7 @@ async def test_create_books_from_parent_directory_keeps_embedded_title(
|
|||||||
|
|
||||||
assert response.status_code == 201
|
assert response.status_code == 201
|
||||||
|
|
||||||
books = response.json()["items"]
|
books = response.json()["created"]
|
||||||
assert len(books) == 1
|
assert len(books) == 1
|
||||||
assert books[0]["title"] == "Metamorphosis"
|
assert books[0]["title"] == "Metamorphosis"
|
||||||
|
|
||||||
@@ -419,11 +436,336 @@ async def test_create_books_groups_formats_within_one_folder(
|
|||||||
|
|
||||||
assert response.status_code == 201
|
assert response.status_code == 201
|
||||||
|
|
||||||
books = response.json()["items"]
|
books = response.json()["created"]
|
||||||
assert len(books) == 1
|
assert len(books) == 1
|
||||||
assert len(books[0]["files"]) == 2
|
assert len(books[0]["files"]) == 2
|
||||||
|
|
||||||
|
|
||||||
|
class TestDuplicateHandling:
|
||||||
|
"""A file the library already holds must not be stored a second time."""
|
||||||
|
|
||||||
|
epub_path = Path("tests/data_files/Metamorphosis - Franz Kafka.epub")
|
||||||
|
|
||||||
|
def upload(self, name: str | None = None) -> list[tuple[str, tuple]]:
|
||||||
|
return [
|
||||||
|
(
|
||||||
|
"files",
|
||||||
|
(
|
||||||
|
name or self.epub_path.name,
|
||||||
|
self.epub_path.read_bytes(),
|
||||||
|
"application/epub+zip",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
]
|
||||||
|
|
||||||
|
async def test_bulk_upload_reports_skipped_files(
|
||||||
|
self, authenticated_client: AsyncClient
|
||||||
|
) -> None:
|
||||||
|
"""Re-dropping a folder must import what is new and name what was not."""
|
||||||
|
first = await authenticated_client.post(
|
||||||
|
"/books/fromFiles?library_id=1", files=self.upload()
|
||||||
|
)
|
||||||
|
assert first.status_code == 201
|
||||||
|
created = first.json()["created"][0]
|
||||||
|
|
||||||
|
second = await authenticated_client.post(
|
||||||
|
"/books/fromFiles?library_id=1", files=self.upload()
|
||||||
|
)
|
||||||
|
|
||||||
|
assert second.status_code == 201
|
||||||
|
result = second.json()
|
||||||
|
assert result["created"] == []
|
||||||
|
assert len(result["skipped"]) == 1
|
||||||
|
|
||||||
|
skipped = result["skipped"][0]
|
||||||
|
assert skipped["filename"] == self.epub_path.name
|
||||||
|
assert skipped["book_id"] == created["id"]
|
||||||
|
assert skipped["book_title"] == created["title"]
|
||||||
|
|
||||||
|
async def test_bulk_upload_can_be_forced(
|
||||||
|
self, authenticated_client: AsyncClient
|
||||||
|
) -> None:
|
||||||
|
await authenticated_client.post(
|
||||||
|
"/books/fromFiles?library_id=1", files=self.upload()
|
||||||
|
)
|
||||||
|
|
||||||
|
response = await authenticated_client.post(
|
||||||
|
"/books/fromFiles?library_id=1&allow_duplicates=true", files=self.upload()
|
||||||
|
)
|
||||||
|
|
||||||
|
assert response.status_code == 201
|
||||||
|
assert len(response.json()["created"]) == 1
|
||||||
|
assert response.json()["skipped"] == []
|
||||||
|
|
||||||
|
async def test_single_book_create_conflicts(
|
||||||
|
self, authenticated_client: AsyncClient
|
||||||
|
) -> None:
|
||||||
|
"""Naming files deliberately earns a refusal rather than a silent drop."""
|
||||||
|
await authenticated_client.post(
|
||||||
|
"/books/fromFiles?library_id=1", files=self.upload()
|
||||||
|
)
|
||||||
|
|
||||||
|
response = await authenticated_client.post(
|
||||||
|
"/books?library_id=1", files=self.upload(), data={"library_id": 1}
|
||||||
|
)
|
||||||
|
|
||||||
|
assert response.status_code == 409
|
||||||
|
assert response.json()["extra"][0]["filename"] == self.epub_path.name
|
||||||
|
|
||||||
|
forced = await authenticated_client.post(
|
||||||
|
"/books?library_id=1&allow_duplicates=true",
|
||||||
|
files=self.upload(),
|
||||||
|
data={"library_id": 1},
|
||||||
|
)
|
||||||
|
assert forced.status_code == 201
|
||||||
|
|
||||||
|
async def test_adding_another_books_file_conflicts(
|
||||||
|
self, authenticated_client: AsyncClient
|
||||||
|
) -> None:
|
||||||
|
created = await authenticated_client.post(
|
||||||
|
"/books/fromFiles?library_id=1", files=self.upload()
|
||||||
|
)
|
||||||
|
book_id = created.json()["created"][0]["id"]
|
||||||
|
|
||||||
|
other = await authenticated_client.post(
|
||||||
|
"/books?library_id=1",
|
||||||
|
files=[
|
||||||
|
(
|
||||||
|
"files",
|
||||||
|
(
|
||||||
|
"war.epub",
|
||||||
|
Path("tests/data_files/The Art of War - Sun Tzu.epub").read_bytes(),
|
||||||
|
"application/epub+zip",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
],
|
||||||
|
data={"library_id": 1},
|
||||||
|
)
|
||||||
|
other_id = other.json()["id"]
|
||||||
|
|
||||||
|
response = await authenticated_client.post(
|
||||||
|
f"/books/{book_id}/files",
|
||||||
|
files=[
|
||||||
|
(
|
||||||
|
"files",
|
||||||
|
(
|
||||||
|
"war.epub",
|
||||||
|
Path("tests/data_files/The Art of War - Sun Tzu.epub").read_bytes(),
|
||||||
|
"application/epub+zip",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
],
|
||||||
|
)
|
||||||
|
|
||||||
|
assert response.status_code == 409
|
||||||
|
assert response.json()["extra"][0]["book_id"] == other_id
|
||||||
|
|
||||||
|
async def test_resending_a_books_own_file_changes_nothing(
|
||||||
|
self, authenticated_client: AsyncClient
|
||||||
|
) -> None:
|
||||||
|
created = await authenticated_client.post(
|
||||||
|
"/books/fromFiles?library_id=1", files=self.upload()
|
||||||
|
)
|
||||||
|
book_id = created.json()["created"][0]["id"]
|
||||||
|
|
||||||
|
response = await authenticated_client.post(
|
||||||
|
f"/books/{book_id}/files", files=self.upload()
|
||||||
|
)
|
||||||
|
|
||||||
|
assert response.status_code == 201
|
||||||
|
assert len(response.json()["files"]) == 1
|
||||||
|
|
||||||
|
async def test_duplicates_can_be_checked_before_uploading(
|
||||||
|
self, authenticated_client: AsyncClient
|
||||||
|
) -> None:
|
||||||
|
"""The pre-flight check answers from hashes alone, with no file sent."""
|
||||||
|
created = await authenticated_client.post(
|
||||||
|
"/books/fromFiles?library_id=1", files=self.upload()
|
||||||
|
)
|
||||||
|
book = created.json()["created"][0]
|
||||||
|
stored = book["files"][0]
|
||||||
|
|
||||||
|
response = await authenticated_client.post(
|
||||||
|
"/books/duplicate-files?library_id=1",
|
||||||
|
json=[
|
||||||
|
{
|
||||||
|
"hash": stored["hash"],
|
||||||
|
"size": stored["size"],
|
||||||
|
"filename": "local-copy.epub",
|
||||||
|
},
|
||||||
|
{"hash": stored["hash"], "size": stored["size"] + 1},
|
||||||
|
{"hash": "0" * 32, "size": 1234},
|
||||||
|
],
|
||||||
|
)
|
||||||
|
|
||||||
|
assert response.status_code == 200
|
||||||
|
|
||||||
|
matches = response.json()
|
||||||
|
assert len(matches) == 1
|
||||||
|
assert matches[0]["filename"] == "local-copy.epub"
|
||||||
|
assert matches[0]["book_id"] == book["id"]
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
class TestDuplicateBooks:
|
||||||
|
"""A second copy of a book is imported and reported, never refused."""
|
||||||
|
|
||||||
|
epub_path = Path("tests/data_files/Metamorphosis - Franz Kafka.epub")
|
||||||
|
|
||||||
|
def upload(self, name: str, pad: bool = False) -> list[tuple[str, tuple]]:
|
||||||
|
"""
|
||||||
|
The fixture, optionally padded so it is a different file and the same book.
|
||||||
|
|
||||||
|
Padding the archive changes its size and its sampled hash without disturbing
|
||||||
|
the metadata, which is exactly the case the file-level check cannot see.
|
||||||
|
"""
|
||||||
|
data = self.epub_path.read_bytes() + (b"\0" * 64 if pad else b"")
|
||||||
|
|
||||||
|
return [("files", (name, data, "application/epub+zip"))]
|
||||||
|
|
||||||
|
async def test_a_second_edition_is_created_and_reported(
|
||||||
|
self, authenticated_client: AsyncClient
|
||||||
|
) -> None:
|
||||||
|
first = await authenticated_client.post(
|
||||||
|
"/books/fromFiles?library_id=1", files=self.upload("first.epub")
|
||||||
|
)
|
||||||
|
original = first.json()["created"][0]
|
||||||
|
|
||||||
|
second = await authenticated_client.post(
|
||||||
|
"/books/fromFiles?library_id=1", files=self.upload("second.epub", pad=True)
|
||||||
|
)
|
||||||
|
|
||||||
|
assert second.status_code == 201
|
||||||
|
body = second.json()
|
||||||
|
|
||||||
|
# Created, not skipped: a metadata match is a guess, and refusing a legitimate
|
||||||
|
# second edition costs more than a note does.
|
||||||
|
assert len(body["created"]) == 1
|
||||||
|
assert body["skipped"] == []
|
||||||
|
|
||||||
|
assert len(body["possible_duplicates"]) == 1
|
||||||
|
possible = body["possible_duplicates"][0]
|
||||||
|
assert possible["book_id"] == body["created"][0]["id"]
|
||||||
|
|
||||||
|
candidate = possible["candidates"][0]
|
||||||
|
assert candidate["book_id"] == original["id"]
|
||||||
|
assert candidate["title"] == original["title"]
|
||||||
|
assert candidate["authors"] == ["Franz Kafka"]
|
||||||
|
assert "title-author" in candidate["matched_on"]
|
||||||
|
|
||||||
|
async def test_the_review_screen_groups_them(
|
||||||
|
self, authenticated_client: AsyncClient
|
||||||
|
) -> None:
|
||||||
|
first = await authenticated_client.post(
|
||||||
|
"/books/fromFiles?library_id=1", files=self.upload("first.epub")
|
||||||
|
)
|
||||||
|
second = await authenticated_client.post(
|
||||||
|
"/books/fromFiles?library_id=1", files=self.upload("second.epub", pad=True)
|
||||||
|
)
|
||||||
|
|
||||||
|
book_ids = sorted(
|
||||||
|
[first.json()["created"][0]["id"], second.json()["created"][0]["id"]]
|
||||||
|
)
|
||||||
|
|
||||||
|
response = await authenticated_client.get("/books/duplicate-books?library_id=1")
|
||||||
|
|
||||||
|
assert response.status_code == 200
|
||||||
|
groups = response.json()
|
||||||
|
assert len(groups) == 1
|
||||||
|
assert [book["book_id"] for book in groups[0]["books"]] == book_ids
|
||||||
|
|
||||||
|
async def test_a_dismissed_group_stays_dismissed(
|
||||||
|
self, authenticated_client: AsyncClient
|
||||||
|
) -> None:
|
||||||
|
first = await authenticated_client.post(
|
||||||
|
"/books/fromFiles?library_id=1", files=self.upload("first.epub")
|
||||||
|
)
|
||||||
|
second = await authenticated_client.post(
|
||||||
|
"/books/fromFiles?library_id=1", files=self.upload("second.epub", pad=True)
|
||||||
|
)
|
||||||
|
|
||||||
|
pair = {
|
||||||
|
"book_a_id": first.json()["created"][0]["id"],
|
||||||
|
"book_b_id": second.json()["created"][0]["id"],
|
||||||
|
}
|
||||||
|
|
||||||
|
dismissed = await authenticated_client.post(
|
||||||
|
"/books/duplicate-books/dismissals", json=pair
|
||||||
|
)
|
||||||
|
assert dismissed.status_code == 204
|
||||||
|
|
||||||
|
response = await authenticated_client.get("/books/duplicate-books?library_id=1")
|
||||||
|
assert response.json() == []
|
||||||
|
|
||||||
|
restored = await authenticated_client.delete(
|
||||||
|
"/books/duplicate-books/dismissals"
|
||||||
|
f"?book_a_id={pair['book_b_id']}&book_b_id={pair['book_a_id']}"
|
||||||
|
)
|
||||||
|
assert restored.status_code == 204
|
||||||
|
|
||||||
|
response = await authenticated_client.get("/books/duplicate-books?library_id=1")
|
||||||
|
assert len(response.json()) == 1
|
||||||
|
|
||||||
|
async def test_two_books_merge_into_one(
|
||||||
|
self, authenticated_client: AsyncClient
|
||||||
|
) -> None:
|
||||||
|
"""The survivor keeps its id and gains the other's file; the other is gone."""
|
||||||
|
first = await authenticated_client.post(
|
||||||
|
"/books/fromFiles?library_id=1", files=self.upload("first.epub")
|
||||||
|
)
|
||||||
|
second = await authenticated_client.post(
|
||||||
|
"/books/fromFiles?library_id=1", files=self.upload("second.epub", pad=True)
|
||||||
|
)
|
||||||
|
keep = first.json()["created"][0]
|
||||||
|
fold = second.json()["created"][0]
|
||||||
|
|
||||||
|
response = await authenticated_client.post(
|
||||||
|
"/books/merge?library_id=1",
|
||||||
|
json={
|
||||||
|
"survivor_id": keep["id"],
|
||||||
|
"merged_ids": [fold["id"]],
|
||||||
|
"metadata": {"title": "Metamorphosis", "edition": 2},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert response.status_code == 201
|
||||||
|
merged = response.json()
|
||||||
|
|
||||||
|
assert merged["id"] == keep["id"]
|
||||||
|
assert merged["edition"] == 2
|
||||||
|
assert len(merged["files"]) == 2
|
||||||
|
|
||||||
|
# The folded record is gone, and the group it formed with it.
|
||||||
|
assert (await authenticated_client.get(f"/books/{fold['id']}")).status_code == 404
|
||||||
|
groups = await authenticated_client.get("/books/duplicate-books?library_id=1")
|
||||||
|
assert groups.json() == []
|
||||||
|
|
||||||
|
async def test_merging_an_unknown_book_is_refused(
|
||||||
|
self, authenticated_client: AsyncClient
|
||||||
|
) -> None:
|
||||||
|
created = await authenticated_client.post(
|
||||||
|
"/books/fromFiles?library_id=1", files=self.upload("first.epub")
|
||||||
|
)
|
||||||
|
keep = created.json()["created"][0]["id"]
|
||||||
|
|
||||||
|
response = await authenticated_client.post(
|
||||||
|
"/books/merge?library_id=1",
|
||||||
|
json={"survivor_id": keep, "merged_ids": [9999]},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert response.status_code == 400
|
||||||
|
|
||||||
|
async def test_dismissing_an_unknown_book_is_refused(
|
||||||
|
self, authenticated_client: AsyncClient
|
||||||
|
) -> None:
|
||||||
|
response = await authenticated_client.post(
|
||||||
|
"/books/duplicate-books/dismissals",
|
||||||
|
json={"book_a_id": 1, "book_b_id": 9999},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert response.status_code == 400
|
||||||
|
|
||||||
|
|
||||||
# NOTE: the multi-book ZIP download is covered at the service level, in
|
# NOTE: the multi-book ZIP download is covered at the service level, in
|
||||||
# tests/unit/test_services/test_book_service.py. Driving `/books/download` through
|
# tests/unit/test_services/test_book_service.py. Driving `/books/download` through
|
||||||
# AsyncTestClient hangs in fixture teardown: it is the only `Stream` endpoint in the
|
# AsyncTestClient hangs in fixture teardown: it is the only `Stream` endpoint in the
|
||||||
|
|||||||
@@ -0,0 +1,233 @@
|
|||||||
|
"""Tests for the normalization behind book-level duplicate detection."""
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from chitai.services.matching import (
|
||||||
|
format_author_name,
|
||||||
|
normalize_author,
|
||||||
|
normalize_identifier,
|
||||||
|
normalize_title,
|
||||||
|
)
|
||||||
|
from chitai.services.metadata_extractor import parse_identifier
|
||||||
|
from chitai.services.utils import isbn10_to_isbn13
|
||||||
|
|
||||||
|
|
||||||
|
class TestNormalizeTitle:
|
||||||
|
"""Two copies of one book rarely agree on how the title is written."""
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("title", "expected"),
|
||||||
|
[
|
||||||
|
("The Metamorphosis", "metamorphosis"),
|
||||||
|
("Metamorphosis", "metamorphosis"),
|
||||||
|
("METAMORPHOSIS", "metamorphosis"),
|
||||||
|
("A Tale of Two Cities", "tale of two cities"),
|
||||||
|
("An Enquiry", "enquiry"),
|
||||||
|
# Accents, punctuation and ampersands are spelling, not identity.
|
||||||
|
("Les Misérables", "les miserables"),
|
||||||
|
("Moby Dick; Or, The Whale", "moby dick or the whale"),
|
||||||
|
("Sense & Sensibility", "sense and sensibility"),
|
||||||
|
# Bracketed asides and trailing edition noise say nothing about the book.
|
||||||
|
("Frankenstein (Illustrated)", "frankenstein"),
|
||||||
|
("Frankenstein [Kindle Edition]", "frankenstein"),
|
||||||
|
("Frankenstein, 2nd Edition", "frankenstein"),
|
||||||
|
# The compact forms a cover actually carries.
|
||||||
|
("Building Microservices, 2E", "building microservices"),
|
||||||
|
("Building Microservices 2e", "building microservices"),
|
||||||
|
("Frankenstein 3 Ed", "frankenstein"),
|
||||||
|
("Dungeons & Dragons 5e", "dungeons and dragons"),
|
||||||
|
("Frankenstein Revised Edition", "frankenstein"),
|
||||||
|
("Dune Deluxe Edition Illustrated", "dune"),
|
||||||
|
("", ""),
|
||||||
|
(None, ""),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_titles_that_should_agree(self, title: str | None, expected: str) -> None:
|
||||||
|
assert normalize_title(title) == expected
|
||||||
|
|
||||||
|
def test_a_qualifier_that_is_the_title_survives(self) -> None:
|
||||||
|
"""A trailing qualifier is noise; the same word at the front is the book."""
|
||||||
|
assert normalize_title("The Illustrated Man") == "illustrated man"
|
||||||
|
|
||||||
|
def test_normalization_never_empties_a_title(self) -> None:
|
||||||
|
"""An article-only title is not improved by having no article left."""
|
||||||
|
assert normalize_title("The") == "the"
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"title", ["Catch 22", "Fahrenheit 451", "Blade Runner 2049", "1984", "Apollo 13"]
|
||||||
|
)
|
||||||
|
def test_a_number_is_not_an_edition(self, title: str) -> None:
|
||||||
|
"""Edition stripping keys on the `e`; a bare number is part of the title."""
|
||||||
|
assert normalize_title(title) == title.casefold()
|
||||||
|
|
||||||
|
|
||||||
|
class TestNormalizeAuthor:
|
||||||
|
"""One person, written down several ways."""
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("name", "expected"),
|
||||||
|
[
|
||||||
|
("Franz Kafka", "franz kafka"),
|
||||||
|
("Kafka, Franz", "franz kafka"),
|
||||||
|
("KAFKA, FRANZ", "franz kafka"),
|
||||||
|
("Émile Zola", "emile zola"),
|
||||||
|
("Doyle, Arthur Conan", "arthur conan doyle"),
|
||||||
|
# Runs of initials are joined, so spacing them out changes nothing.
|
||||||
|
("J.R.R. Tolkien", "jrr tolkien"),
|
||||||
|
("J. R. R. Tolkien", "jrr tolkien"),
|
||||||
|
("JRR Tolkien", "jrr tolkien"),
|
||||||
|
("", ""),
|
||||||
|
(None, ""),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_names_that_should_agree(self, name: str | None, expected: str) -> None:
|
||||||
|
assert normalize_author(name) == expected
|
||||||
|
|
||||||
|
def test_two_people_are_not_reduced_together(self) -> None:
|
||||||
|
"""Surname plus initial would collide unrelated writers; it is not used."""
|
||||||
|
assert normalize_author("Charles Dickens") != normalize_author("Colin Dexter")
|
||||||
|
|
||||||
|
def test_a_list_is_left_alone(self) -> None:
|
||||||
|
"""More than one comma is a list or a suffix, and guessing does more harm."""
|
||||||
|
assert normalize_author("Smith, John, Jr.") == "smith john jr"
|
||||||
|
|
||||||
|
|
||||||
|
class TestFormatAuthorName:
|
||||||
|
"""What gets stored and shown, as opposed to what gets compared."""
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("written", "expected"),
|
||||||
|
[
|
||||||
|
# A leftover separator from a `DC:creator` list.
|
||||||
|
("Newman, Sam;", "Sam Newman"),
|
||||||
|
("Sam Newman ", "Sam Newman"),
|
||||||
|
(" Dan Vanderkam ", "Dan Vanderkam"),
|
||||||
|
# `Surname, Given` is how EPUBs file a name, not how anyone reads it.
|
||||||
|
("Kleppmann, Martin", "Martin Kleppmann"),
|
||||||
|
("Huxley, Aldous", "Aldous Huxley"),
|
||||||
|
("Liu, Cixin", "Cixin Liu"),
|
||||||
|
# An extension carried in from the filename the name was read out of.
|
||||||
|
("Sam Newman.epub", "Sam Newman"),
|
||||||
|
("Franz Kafka.mobi", "Franz Kafka"),
|
||||||
|
("Brian W. Kernighan.epub", "Brian W. Kernighan"),
|
||||||
|
("", ""),
|
||||||
|
(None, ""),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_names_are_tidied(self, written: str | None, expected: str) -> None:
|
||||||
|
assert format_author_name(written) == expected
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"written",
|
||||||
|
[
|
||||||
|
# Two people in one string. Flipping it would invent a third.
|
||||||
|
"Dave Thomas, Andy Hunt",
|
||||||
|
"Mark Richards, Neal Ford",
|
||||||
|
# A compound surname is not recognised, and is left alone rather than
|
||||||
|
# rearranged wrongly.
|
||||||
|
"García Márquez, Gabriel",
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_an_unrecognised_form_is_left_alone(self, written: str) -> None:
|
||||||
|
assert format_author_name(written) == written
|
||||||
|
|
||||||
|
def test_case_and_accents_belong_to_the_author(self) -> None:
|
||||||
|
"""Tidying removes what an extractor added; it does not correct spelling."""
|
||||||
|
assert format_author_name("Michał Płachta.epub") == "Michał Płachta"
|
||||||
|
assert format_author_name("Steve McConnell") == "Steve McConnell"
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"written", ["Newman, Sam;", "Sam Newman.epub", "Kleppmann, Martin"]
|
||||||
|
)
|
||||||
|
def test_tidying_is_idempotent(self, written: str) -> None:
|
||||||
|
"""`unique_filter` tidies a name that may already be tidy; it must not drift."""
|
||||||
|
once = format_author_name(written)
|
||||||
|
assert format_author_name(once) == once
|
||||||
|
|
||||||
|
|
||||||
|
class TestNormalizeIdentifier:
|
||||||
|
"""Identifiers only help if the same edition produces the same key."""
|
||||||
|
|
||||||
|
def test_isbn_10_and_isbn_13_are_one_key(self) -> None:
|
||||||
|
assert normalize_identifier("isbn-10", "0486282112") == "isbn:9780486282114"
|
||||||
|
assert normalize_identifier("isbn-13", "9780486282114") == "isbn:9780486282114"
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"written", ["978-0-486-28211-4", "978 0 486 28211 4", "9780486282114"]
|
||||||
|
)
|
||||||
|
def test_formatting_is_not_part_of_an_isbn(self, written: str) -> None:
|
||||||
|
assert normalize_identifier("isbn", written) == "isbn:9780486282114"
|
||||||
|
|
||||||
|
def test_an_isbn_that_fails_its_checksum_is_no_evidence(self) -> None:
|
||||||
|
assert normalize_identifier("isbn-13", "9780486282115") is None
|
||||||
|
|
||||||
|
def test_uuids_are_refused(self) -> None:
|
||||||
|
"""Generated per build, so they only re-find what the hash check catches."""
|
||||||
|
assert normalize_identifier("uuid", "3f2b1c4e-1111-2222-3333-444455556666") is None
|
||||||
|
assert normalize_identifier("urn:uuid", "3f2b1c4e-1111-2222-3333-444455556666") is None
|
||||||
|
|
||||||
|
def test_other_schemes_keep_their_own_key(self) -> None:
|
||||||
|
assert normalize_identifier("asin", "B000FC0PDA") == "asin:b000fc0pda"
|
||||||
|
assert normalize_identifier("ASIN", "b000fc0pda") == "asin:b000fc0pda"
|
||||||
|
|
||||||
|
def test_something_too_short_is_not_evidence(self) -> None:
|
||||||
|
"""A Calibre id of "42" would otherwise pair two unrelated books."""
|
||||||
|
assert normalize_identifier("calibre", "42") is None
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(("name", "value"), [("", "1234567"), ("asin", "")])
|
||||||
|
def test_half_an_identifier_is_no_identifier(self, name: str, value: str) -> None:
|
||||||
|
assert normalize_identifier(name, value) is None
|
||||||
|
|
||||||
|
|
||||||
|
class TestIsbnConversion:
|
||||||
|
def test_isbn_10_converts_to_its_isbn_13(self) -> None:
|
||||||
|
assert isbn10_to_isbn13("0486282112") == "9780486282114"
|
||||||
|
|
||||||
|
def test_a_trailing_x_is_a_digit(self) -> None:
|
||||||
|
assert isbn10_to_isbn13("043942089X") == "9780439420891"
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("isbn", ["0486282113", "9780486282114", "nonsense"])
|
||||||
|
def test_anything_that_is_not_an_isbn_10_converts_to_nothing(self, isbn: str) -> None:
|
||||||
|
assert isbn10_to_isbn13(isbn) is None
|
||||||
|
|
||||||
|
|
||||||
|
class TestParseIdentifier:
|
||||||
|
"""What an EPUB writes, and what is worth storing for it."""
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"written",
|
||||||
|
[
|
||||||
|
"9780486282114",
|
||||||
|
"978-0-486-28211-4",
|
||||||
|
"urn:isbn:9780486282114",
|
||||||
|
"urn:isbn:978-0-486-28211-4",
|
||||||
|
"ISBN:978-0-486-28211-4",
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_isbns_survive_however_they_are_written(self, written: str) -> None:
|
||||||
|
assert parse_identifier(written) == ("isbn-13", "9780486282114")
|
||||||
|
|
||||||
|
def test_the_scheme_attribute_is_read_too(self) -> None:
|
||||||
|
assert parse_identifier("0-486-28211-2", "ISBN") == ("isbn-10", "0486282112")
|
||||||
|
|
||||||
|
def test_non_isbn_identifiers_are_kept(self) -> None:
|
||||||
|
assert parse_identifier("urn:uuid:3f2b1c4e-1111-2222-3333-444455556666") == (
|
||||||
|
"uuid",
|
||||||
|
"3f2b1c4e-1111-2222-3333-444455556666",
|
||||||
|
)
|
||||||
|
assert parse_identifier("calibre:1234") == ("calibre", "1234")
|
||||||
|
assert parse_identifier("B000FC0PDA", "mobi-asin") == ("asin", "B000FC0PDA")
|
||||||
|
|
||||||
|
def test_an_unrecognised_prefix_is_part_of_the_value(self) -> None:
|
||||||
|
""""http://example.com/book" is not an identifier called "http"."""
|
||||||
|
assert parse_identifier("http://www.gutenberg.org/5200") == (
|
||||||
|
"id",
|
||||||
|
"http://www.gutenberg.org/5200",
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_a_declared_isbn_that_is_not_one_is_dropped(self) -> None:
|
||||||
|
assert parse_identifier("urn:isbn:not-an-isbn") is None
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("written", ["", " ", None])
|
||||||
|
def test_nothing_yields_nothing(self, written: str | None) -> None:
|
||||||
|
assert parse_identifier(written) is None
|
||||||
@@ -1,7 +1,13 @@
|
|||||||
import pytest
|
import pytest
|
||||||
|
from ebooklib import epub
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from datetime import date
|
from datetime import date
|
||||||
from chitai.services.metadata_extractor import EpubExtractor
|
from chitai.services.metadata_extractor import (
|
||||||
|
EpubExtractor,
|
||||||
|
Extractor,
|
||||||
|
PdfExtractor,
|
||||||
|
split_edition,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio()
|
@pytest.mark.asyncio()
|
||||||
@@ -15,3 +21,172 @@ class TestEpubExtractor:
|
|||||||
assert metadata["authors"] == ["Herman Melville"]
|
assert metadata["authors"] == ["Herman Melville"]
|
||||||
assert metadata["published_date"] == date(year=2001, month=7, day=1)
|
assert metadata["published_date"] == date(year=2001, month=7, day=1)
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
EPUB = Path("tests/data_files/Metamorphosis - Franz Kafka.epub")
|
||||||
|
PDF = Path("tests/data_files/Calculus Made Easy - Silvanus Thompson.pdf")
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio()
|
||||||
|
class TestIdentifierMerging:
|
||||||
|
"""A book's formats each contribute identifiers; none of them replaces the rest."""
|
||||||
|
|
||||||
|
async def test_every_format_contributes(self, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
"""
|
||||||
|
Identifiers are a collection, not a single value.
|
||||||
|
|
||||||
|
Merging the whole dict meant the last format to report won outright: an EPUB
|
||||||
|
declaring an ASIN, a Google volume id and a Calibre id kept none of them once
|
||||||
|
a PDF contributed one ISBN.
|
||||||
|
"""
|
||||||
|
|
||||||
|
async def epub(_file):
|
||||||
|
return {
|
||||||
|
"title": "How Linux Works",
|
||||||
|
"identifiers": {"isbn-13": "9781718500419", "asin": "1718500408"},
|
||||||
|
}
|
||||||
|
|
||||||
|
async def pdf(_file):
|
||||||
|
return {"identifiers": {"isbn-13": "9781718500402", "isbn-10": "1593270356"}}
|
||||||
|
|
||||||
|
monkeypatch.setattr(EpubExtractor, "extract_metadata", epub)
|
||||||
|
monkeypatch.setattr(PdfExtractor, "extract_metadata", pdf)
|
||||||
|
|
||||||
|
metadata = await Extractor.extract_metadata(
|
||||||
|
[Path("How Linux Works.epub"), Path("How Linux Works.pdf")]
|
||||||
|
)
|
||||||
|
|
||||||
|
assert metadata["identifiers"] == {
|
||||||
|
# Declared by the publisher's toolchain, so it outranks the PDF's, which
|
||||||
|
# was scraped off a copyright page that also prints the print edition's.
|
||||||
|
"isbn-13": "9781718500419",
|
||||||
|
"asin": "1718500408",
|
||||||
|
"isbn-10": "1593270356",
|
||||||
|
}
|
||||||
|
|
||||||
|
async def test_a_second_format_that_finds_nothing_erases_nothing(self) -> None:
|
||||||
|
"""The PDF fixture carries no ISBN, so it must leave the EPUB's alone."""
|
||||||
|
metadata = await Extractor.extract_metadata([EPUB, PDF])
|
||||||
|
|
||||||
|
assert metadata["identifiers"] == {"id": "http://www.gutenberg.org/5200"}
|
||||||
|
|
||||||
|
async def test_one_format_on_its_own_is_unaffected(self) -> None:
|
||||||
|
metadata = await Extractor.extract_metadata([EPUB])
|
||||||
|
|
||||||
|
assert metadata["identifiers"] == {"id": "http://www.gutenberg.org/5200"}
|
||||||
|
|
||||||
|
async def test_no_identifiers_anywhere_leaves_the_field_absent(self) -> None:
|
||||||
|
"""An empty dict would count as extracted metadata and overwrite nothing."""
|
||||||
|
metadata = await Extractor.extract_metadata([PDF])
|
||||||
|
|
||||||
|
assert "identifiers" not in metadata
|
||||||
|
|
||||||
|
|
||||||
|
class TestSplitEdition:
|
||||||
|
"""An edition is a field on the book, not part of what the book is called."""
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("title", "stripped", "edition"),
|
||||||
|
[
|
||||||
|
# Every form below is one that turned up in a real library.
|
||||||
|
("Fluent Python, 2nd Edition", "Fluent Python", 2),
|
||||||
|
("Building Microservices, 2E", "Building Microservices", 2),
|
||||||
|
("Digital Image Processing, 4e", "Digital Image Processing", 4),
|
||||||
|
(
|
||||||
|
"Network Security Essentials: Applications and Standards/6e",
|
||||||
|
"Network Security Essentials: Applications and Standards",
|
||||||
|
6,
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"Refactoring: Improving the Design of Existing Code (2nd edition)",
|
||||||
|
"Refactoring: Improving the Design of Existing Code",
|
||||||
|
2,
|
||||||
|
),
|
||||||
|
# Ordinal words, including a qualifier sitting inside the statement.
|
||||||
|
(
|
||||||
|
"The Art of Computer Programming: Volume 1 / Fundamental Algorithms, Third Edition",
|
||||||
|
"The Art of Computer Programming: Volume 1 / Fundamental Algorithms",
|
||||||
|
3,
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"Introduction to the Theory of Computation, Third International Edition",
|
||||||
|
"Introduction to the Theory of Computation",
|
||||||
|
3,
|
||||||
|
),
|
||||||
|
# Mid-title, before a subtitle and before a trailing author.
|
||||||
|
(
|
||||||
|
"How Linux Works, 3rd Edition: What Every Superuser Should Know",
|
||||||
|
"How Linux Works: What Every Superuser Should Know",
|
||||||
|
3,
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"Code Complete, 2nd Edition - Steve McConnell",
|
||||||
|
"Code Complete - Steve McConnell",
|
||||||
|
2,
|
||||||
|
),
|
||||||
|
# An underscore between the number and the "e", beside an unnumbered
|
||||||
|
# qualifier that has nowhere to go in an integer column and so stays put.
|
||||||
|
(
|
||||||
|
"Cryptography and Network Security, Global Edition, 8_e - Stallings",
|
||||||
|
"Cryptography and Network Security, Global Edition - Stallings",
|
||||||
|
8,
|
||||||
|
),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_editions_are_split_out(self, title: str, stripped: str, edition: int) -> None:
|
||||||
|
assert split_edition(title) == (stripped, edition)
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"title",
|
||||||
|
[
|
||||||
|
# A number alone is never an edition — these are titles.
|
||||||
|
"Catch 22",
|
||||||
|
"Fahrenheit 451",
|
||||||
|
"Blade Runner 2049",
|
||||||
|
"1984",
|
||||||
|
"Apollo 13",
|
||||||
|
"Slaughterhouse 5",
|
||||||
|
"The Art of Computer Programming: Volume 1",
|
||||||
|
# "Edition" with no number cannot be stored, so it stays where it can
|
||||||
|
# still be read.
|
||||||
|
"Cryptography and Network Security: Principles and Practice, Global Edition",
|
||||||
|
"Building Microservices",
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_titles_are_left_alone(self, title: str) -> None:
|
||||||
|
assert split_edition(title) == (title, None)
|
||||||
|
|
||||||
|
def test_a_title_that_is_only_an_edition_is_kept(self) -> None:
|
||||||
|
"""Stripping must never leave a book with no title at all."""
|
||||||
|
assert split_edition("2nd Edition") == ("2nd Edition", None)
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("title", ["", None])
|
||||||
|
def test_nothing_yields_nothing(self, title: str | None) -> None:
|
||||||
|
assert split_edition(title) == (title, None)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio()
|
||||||
|
class TestEditionFromFiles:
|
||||||
|
async def test_extraction_moves_the_edition_off_the_title(self) -> None:
|
||||||
|
"""The PDF fixture calls itself a 2nd edition in its own metadata title."""
|
||||||
|
metadata = await Extractor.extract_metadata([PDF])
|
||||||
|
|
||||||
|
assert metadata["title"] == "The Project Gutenberg eBook #33283: Calculus Made Easy"
|
||||||
|
assert metadata["edition"] == 2
|
||||||
|
|
||||||
|
|
||||||
|
class TestEpubPublisher:
|
||||||
|
"""The publisher was looked up and then dropped on the floor."""
|
||||||
|
|
||||||
|
def test_a_declared_publisher_is_returned(self) -> None:
|
||||||
|
"""
|
||||||
|
The lookup discarded its own result and fell off the end of the function, so
|
||||||
|
every EPUB reported no publisher no matter what it said.
|
||||||
|
"""
|
||||||
|
book = epub.EpubBook()
|
||||||
|
book.add_metadata("DC", "publisher", "No Starch Press")
|
||||||
|
|
||||||
|
assert EpubExtractor._extract_publisher(book) == "No Starch Press"
|
||||||
|
|
||||||
|
def test_no_publisher_is_none(self) -> None:
|
||||||
|
assert EpubExtractor._extract_publisher(epub.EpubBook()) is None
|
||||||
|
|||||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,146 @@
|
|||||||
|
# Implementation brief: move Duplicates into library settings
|
||||||
|
|
||||||
|
Written for an agent picking this up cold. Read the repo-root `AGENTS.md` and
|
||||||
|
`frontend/AGENTS.md` first — this brief assumes both.
|
||||||
|
|
||||||
|
**This is a frontend-only change.** The backend already scopes everything by library
|
||||||
|
(`GET /books/duplicate-books?library_id=`), so no endpoint, schema or migration is
|
||||||
|
involved.
|
||||||
|
|
||||||
|
## Where this starts from
|
||||||
|
|
||||||
|
The duplicates review screen exists and works. It currently lives at
|
||||||
|
|
||||||
|
```
|
||||||
|
frontend/src/routes/(root)/(library)/library/[libraryId]/duplicates/
|
||||||
|
+page.server.ts loads the groups, plus the full Book records merge needs
|
||||||
|
+page.svelte group cards, "Not duplicates", "Merge…"
|
||||||
|
```
|
||||||
|
|
||||||
|
and is reached from a **Duplicates entry in the main sidebar**
|
||||||
|
(`frontend/src/lib/components/layout/nav-main.svelte`), which is what this change
|
||||||
|
removes.
|
||||||
|
|
||||||
|
Settings today is a flat, entirely global four-item nav
|
||||||
|
(`frontend/src/routes/(root)/settings/+layout.svelte`): Account, Appearance, Libraries,
|
||||||
|
Devices. `settings/libraries/+page.svelte` is a single table of every library whose rows
|
||||||
|
link *out* to the library itself. **There is nowhere that means "settings for this
|
||||||
|
library"** — that is the gap this change fills.
|
||||||
|
|
||||||
|
## What to build — option B
|
||||||
|
|
||||||
|
Libraries expands in the settings nav. Every library is a sub-item; selecting one swaps
|
||||||
|
the pane; Duplicates is a section inside that pane. All libraries and all their sections
|
||||||
|
end up one click apart.
|
||||||
|
|
||||||
|
```
|
||||||
|
/settings/libraries the existing table (leave it as the index)
|
||||||
|
/settings/libraries/[libraryId] redirects to the first section
|
||||||
|
/settings/libraries/[libraryId]/duplicates the review screen, moved
|
||||||
|
```
|
||||||
|
|
||||||
|
Suggested files:
|
||||||
|
|
||||||
|
| Path | What |
|
||||||
|
| --- | --- |
|
||||||
|
| `settings/libraries/[libraryId]/+layout.svelte` | Library name, and the section tabs |
|
||||||
|
| `settings/libraries/[libraryId]/+page.ts` | `redirect(303, …/duplicates)` |
|
||||||
|
| `settings/libraries/[libraryId]/duplicates/+page.server.ts` | Moved verbatim |
|
||||||
|
| `settings/libraries/[libraryId]/duplicates/+page.svelte` | Moved verbatim |
|
||||||
|
|
||||||
|
Duplicates is the **only** real section today. Build the tab strip so General and Danger
|
||||||
|
zone have somewhere obvious to land, but do not invent them now — an empty tab is worse
|
||||||
|
than no tab.
|
||||||
|
|
||||||
|
## The nav
|
||||||
|
|
||||||
|
In `settings/+layout.svelte`, `items` is a flat `as const` array matched on
|
||||||
|
`page.route.id`. Libraries needs to render its children beneath it:
|
||||||
|
|
||||||
|
```svelte
|
||||||
|
{#each libraryState.libraries as library (library.id)}
|
||||||
|
<a href={resolve('/(root)/settings/libraries/[libraryId]/duplicates', {
|
||||||
|
libraryId: String(library.id) })}> … </a>
|
||||||
|
{/each}
|
||||||
|
```
|
||||||
|
|
||||||
|
`getLibraryState()` **is** available under `/settings` — it is set in
|
||||||
|
`(root)/+layout.svelte`, above the settings group, and `settings/libraries/+page.svelte`
|
||||||
|
already uses it. No new load function is needed to list the libraries.
|
||||||
|
|
||||||
|
**Active state is matched on route id, not pathname.** There is a comment in
|
||||||
|
`settings/+layout.svelte` explaining why: `resolve()` returns an absolute path on the
|
||||||
|
client and a relative one during SSR, so a pathname comparison is false on the server and
|
||||||
|
true after hydration, and the highlight flashes in. A nested library item is active when
|
||||||
|
the route id matches **and** `page.params.libraryId === String(library.id)` — both, or
|
||||||
|
every library lights up at once.
|
||||||
|
|
||||||
|
## Things that will bite
|
||||||
|
|
||||||
|
1. **Remove the sidebar entry in the same change.** `nav-main.svelte` gained a
|
||||||
|
`Duplicates` item and a `CopyCheck` import when the screen was built. Delete both, and
|
||||||
|
delete the old route directory. Doing the removal and the move together is the point —
|
||||||
|
split across two commits the screen is unreachable in between.
|
||||||
|
|
||||||
|
2. **Delete the old route, do not leave it.** Two live copies of a screen that both write
|
||||||
|
is how they drift.
|
||||||
|
|
||||||
|
3. **`setBookSelectionState` is not available under `/settings`.** It is set in
|
||||||
|
`(root)/(library)/+layout.svelte`, which the settings group is not inside. This is
|
||||||
|
fine — the duplicates page uses `BookImage` directly, not `book-thumbnail.svelte`, and
|
||||||
|
`MergeBooks` takes its `libraryId` as a prop. **Verify this stays true** if you touch
|
||||||
|
either component; a `getBookSelectionState()` under settings returns `undefined` and
|
||||||
|
fails at the first access, not at import.
|
||||||
|
|
||||||
|
4. **Keep `depends('app:duplicate-books')`.** Both the dismiss action and `MergeBooks`
|
||||||
|
call `invalidate('app:duplicate-books')` to make a resolved group leave the screen.
|
||||||
|
Drop it and the page silently stops refreshing. `MergeBooks` also invalidates
|
||||||
|
`app:books`, which is a no-op under settings and should stay that way.
|
||||||
|
|
||||||
|
5. **The settings shell is height-constrained.** `settings/+layout.svelte` is
|
||||||
|
`h-[calc(100vh-var(--header-height)-2rem)]` with `overflow-auto` on the content pane.
|
||||||
|
The review screen is a long list of cards — it must scroll *inside* that pane. Its
|
||||||
|
current `mx-auto max-w-5xl` wrapper will want revisiting.
|
||||||
|
|
||||||
|
6. **Three levels of nav is option B's known cost.** Nav → library → section, and the
|
||||||
|
pane is narrower than the full-width route the screen was designed against. The group
|
||||||
|
cards are `w-36` covers in a wrapping flex row, so they reflow, but check a group of
|
||||||
|
four at a narrow window before calling it done.
|
||||||
|
|
||||||
|
7. **`resolve()` must be a direct call in markup** for `svelte/no-navigation-without-resolve`.
|
||||||
|
Where `nav-main.svelte` computes a url through a variable it carries an
|
||||||
|
`eslint-disable-next-line`; prefer the direct call over inheriting that.
|
||||||
|
|
||||||
|
8. **The loader depends on the `?ids=` fix.** `+page.server.ts` fetches full `Book`
|
||||||
|
records with `listBooks({ ids, pageSize })` because the merge workbench needs
|
||||||
|
identifiers, description and publisher, which `DuplicateBookRead` does not carry.
|
||||||
|
advanced_alchemy's stock id filter types that parameter as `list[str]` regardless of
|
||||||
|
config, which made Postgres refuse `bigint = character varying`; the override lives in
|
||||||
|
`backend/src/chitai/services/dependencies.py` (`create_book_filter_dependencies`).
|
||||||
|
If `GET /books?ids=1&ids=2` 500s, that override is missing — do not work around it in
|
||||||
|
the loader.
|
||||||
|
|
||||||
|
## Out of scope
|
||||||
|
|
||||||
|
The **General** and **Danger zone** sections (rename, path template, read-only, consume
|
||||||
|
directory, delete), and any change to the merge workbench itself. The toolbar entry point
|
||||||
|
for merge — select 2+ books in the library view — is unrelated and stays where it is.
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd frontend
|
||||||
|
pnpm check # baseline: 30 errors, 1 warning, 8 files — none of them yours
|
||||||
|
pnpm lint # not clean either; check the files you touched, not the tree
|
||||||
|
pnpm build
|
||||||
|
```
|
||||||
|
|
||||||
|
By hand, with a library that has a duplicate group:
|
||||||
|
|
||||||
|
- Settings → Libraries lists every library beneath it; clicking one opens its pane.
|
||||||
|
- Duplicates shows the same groups the old route did, and scrolls inside the settings pane.
|
||||||
|
- **Not duplicates** removes the group and it stays gone after a reload.
|
||||||
|
- **Merge…** opens the workbench, merges, and the group leaves the screen.
|
||||||
|
- The main sidebar no longer has a Duplicates entry, and
|
||||||
|
`/library/<id>/duplicates` no longer resolves.
|
||||||
|
- A library with no duplicates shows the empty state, not a blank pane.
|
||||||
+6
-4
@@ -157,10 +157,12 @@ but take a baseline first, because neither is clean (see below).
|
|||||||
|
|
||||||
Observed in the current tree — don't mistake these for intentional patterns to copy:
|
Observed in the current tree — don't mistake these for intentional patterns to copy:
|
||||||
|
|
||||||
- `src/lib/schema/openapi/schema.d.ts` is **stale** — it predates the `BookProgress` rework and has
|
- `pnpm check` is not clean. **Baseline as of 2026-08-13: 30 errors, 1 warning, 8 files**, most of
|
||||||
no `BookProgressRead`, so `book.progress` types as `{}`. It is the source of most of the errors
|
them in `src/routes/api/[...path]/+server.ts` (see below). Get your own baseline before assuming
|
||||||
`pnpm check` reports; regenerating it should clear them. **Baseline as of 2026-08-11: 37 errors,
|
an error is yours. `src/lib/schema/openapi/schema.d.ts` was regenerated on that date and is
|
||||||
1 warning, 11 files.** Get your own baseline before assuming an error is yours.
|
current; regenerate it again after any backend API change, with
|
||||||
|
`pnpm exec openapi-typescript http://localhost:8000/schema/openapi.json -o src/lib/schema/openapi/schema.d.ts`
|
||||||
|
against a backend running **your** branch — a stale server silently writes a stale file.
|
||||||
- `pnpm lint` does not pass either — 131 pre-existing ESLint errors, 35 of them
|
- `pnpm lint` does not pass either — 131 pre-existing ESLint errors, 35 of them
|
||||||
`svelte/no-navigation-without-resolve` on plain `href`s, plus ~59 files Prettier would rewrite
|
`svelte/no-navigation-without-resolve` on plain `href`s, plus ~59 files Prettier would rewrite
|
||||||
(mostly vendored shadcn components). Check the files you touched, not the whole tree.
|
(mostly vendored shadcn components). Check the files you touched, not the whole tree.
|
||||||
|
|||||||
@@ -8,12 +8,31 @@ import {
|
|||||||
deleteBooksSchema,
|
deleteBooksSchema,
|
||||||
editBookMetadataSchema,
|
editBookMetadataSchema,
|
||||||
updateBookProgressSchema,
|
updateBookProgressSchema,
|
||||||
|
duplicateDismissalSchema,
|
||||||
|
bookMergeSchema,
|
||||||
type Book,
|
type Book,
|
||||||
|
type BooksUploadResult,
|
||||||
|
type DuplicateBookGroup,
|
||||||
bookFilesUpload
|
bookFilesUpload
|
||||||
} from '$lib/schema/index';
|
} from '$lib/schema/index';
|
||||||
import { stringCoerce, type PaginatedResponse } from '$lib/schema/common';
|
import { stringCoerce, type PaginatedResponse } from '$lib/schema/common';
|
||||||
import { createQueryParams } from '$lib/utils';
|
import { createQueryParams } from '$lib/utils';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The backend's own message for a failed response, rather than its JSON envelope.
|
||||||
|
*
|
||||||
|
* A refused duplicate answers 409 with a `detail` worth reading and the offending
|
||||||
|
* files in `extra`; passing the body through whole puts JSON in front of the reader.
|
||||||
|
*/
|
||||||
|
function detailOf(body: string): string {
|
||||||
|
try {
|
||||||
|
const parsed = JSON.parse(body);
|
||||||
|
return typeof parsed?.detail === 'string' ? parsed.detail : body;
|
||||||
|
} catch {
|
||||||
|
return body;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
export const getBook = query(stringCoerce, async (id): Promise<Book> => {
|
export const getBook = query(stringCoerce, async (id): Promise<Book> => {
|
||||||
const { locals } = getRequestEvent();
|
const { locals } = getRequestEvent();
|
||||||
|
|
||||||
@@ -68,26 +87,29 @@ export const updateBookCover = form(bookCoverUpload, async ({ book_id, file }) =
|
|||||||
return await response.json();
|
return await response.json();
|
||||||
});
|
});
|
||||||
|
|
||||||
export const uploadBooks = form(booksUpload, async ({ library_id, files }) => {
|
export const uploadBooks = form(
|
||||||
const { locals } = getRequestEvent();
|
booksUpload,
|
||||||
|
async ({ library_id, files }): Promise<BooksUploadResult> => {
|
||||||
|
const { locals } = getRequestEvent();
|
||||||
|
|
||||||
const formData = new FormData();
|
const formData = new FormData();
|
||||||
files.forEach((file) => {
|
files.forEach((file) => {
|
||||||
formData.append('files', file);
|
formData.append('files', file);
|
||||||
});
|
});
|
||||||
|
|
||||||
const response = await locals.api.postMultipart(
|
const response = await locals.api.postMultipart(
|
||||||
`/books/fromFiles?library_id=${library_id}`,
|
`/books/fromFiles?library_id=${library_id}`,
|
||||||
formData
|
formData
|
||||||
);
|
);
|
||||||
|
|
||||||
if (!response.ok) {
|
if (!response.ok) {
|
||||||
const message = await response.text();
|
const message = await response.text();
|
||||||
error(response.status, message);
|
error(response.status, message);
|
||||||
|
}
|
||||||
|
|
||||||
|
return await response.json();
|
||||||
}
|
}
|
||||||
|
);
|
||||||
return await response.json();
|
|
||||||
});
|
|
||||||
|
|
||||||
export const uploadBookFiles = form(bookFilesUpload, async ({ book_id, files }) => {
|
export const uploadBookFiles = form(bookFilesUpload, async ({ book_id, files }) => {
|
||||||
const { locals } = getRequestEvent();
|
const { locals } = getRequestEvent();
|
||||||
@@ -100,8 +122,9 @@ export const uploadBookFiles = form(bookFilesUpload, async ({ book_id, files })
|
|||||||
const response = await locals.api.postMultipart(`/books/${book_id}/files`, formData);
|
const response = await locals.api.postMultipart(`/books/${book_id}/files`, formData);
|
||||||
|
|
||||||
if (!response.ok) {
|
if (!response.ok) {
|
||||||
const message = await response.text();
|
// 409 here means the file is already stored under a different book, which is
|
||||||
error(response.status, message);
|
// something the reader can act on — so the message has to survive the trip.
|
||||||
|
error(response.status, detailOf(await response.text()));
|
||||||
}
|
}
|
||||||
|
|
||||||
return await response.json();
|
return await response.json();
|
||||||
@@ -133,6 +156,65 @@ export const deleteBookFiles = command(deleteBookFilesSchema, async ({ book_id,
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Books already in the library that look like copies of one another.
|
||||||
|
*
|
||||||
|
* Metadata only, so every group is a question rather than a verdict — which is why
|
||||||
|
* the screen it feeds offers a way to disagree.
|
||||||
|
*/
|
||||||
|
export const listDuplicateBooks = query(
|
||||||
|
stringCoerce,
|
||||||
|
async (libraryId): Promise<DuplicateBookGroup[]> => {
|
||||||
|
const { locals } = getRequestEvent();
|
||||||
|
|
||||||
|
const response = await locals.api.get(`/books/duplicate-books?library_id=${libraryId}`);
|
||||||
|
|
||||||
|
if (!response.ok) error(response.status, detailOf(await response.text()));
|
||||||
|
|
||||||
|
return await response.json();
|
||||||
|
}
|
||||||
|
);
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Fold several books into one and delete the records folded in.
|
||||||
|
*
|
||||||
|
* Irreversible, so the caller is expected to have shown what is about to happen.
|
||||||
|
*/
|
||||||
|
export const mergeBooks = command(
|
||||||
|
bookMergeSchema,
|
||||||
|
async ({ library_id, ...data }): Promise<Book> => {
|
||||||
|
const { locals } = getRequestEvent();
|
||||||
|
|
||||||
|
const response = await locals.api.post(`/books/merge?library_id=${library_id}`, data);
|
||||||
|
|
||||||
|
if (!response.ok) error(response.status, detailOf(await response.text()));
|
||||||
|
|
||||||
|
return await response.json();
|
||||||
|
}
|
||||||
|
);
|
||||||
|
|
||||||
|
/** Record that two books are not the same book, so the pair stops being proposed. */
|
||||||
|
export const dismissDuplicateBooks = command(duplicateDismissalSchema, async (data) => {
|
||||||
|
const { locals } = getRequestEvent();
|
||||||
|
|
||||||
|
const response = await locals.api.post('/books/duplicate-books/dismissals', data);
|
||||||
|
|
||||||
|
if (!response.ok) error(response.status, detailOf(await response.text()));
|
||||||
|
});
|
||||||
|
|
||||||
|
/** Undo a dismissal, so the pair is proposed again. */
|
||||||
|
export const restoreDuplicateBooks = command(duplicateDismissalSchema, async (data) => {
|
||||||
|
const { locals } = getRequestEvent();
|
||||||
|
|
||||||
|
const params = createQueryParams(data);
|
||||||
|
|
||||||
|
const response = await locals.api.delete(
|
||||||
|
`/books/duplicate-books/dismissals?${params.toString()}`
|
||||||
|
);
|
||||||
|
|
||||||
|
if (!response.ok) error(response.status, detailOf(await response.text()));
|
||||||
|
});
|
||||||
|
|
||||||
export const updateBookProgress = command(
|
export const updateBookProgress = command(
|
||||||
updateBookProgressSchema,
|
updateBookProgressSchema,
|
||||||
async ({ book_ids, ...data }) => {
|
async ({ book_ids, ...data }) => {
|
||||||
|
|||||||
@@ -50,6 +50,22 @@
|
|||||||
toast.error(`${file.name} was not added`, { description: reason });
|
toast.error(`${file.name} was not added`, { description: reason });
|
||||||
};
|
};
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The API's own words, when it has any.
|
||||||
|
*
|
||||||
|
* Adding a file the library already holds under another book is refused with a
|
||||||
|
* 409 naming it — far more use than "failed to add files". SvelteKit hands an
|
||||||
|
* `error()` back as an HttpError on the client, so the message sits on `body`.
|
||||||
|
*/
|
||||||
|
function apiMessage(error: unknown): string | undefined {
|
||||||
|
if (typeof error !== 'object' || error === null) return undefined;
|
||||||
|
|
||||||
|
const body = (error as { body?: { message?: string } }).body;
|
||||||
|
if (typeof body?.message === 'string') return body.message;
|
||||||
|
|
||||||
|
return error instanceof Error ? error.message : undefined;
|
||||||
|
}
|
||||||
|
|
||||||
function confirmDelete(file: BookFile) {
|
function confirmDelete(file: BookFile) {
|
||||||
fileToDelete = file;
|
fileToDelete = file;
|
||||||
confirmOpen = true;
|
confirmOpen = true;
|
||||||
@@ -148,7 +164,8 @@
|
|||||||
toast.success('Files added');
|
toast.success('Files added');
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
console.error('Failed to add files: ', error);
|
console.error('Failed to add files: ', error);
|
||||||
toast.error('Failed to add files');
|
toast.error(apiMessage(error) ?? 'Failed to add files');
|
||||||
|
uploadBookFiles.fields.files.set([]);
|
||||||
}
|
}
|
||||||
})}
|
})}
|
||||||
enctype="multipart/form-data"
|
enctype="multipart/form-data"
|
||||||
|
|||||||
@@ -0,0 +1,149 @@
|
|||||||
|
import type { Book } from '$lib/schema';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* What kind of thing a metadata field holds, which is the only thing that decides
|
||||||
|
* what you can do with it when two records disagree.
|
||||||
|
*/
|
||||||
|
export type FieldKind = 'text' | 'number' | 'date' | 'list' | 'keyed' | 'longtext';
|
||||||
|
|
||||||
|
/** An action offered on a field, beyond replacing it outright. */
|
||||||
|
export type FieldAction = 'replace' | 'merge' | 'append';
|
||||||
|
|
||||||
|
export interface FieldSpec {
|
||||||
|
/** The key sent in `BookMetadataUpdate`. */
|
||||||
|
key: string;
|
||||||
|
label: string;
|
||||||
|
kind: FieldKind;
|
||||||
|
group: string;
|
||||||
|
/** Extra actions past `replace`, which every field has. */
|
||||||
|
extra: FieldAction[];
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The fields a merge can resolve, in the order and grouping the edit form uses.
|
||||||
|
*
|
||||||
|
* Deliberately a spec rather than markup: the merge workbench and, later, the
|
||||||
|
* provider review screen both render from this, so a field cannot exist in one and
|
||||||
|
* not the other. `cover` and `files` are absent because they are not choices —
|
||||||
|
* files always come across and the cover has its own endpoint.
|
||||||
|
*/
|
||||||
|
export const MERGE_FIELDS: FieldSpec[] = [
|
||||||
|
{ key: 'title', label: 'Title', kind: 'text', group: 'Identity', extra: [] },
|
||||||
|
{ key: 'subtitle', label: 'Subtitle', kind: 'text', group: 'Identity', extra: [] },
|
||||||
|
{ key: 'edition', label: 'Edition', kind: 'number', group: 'Identity', extra: [] },
|
||||||
|
{ key: 'series', label: 'Series', kind: 'text', group: 'Identity', extra: [] },
|
||||||
|
{ key: 'series_position', label: 'No.', kind: 'text', group: 'Identity', extra: [] },
|
||||||
|
{ key: 'language', label: 'Language', kind: 'text', group: 'Identity', extra: [] },
|
||||||
|
|
||||||
|
// Order is meaningful for authors, so the second list is appended rather than
|
||||||
|
// interleaved; tags are a set, so they merge.
|
||||||
|
{
|
||||||
|
key: 'authors',
|
||||||
|
label: 'Authors',
|
||||||
|
kind: 'list',
|
||||||
|
group: 'People and subjects',
|
||||||
|
extra: ['append']
|
||||||
|
},
|
||||||
|
{ key: 'tags', label: 'Tags', kind: 'list', group: 'People and subjects', extra: ['merge'] },
|
||||||
|
|
||||||
|
{ key: 'publisher', label: 'Publisher', kind: 'text', group: 'Publication', extra: [] },
|
||||||
|
{ key: 'published_date', label: 'Published', kind: 'date', group: 'Publication', extra: [] },
|
||||||
|
{ key: 'pages', label: 'Pages', kind: 'number', group: 'Publication', extra: [] },
|
||||||
|
{
|
||||||
|
key: 'identifiers',
|
||||||
|
label: 'Identifiers',
|
||||||
|
kind: 'keyed',
|
||||||
|
group: 'Publication',
|
||||||
|
extra: ['merge']
|
||||||
|
},
|
||||||
|
|
||||||
|
{
|
||||||
|
key: 'description',
|
||||||
|
label: 'Description',
|
||||||
|
kind: 'longtext',
|
||||||
|
group: 'Description',
|
||||||
|
extra: ['append']
|
||||||
|
}
|
||||||
|
];
|
||||||
|
|
||||||
|
/** A field's value, in the shape `BookMetadataUpdate` expects to receive it. */
|
||||||
|
export type FieldValue = string | number | string[] | Record<string, string> | null;
|
||||||
|
|
||||||
|
/** Read one field off a book, flattening the relations the API returns as objects. */
|
||||||
|
export function readField(book: Book, key: string): FieldValue {
|
||||||
|
switch (key) {
|
||||||
|
case 'authors':
|
||||||
|
return book.authors.map((author) => author.name);
|
||||||
|
case 'tags':
|
||||||
|
return book.tags.map((tag) => tag.name);
|
||||||
|
case 'publisher':
|
||||||
|
return book.publisher?.name ?? null;
|
||||||
|
case 'series':
|
||||||
|
return book.series?.title ?? null;
|
||||||
|
case 'identifiers':
|
||||||
|
return book.identifiers ?? {};
|
||||||
|
default:
|
||||||
|
return (book as unknown as Record<string, FieldValue>)[key] ?? null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Whether a field holds nothing, and so has no decision attached to it. */
|
||||||
|
export function isEmpty(value: FieldValue): boolean {
|
||||||
|
if (value === null || value === undefined || value === '') return true;
|
||||||
|
if (Array.isArray(value)) return value.length === 0;
|
||||||
|
if (typeof value === 'object') return Object.keys(value).length === 0;
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Whether two field values say the same thing, order included for lists. */
|
||||||
|
export function isSame(left: FieldValue, right: FieldValue): boolean {
|
||||||
|
if (isEmpty(left) && isEmpty(right)) return true;
|
||||||
|
return JSON.stringify(left ?? null) === JSON.stringify(right ?? null);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Apply an action to a pair of values and return what the target becomes.
|
||||||
|
*
|
||||||
|
* `merge` on a keyed collection is per name and the target wins a clash, because
|
||||||
|
* `Identifier` is unique on `(name, book_id)` — a book cannot hold both its print
|
||||||
|
* and its ebook ISBN, so the second one has nowhere to go.
|
||||||
|
*/
|
||||||
|
export function applyAction(
|
||||||
|
action: FieldAction,
|
||||||
|
kind: FieldKind,
|
||||||
|
target: FieldValue,
|
||||||
|
incoming: FieldValue
|
||||||
|
): FieldValue {
|
||||||
|
if (action === 'replace') return incoming;
|
||||||
|
|
||||||
|
if (kind === 'list') {
|
||||||
|
const current = Array.isArray(target) ? target : [];
|
||||||
|
const extra = Array.isArray(incoming) ? incoming : [];
|
||||||
|
// Order preserved, duplicates dropped — works for both append and merge.
|
||||||
|
return [...new Set([...current, ...extra])];
|
||||||
|
}
|
||||||
|
|
||||||
|
if (kind === 'keyed') {
|
||||||
|
return { ...(incoming as Record<string, string>), ...(target as Record<string, string>) };
|
||||||
|
}
|
||||||
|
|
||||||
|
if (kind === 'longtext') {
|
||||||
|
const current = typeof target === 'string' ? target.trim() : '';
|
||||||
|
const extra = typeof incoming === 'string' ? incoming.trim() : '';
|
||||||
|
return [current, extra].filter(Boolean).join('\n\n');
|
||||||
|
}
|
||||||
|
|
||||||
|
return incoming;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** How a value reads in the inert reference column. */
|
||||||
|
export function displayValue(value: FieldValue): string {
|
||||||
|
if (isEmpty(value)) return '';
|
||||||
|
if (Array.isArray(value)) return value.join(', ');
|
||||||
|
if (typeof value === 'object') {
|
||||||
|
return Object.entries(value as Record<string, string>)
|
||||||
|
.map(([name, id]) => `${name}: ${id}`)
|
||||||
|
.join(' · ');
|
||||||
|
}
|
||||||
|
return String(value);
|
||||||
|
}
|
||||||
@@ -0,0 +1,394 @@
|
|||||||
|
<script lang="ts">
|
||||||
|
import { untrack } from 'svelte';
|
||||||
|
import { ArrowRight, GitMerge, Plus, Undo2 } from '@lucide/svelte';
|
||||||
|
|
||||||
|
import * as Dialog from '$lib/components/ui/dialog/index.js';
|
||||||
|
import { Button } from '$lib/components/ui/button/index.js';
|
||||||
|
import { Input } from '$lib/components/ui/input/index.js';
|
||||||
|
import { Textarea } from '$lib/components/ui/textarea/index.js';
|
||||||
|
import { Badge } from '$lib/components/ui/badge/index.js';
|
||||||
|
import BookImage from '$lib/components/view/book-image.svelte';
|
||||||
|
import GeneratedCover from '$lib/components/view/generated-cover.svelte';
|
||||||
|
import type { Book } from '$lib/schema';
|
||||||
|
|
||||||
|
import { mergeInto } from './merge';
|
||||||
|
|
||||||
|
import {
|
||||||
|
MERGE_FIELDS,
|
||||||
|
applyAction,
|
||||||
|
displayValue,
|
||||||
|
isEmpty,
|
||||||
|
isSame,
|
||||||
|
readField,
|
||||||
|
type FieldAction,
|
||||||
|
type FieldSpec,
|
||||||
|
type FieldValue
|
||||||
|
} from './field-spec';
|
||||||
|
|
||||||
|
let {
|
||||||
|
books,
|
||||||
|
libraryId,
|
||||||
|
open = $bindable(),
|
||||||
|
onmerged
|
||||||
|
}: {
|
||||||
|
books: Book[];
|
||||||
|
libraryId: number | string;
|
||||||
|
open: boolean;
|
||||||
|
/** Given the record that survived and the ones folded into it and deleted. */
|
||||||
|
onmerged?: (survivor: Book, folded: Book[]) => void;
|
||||||
|
} = $props();
|
||||||
|
|
||||||
|
// Seeded once per mount. The dialog is keyed on the group upstream, so a
|
||||||
|
// different group gets a fresh workbench rather than the previous one's draft.
|
||||||
|
let survivorId = $state(untrack(() => books[0]?.id));
|
||||||
|
let candidateId = $state(untrack(() => books[1]?.id));
|
||||||
|
let draft = $state<Record<string, FieldValue>>({});
|
||||||
|
let busy = $state(false);
|
||||||
|
|
||||||
|
const survivor = $derived(books.find((book) => book.id === survivorId) ?? books[0]);
|
||||||
|
const candidates = $derived(books.filter((book) => book.id !== survivorId));
|
||||||
|
/** The records that will be deleted — the same set, named for what happens to them. */
|
||||||
|
const folded = $derived(candidates);
|
||||||
|
const candidate = $derived(candidates.find((book) => book.id === candidateId) ?? candidates[0]);
|
||||||
|
|
||||||
|
/** The survivor's stored value for a field, or the draft if it has been touched. */
|
||||||
|
function current(field: FieldSpec): FieldValue {
|
||||||
|
return field.key in draft ? draft[field.key] : readField(survivor, field.key);
|
||||||
|
}
|
||||||
|
|
||||||
|
function take(field: FieldSpec, action: FieldAction) {
|
||||||
|
draft[field.key] = applyAction(
|
||||||
|
action,
|
||||||
|
field.kind,
|
||||||
|
current(field),
|
||||||
|
readField(candidate, field.key)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
function undo(field: FieldSpec) {
|
||||||
|
delete draft[field.key];
|
||||||
|
draft = { ...draft };
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Fill only the fields the survivor has nothing in.
|
||||||
|
*
|
||||||
|
* The safe bulk action, and the one worth reaching for: it cannot overwrite a
|
||||||
|
* value, so it needs no per-field protection to be pressed without reading.
|
||||||
|
*/
|
||||||
|
function fillEmpty() {
|
||||||
|
for (const field of MERGE_FIELDS) {
|
||||||
|
const incoming = readField(candidate, field.key);
|
||||||
|
if (isEmpty(current(field)) && !isEmpty(incoming)) draft[field.key] = incoming;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const changed = $derived(Object.keys(draft));
|
||||||
|
|
||||||
|
const differing = $derived(
|
||||||
|
candidate
|
||||||
|
? MERGE_FIELDS.filter((field) => !isSame(current(field), readField(candidate, field.key)))
|
||||||
|
: []
|
||||||
|
);
|
||||||
|
|
||||||
|
/** Fields shown as a row: anything the two disagree on, plus anything edited. */
|
||||||
|
const shown = $derived(
|
||||||
|
MERGE_FIELDS.filter((field) => differing.includes(field) || field.key in draft)
|
||||||
|
);
|
||||||
|
|
||||||
|
const agreed = $derived(MERGE_FIELDS.filter((field) => !shown.includes(field)));
|
||||||
|
|
||||||
|
const groups = $derived([...new Set(shown.map((field) => field.group))]);
|
||||||
|
|
||||||
|
async function submit() {
|
||||||
|
if (busy) return;
|
||||||
|
busy = true;
|
||||||
|
|
||||||
|
const merged = await mergeInto(libraryId, survivor, folded, changed.length ? draft : undefined);
|
||||||
|
|
||||||
|
busy = false;
|
||||||
|
|
||||||
|
if (merged) {
|
||||||
|
open = false;
|
||||||
|
onmerged?.(survivor, folded);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
</script>
|
||||||
|
|
||||||
|
{#snippet cover(book: Book, size: string)}
|
||||||
|
<span class="{size} shrink-0 overflow-hidden rounded-sm border bg-muted">
|
||||||
|
{#if book.cover_image}
|
||||||
|
<BookImage src="/api/{book.cover_image}" class="h-full w-full object-cover" />
|
||||||
|
{:else}
|
||||||
|
<!-- Drawn rather than left blank, so the rail tells two coverless books
|
||||||
|
apart the same way the shelves do. -->
|
||||||
|
<GeneratedCover {book} />
|
||||||
|
{/if}
|
||||||
|
</span>
|
||||||
|
{/snippet}
|
||||||
|
|
||||||
|
<!-- The inert reference column: the same shape as the control opposite it, with
|
||||||
|
nothing that invites a click. -->
|
||||||
|
{#snippet reference(field: FieldSpec, book: Book)}
|
||||||
|
{@const value = readField(book, field.key)}
|
||||||
|
<div class="flex min-w-0 flex-col gap-1">
|
||||||
|
<span class="text-[11px] text-muted-foreground">{field.label}</span>
|
||||||
|
{#if isEmpty(value)}
|
||||||
|
<span class="min-h-8 py-1 text-sm text-muted-foreground italic">empty</span>
|
||||||
|
{:else if field.kind === 'list'}
|
||||||
|
<span class="flex min-h-8 flex-wrap items-center gap-1 py-0.5">
|
||||||
|
{#each value as string[] as item (item)}
|
||||||
|
<Badge variant="secondary" class="font-normal">{item}</Badge>
|
||||||
|
{/each}
|
||||||
|
</span>
|
||||||
|
{:else}
|
||||||
|
<span class="min-h-8 py-1 text-sm break-words">{displayValue(value)}</span>
|
||||||
|
{/if}
|
||||||
|
</div>
|
||||||
|
{/snippet}
|
||||||
|
|
||||||
|
<Dialog.Root bind:open>
|
||||||
|
<Dialog.Content
|
||||||
|
class="flex h-[85vh] max-w-[min(72rem,95vw)] flex-col gap-0 overflow-hidden p-0 sm:max-w-[min(72rem,95vw)]"
|
||||||
|
>
|
||||||
|
<Dialog.Header class="shrink-0 space-y-0 border-b px-5 py-3 text-left">
|
||||||
|
<Dialog.Title class="font-serif text-base font-normal">
|
||||||
|
Merge {books.length} books
|
||||||
|
</Dialog.Title>
|
||||||
|
<Dialog.Description class="text-xs">
|
||||||
|
{candidates.length}
|
||||||
|
{candidates.length === 1 ? 'record is' : 'records are'} deleted. Their files move onto the book
|
||||||
|
you keep — nothing is removed from disk.
|
||||||
|
</Dialog.Description>
|
||||||
|
</Dialog.Header>
|
||||||
|
|
||||||
|
<div class="grid min-h-0 flex-1 grid-cols-1 md:grid-cols-[15rem_1fr]">
|
||||||
|
<!-- Rail: the books being folded in, one open at a time -->
|
||||||
|
<aside
|
||||||
|
class="flex min-w-0 flex-col overflow-y-auto border-b bg-sidebar md:border-r md:border-b-0"
|
||||||
|
>
|
||||||
|
<p
|
||||||
|
class="px-3 pt-3 pb-1 font-mono text-[10px] tracking-widest text-muted-foreground uppercase"
|
||||||
|
>
|
||||||
|
Taking from · {candidates.length}
|
||||||
|
</p>
|
||||||
|
{#each candidates as book (book.id)}
|
||||||
|
{@const count = MERGE_FIELDS.filter(
|
||||||
|
(field) => !isSame(readField(survivor, field.key), readField(book, field.key))
|
||||||
|
).length}
|
||||||
|
<button
|
||||||
|
type="button"
|
||||||
|
onclick={() => (candidateId = book.id)}
|
||||||
|
class="flex items-start gap-2 border-l-2 px-3 py-2 text-left transition-colors hover:bg-muted/50 {book.id ===
|
||||||
|
candidate?.id
|
||||||
|
? 'border-l-primary bg-background'
|
||||||
|
: 'border-l-transparent'}"
|
||||||
|
>
|
||||||
|
{@render cover(book, 'h-10 w-7')}
|
||||||
|
<span class="min-w-0 flex-1">
|
||||||
|
<span class="line-clamp-2 font-serif text-[13px]">{book.title}</span>
|
||||||
|
<span class="block text-[10px] text-muted-foreground">
|
||||||
|
#{book.id} · {book.files.length}
|
||||||
|
{book.files.length === 1 ? 'file' : 'files'}
|
||||||
|
</span>
|
||||||
|
{#if count === 0}
|
||||||
|
<Badge variant="secondary" class="mt-1 text-[10px] font-normal">
|
||||||
|
nothing to take
|
||||||
|
</Badge>
|
||||||
|
{/if}
|
||||||
|
</span>
|
||||||
|
</button>
|
||||||
|
{/each}
|
||||||
|
</aside>
|
||||||
|
|
||||||
|
<div class="flex min-h-0 flex-col">
|
||||||
|
<!-- Which record survives -->
|
||||||
|
<div class="flex shrink-0 flex-wrap items-center gap-2 border-b px-5 py-2.5">
|
||||||
|
<span class="text-[11px] text-muted-foreground">Keeping</span>
|
||||||
|
{#each books as book (book.id)}
|
||||||
|
<button
|
||||||
|
type="button"
|
||||||
|
onclick={() => (survivorId = book.id)}
|
||||||
|
class="flex items-center gap-1.5 rounded-md border px-2 py-1 text-xs transition-colors {book.id ===
|
||||||
|
survivorId
|
||||||
|
? 'border-primary bg-accent text-accent-foreground'
|
||||||
|
: 'text-muted-foreground hover:bg-muted'}"
|
||||||
|
>
|
||||||
|
{@render cover(book, 'h-6 w-4')}
|
||||||
|
<span class="max-w-32 truncate">#{book.id}</span>
|
||||||
|
</button>
|
||||||
|
{/each}
|
||||||
|
|
||||||
|
<div class="ml-auto flex gap-2">
|
||||||
|
<Button variant="outline" size="sm" onclick={fillEmpty} disabled={!candidate}>
|
||||||
|
Fill empty fields
|
||||||
|
</Button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="min-h-0 flex-1 overflow-y-auto px-5 py-4">
|
||||||
|
{#if !candidate}
|
||||||
|
<p class="text-sm text-muted-foreground">Nothing left to fold in.</p>
|
||||||
|
{:else if shown.length === 0}
|
||||||
|
<p class="text-sm text-muted-foreground">
|
||||||
|
These records agree on every field. Merging keeps
|
||||||
|
<span class="font-medium text-foreground">#{survivor.id}</span> and moves the others' files
|
||||||
|
onto it.
|
||||||
|
</p>
|
||||||
|
{:else}
|
||||||
|
{#each groups as group (group)}
|
||||||
|
<h3
|
||||||
|
class="mt-5 border-b pb-1 font-mono text-[10px] tracking-widest text-primary uppercase first:mt-0"
|
||||||
|
>
|
||||||
|
{group}
|
||||||
|
</h3>
|
||||||
|
|
||||||
|
{#each shown.filter((field) => field.group === group) as field (field.key)}
|
||||||
|
{@const value = current(field)}
|
||||||
|
{@const incoming = readField(candidate, field.key)}
|
||||||
|
{@const edited = field.key in draft}
|
||||||
|
<div class="grid grid-cols-[1fr_auto_1fr] items-center gap-3 border-b py-2">
|
||||||
|
{@render reference(field, candidate)}
|
||||||
|
|
||||||
|
<!-- Actions, on the row rather than stacked beside it -->
|
||||||
|
<div class="flex items-center gap-1">
|
||||||
|
{#if edited}
|
||||||
|
<Button
|
||||||
|
variant="ghost"
|
||||||
|
size="icon"
|
||||||
|
class="size-7"
|
||||||
|
title="Undo"
|
||||||
|
onclick={() => undo(field)}
|
||||||
|
>
|
||||||
|
<Undo2 class="size-3.5" />
|
||||||
|
<span class="sr-only">Undo {field.label}</span>
|
||||||
|
</Button>
|
||||||
|
{/if}
|
||||||
|
|
||||||
|
{#if !isSame(value, incoming)}
|
||||||
|
{#if isEmpty(value)}
|
||||||
|
<!-- Nothing to weigh, so it is an offer rather than a choice -->
|
||||||
|
<Button
|
||||||
|
size="sm"
|
||||||
|
class="h-7 gap-1 px-2 text-[11px]"
|
||||||
|
onclick={() => take(field, 'replace')}
|
||||||
|
>
|
||||||
|
<ArrowRight class="size-3.5" />
|
||||||
|
Take
|
||||||
|
</Button>
|
||||||
|
{:else}
|
||||||
|
<Button
|
||||||
|
variant="outline"
|
||||||
|
size="icon"
|
||||||
|
class="size-7"
|
||||||
|
title="Replace"
|
||||||
|
onclick={() => take(field, 'replace')}
|
||||||
|
>
|
||||||
|
<ArrowRight class="size-3.5" />
|
||||||
|
<span class="sr-only">Replace {field.label}</span>
|
||||||
|
</Button>
|
||||||
|
{#each field.extra as action (action)}
|
||||||
|
<Button
|
||||||
|
variant="outline"
|
||||||
|
size="icon"
|
||||||
|
class="size-7"
|
||||||
|
title={action === 'merge' ? 'Merge' : 'Append'}
|
||||||
|
onclick={() => take(field, action)}
|
||||||
|
>
|
||||||
|
{#if action === 'merge'}
|
||||||
|
<GitMerge class="size-3.5" />
|
||||||
|
{:else}
|
||||||
|
<Plus class="size-3.5" />
|
||||||
|
{/if}
|
||||||
|
<span class="sr-only">{action} {field.label}</span>
|
||||||
|
</Button>
|
||||||
|
{/each}
|
||||||
|
{/if}
|
||||||
|
{/if}
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- The survivor's side: the edit form, live -->
|
||||||
|
<div class="flex min-w-0 flex-col gap-1">
|
||||||
|
<span class="flex items-center gap-1.5 text-[11px] text-muted-foreground">
|
||||||
|
{field.label}
|
||||||
|
{#if edited}
|
||||||
|
<Badge class="h-4 px-1.5 text-[9px] font-semibold">taken</Badge>
|
||||||
|
{/if}
|
||||||
|
</span>
|
||||||
|
|
||||||
|
{#if field.kind === 'longtext'}
|
||||||
|
<Textarea
|
||||||
|
rows={4}
|
||||||
|
class="text-sm"
|
||||||
|
value={(value as string) ?? ''}
|
||||||
|
oninput={(event) => (draft[field.key] = event.currentTarget.value)}
|
||||||
|
/>
|
||||||
|
{:else if field.kind === 'list' || field.kind === 'keyed'}
|
||||||
|
<!-- Edited through the take actions; typing here would need the
|
||||||
|
tags and key/value editors, which belong to the edit form. -->
|
||||||
|
<div
|
||||||
|
class="flex min-h-8 flex-wrap items-center gap-1 rounded-md border bg-background px-2 py-1"
|
||||||
|
>
|
||||||
|
{#if isEmpty(value)}
|
||||||
|
<span class="text-sm text-muted-foreground">—</span>
|
||||||
|
{:else if field.kind === 'list'}
|
||||||
|
{#each value as string[] as item (item)}
|
||||||
|
<Badge variant="secondary" class="font-normal">{item}</Badge>
|
||||||
|
{/each}
|
||||||
|
{:else}
|
||||||
|
{#each Object.entries(value as Record<string, string>) as [name, id] (name)}
|
||||||
|
<Badge variant="secondary" class="font-mono text-[10px] font-normal">
|
||||||
|
{name}: {id}
|
||||||
|
</Badge>
|
||||||
|
{/each}
|
||||||
|
{/if}
|
||||||
|
</div>
|
||||||
|
{:else}
|
||||||
|
<Input
|
||||||
|
type={field.kind === 'number'
|
||||||
|
? 'number'
|
||||||
|
: field.kind === 'date'
|
||||||
|
? 'date'
|
||||||
|
: 'text'}
|
||||||
|
class="h-8 text-sm"
|
||||||
|
value={(value as string | number) ?? ''}
|
||||||
|
oninput={(event) =>
|
||||||
|
(draft[field.key] =
|
||||||
|
field.kind === 'number'
|
||||||
|
? Number(event.currentTarget.value) || null
|
||||||
|
: event.currentTarget.value)}
|
||||||
|
/>
|
||||||
|
{/if}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
{/each}
|
||||||
|
{/each}
|
||||||
|
|
||||||
|
{#if agreed.length > 0}
|
||||||
|
<p class="pt-3 text-center text-[11px] text-muted-foreground italic">
|
||||||
|
{agreed.map((field) => field.label).join(', ')} — identical in both
|
||||||
|
</p>
|
||||||
|
{/if}
|
||||||
|
{/if}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<Dialog.Footer class="shrink-0 items-center gap-2 border-t px-5 py-3 sm:justify-between">
|
||||||
|
<p class="text-xs text-muted-foreground">
|
||||||
|
{#if changed.length}
|
||||||
|
{changed.length}
|
||||||
|
{changed.length === 1 ? 'change' : 'changes'} pending · this cannot be undone
|
||||||
|
{:else}
|
||||||
|
Metadata is left as #{survivor.id} has it · this cannot be undone
|
||||||
|
{/if}
|
||||||
|
</p>
|
||||||
|
<div class="flex gap-2">
|
||||||
|
<Button variant="outline" onclick={() => (open = false)}>Cancel</Button>
|
||||||
|
<Button onclick={submit} disabled={busy || candidates.length === 0}>
|
||||||
|
Merge into “{survivor.title}”
|
||||||
|
</Button>
|
||||||
|
</div>
|
||||||
|
</Dialog.Footer>
|
||||||
|
</Dialog.Content>
|
||||||
|
</Dialog.Root>
|
||||||
@@ -0,0 +1,44 @@
|
|||||||
|
import { invalidate } from '$app/navigation';
|
||||||
|
import { toast } from 'svelte-sonner';
|
||||||
|
|
||||||
|
import { mergeBooks } from '$lib/api/book.remote';
|
||||||
|
import type { Book } from '$lib/schema';
|
||||||
|
import type { FieldValue } from './field-spec';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Fold `folded` into `survivor`, and tell the reader how it went.
|
||||||
|
*
|
||||||
|
* Shared because a merge is reachable two ways — the workbench, where the reader
|
||||||
|
* resolved the metadata field by field, and the quick action beside it, which
|
||||||
|
* takes the survivor's metadata as it stands. Only the `metadata` argument
|
||||||
|
* differs, and the invalidation and the wording should not.
|
||||||
|
*
|
||||||
|
* @returns whether the merge went through; the caller decides what to close or
|
||||||
|
* clear, and has no toast of its own to write either way.
|
||||||
|
*/
|
||||||
|
export async function mergeInto(
|
||||||
|
libraryId: number | string,
|
||||||
|
survivor: Book,
|
||||||
|
folded: Book[],
|
||||||
|
metadata?: Record<string, FieldValue>
|
||||||
|
): Promise<boolean> {
|
||||||
|
try {
|
||||||
|
await mergeBooks({
|
||||||
|
library_id: libraryId,
|
||||||
|
survivor_id: survivor.id,
|
||||||
|
merged_ids: folded.map((book) => book.id),
|
||||||
|
metadata
|
||||||
|
});
|
||||||
|
|
||||||
|
// Both, because a merge is started from the duplicates review and from the
|
||||||
|
// library's selection toolbar, and each page depends on a different one.
|
||||||
|
await Promise.all([invalidate('app:books'), invalidate('app:duplicate-books')]);
|
||||||
|
|
||||||
|
toast.success(`Merged into “${survivor.title}”`);
|
||||||
|
return true;
|
||||||
|
} catch (error) {
|
||||||
|
console.error('Failed to merge books', error);
|
||||||
|
toast.error(error instanceof Error ? error.message : 'Could not merge these books');
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -33,13 +33,15 @@
|
|||||||
title: 'Home',
|
title: 'Home',
|
||||||
icon: House,
|
icon: House,
|
||||||
routeId: '/(root)/(library)/library/[libraryId]',
|
routeId: '/(root)/(library)/library/[libraryId]',
|
||||||
path: (id?: number) => resolve('/(root)/(library)/library/[libraryId]', { libraryId: String(id ?? '') })
|
path: (id?: number) =>
|
||||||
|
resolve('/(root)/(library)/library/[libraryId]', { libraryId: String(id ?? '') })
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
title: 'Library',
|
title: 'Library',
|
||||||
icon: LibraryBig,
|
icon: LibraryBig,
|
||||||
routeId: '/(root)/(library)/library/[libraryId]/view',
|
routeId: '/(root)/(library)/library/[libraryId]/view',
|
||||||
path: (id?: number) => resolve('/(root)/(library)/library/[libraryId]/view', { libraryId: String(id ?? '') })
|
path: (id?: number) =>
|
||||||
|
resolve('/(root)/(library)/library/[libraryId]/view', { libraryId: String(id ?? '') })
|
||||||
},
|
},
|
||||||
{ title: 'Shelves', icon: Rows3, routeId: null, path: () => '#', shelves: [] }
|
{ title: 'Shelves', icon: Rows3, routeId: null, path: () => '#', shelves: [] }
|
||||||
];
|
];
|
||||||
|
|||||||
@@ -1,11 +1,13 @@
|
|||||||
<script lang="ts">
|
<script lang="ts">
|
||||||
import { fly } from 'svelte/transition';
|
import { fly } from 'svelte/transition';
|
||||||
import { prefersReducedMotion } from 'svelte/motion';
|
import { prefersReducedMotion } from 'svelte/motion';
|
||||||
import { ChevronDown, CircleAlert, RotateCcw, X } from '@lucide/svelte';
|
import { resolve } from '$app/paths';
|
||||||
|
import { ChevronDown, CircleAlert, Copy, RotateCcw, X } from '@lucide/svelte';
|
||||||
|
|
||||||
import { Button } from '$lib/components/ui/button/index.js';
|
import { Button } from '$lib/components/ui/button/index.js';
|
||||||
import { Spinner } from '$lib/components/ui/spinner/index';
|
import { Spinner } from '$lib/components/ui/spinner/index';
|
||||||
import { getUploadQueueState } from '$lib/state/upload-queue.svelte';
|
import { getUploadQueueState } from '$lib/state/upload-queue.svelte';
|
||||||
|
import type { DuplicateBook, DuplicateFile } from '$lib/schema';
|
||||||
import { formatFileSize } from '$lib/utils';
|
import { formatFileSize } from '$lib/utils';
|
||||||
|
|
||||||
const queue = getUploadQueueState();
|
const queue = getUploadQueueState();
|
||||||
@@ -19,10 +21,48 @@
|
|||||||
const heading = $derived.by(() => {
|
const heading = $derived.by(() => {
|
||||||
const noun = queue.total === 1 ? 'book' : 'books';
|
const noun = queue.total === 1 ? 'book' : 'books';
|
||||||
if (queue.active) return `Adding ${queue.settled} of ${queue.total} ${noun}`;
|
if (queue.active) return `Adding ${queue.settled} of ${queue.total} ${noun}`;
|
||||||
if (queue.failed === 0) return `Added ${queue.done} ${noun}`;
|
if (queue.failed === 0 && queue.skipped === 0) return `Added ${queue.done} ${noun}`;
|
||||||
if (queue.done === 0) return `Couldn't add ${queue.failed} ${noun}`;
|
if (queue.done === 0 && queue.skipped === 0) return `Couldn't add ${queue.failed} ${noun}`;
|
||||||
return `Added ${queue.done}, ${queue.failed} failed`;
|
|
||||||
|
const parts = [];
|
||||||
|
if (queue.done > 0) parts.push(`Added ${queue.done}`);
|
||||||
|
if (queue.skipped > 0) parts.push(`${queue.skipped} already here`);
|
||||||
|
if (queue.failed > 0) parts.push(`${queue.failed} failed`);
|
||||||
|
return parts.join(', ');
|
||||||
});
|
});
|
||||||
|
|
||||||
|
/**
|
||||||
|
* What the reader needs to know about files that were not stored.
|
||||||
|
*
|
||||||
|
* One duplicate can name where its bytes already live; several would be a list,
|
||||||
|
* and the row has no room for one, so they collapse to a count.
|
||||||
|
*/
|
||||||
|
function duplicateNote(duplicates: DuplicateFile[]) {
|
||||||
|
if (duplicates.length > 1) return `${duplicates.length} files are already in your library`;
|
||||||
|
|
||||||
|
const [only] = duplicates;
|
||||||
|
// No book to name: it matched another file in this same upload.
|
||||||
|
return only.book_title ? `Already in ${only.book_title}` : 'Already added by this upload';
|
||||||
|
}
|
||||||
|
|
||||||
|
/** The book a note can link to, when a single duplicate points at exactly one. */
|
||||||
|
function noteTarget(duplicates: DuplicateFile[]) {
|
||||||
|
return duplicates.length === 1 ? duplicates[0].book_id : null;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* What the reader needs to know about a book that went in and may already be here.
|
||||||
|
*
|
||||||
|
* Worded weaker than the file-level "Already in …" on purpose. That one means the
|
||||||
|
* library holds these exact bytes; this one means the metadata agrees, which a
|
||||||
|
* second edition, a translation and a re-scan all do. Nothing was refused.
|
||||||
|
*/
|
||||||
|
function possibleNote(candidates: DuplicateBook[]) {
|
||||||
|
if (candidates.length > 1)
|
||||||
|
return `Might already be in your library, ${candidates.length} times`;
|
||||||
|
|
||||||
|
return `Might already be in your library — ${candidates[0].title}`;
|
||||||
|
}
|
||||||
</script>
|
</script>
|
||||||
|
|
||||||
{#if queue.total > 0}
|
{#if queue.total > 0}
|
||||||
@@ -44,6 +84,9 @@
|
|||||||
<Spinner class="size-4 shrink-0" />
|
<Spinner class="size-4 shrink-0" />
|
||||||
{:else if queue.failed > 0}
|
{:else if queue.failed > 0}
|
||||||
<CircleAlert class="size-4 shrink-0 text-destructive" />
|
<CircleAlert class="size-4 shrink-0 text-destructive" />
|
||||||
|
{:else if queue.skipped > 0}
|
||||||
|
<!-- Muted, not alarming: nothing went wrong, the books were already here. -->
|
||||||
|
<Copy class="size-4 shrink-0 text-muted-foreground" />
|
||||||
{/if}
|
{/if}
|
||||||
|
|
||||||
<span class="min-w-0 flex-1 truncate text-sm font-medium">{heading}</span>
|
<span class="min-w-0 flex-1 truncate text-sm font-medium">{heading}</span>
|
||||||
@@ -78,12 +121,61 @@
|
|||||||
{#if !queue.collapsed}
|
{#if !queue.collapsed}
|
||||||
<ul class="flex max-h-64 flex-col divide-y overflow-y-auto">
|
<ul class="flex max-h-64 flex-col divide-y overflow-y-auto">
|
||||||
{#each queue.jobs as job (job.id)}
|
{#each queue.jobs as job (job.id)}
|
||||||
|
{@const duplicates = job.duplicates ?? []}
|
||||||
|
{@const target = noteTarget(duplicates)}
|
||||||
|
{@const possible = job.possibleDuplicates ?? []}
|
||||||
<li class="flex min-w-0 items-center gap-2 px-3 py-2">
|
<li class="flex min-w-0 items-center gap-2 px-3 py-2">
|
||||||
<span class="min-w-0 flex-1">
|
<span class="min-w-0 flex-1">
|
||||||
<span class="block truncate text-sm" title={job.label}>{job.label}</span>
|
<span class="block truncate text-sm" title={job.label}>{job.label}</span>
|
||||||
<span class="block font-mono text-[10px] text-muted-foreground tabular-nums">
|
|
||||||
{job.error ?? formatFileSize(job.size)}
|
{#if job.error}
|
||||||
</span>
|
<span class="block font-mono text-[10px] text-muted-foreground tabular-nums">
|
||||||
|
{job.error}
|
||||||
|
</span>
|
||||||
|
{:else if duplicates.length > 0}
|
||||||
|
<!--
|
||||||
|
Reported, with no offer to override. Storing the same bytes twice
|
||||||
|
splits reading progress and shelves across two records that can
|
||||||
|
never converge, and there is no version of that the reader wants.
|
||||||
|
`allow_duplicates` stays on the API for when the match is wrong.
|
||||||
|
-->
|
||||||
|
<span class="block min-w-0 text-[10px] text-muted-foreground">
|
||||||
|
{#if target}
|
||||||
|
<a
|
||||||
|
href={resolve('/(root)/(library)/book/[bookId]', { bookId: String(target) })}
|
||||||
|
class="truncate underline underline-offset-2"
|
||||||
|
>
|
||||||
|
{duplicateNote(duplicates)}
|
||||||
|
</a>
|
||||||
|
{:else}
|
||||||
|
<span class="truncate">{duplicateNote(duplicates)}</span>
|
||||||
|
{/if}
|
||||||
|
</span>
|
||||||
|
{:else if possible.length > 0}
|
||||||
|
<!--
|
||||||
|
A guess, not a fact, so it says so and stops there: the book was
|
||||||
|
stored, and the library's duplicates screen is where a reader
|
||||||
|
decides what to do about it.
|
||||||
|
-->
|
||||||
|
<span class="block min-w-0 text-[10px] text-muted-foreground">
|
||||||
|
{#if possible.length === 1}
|
||||||
|
<a
|
||||||
|
href={resolve('/(root)/(library)/book/[bookId]', {
|
||||||
|
bookId: String(possible[0].book_id)
|
||||||
|
})}
|
||||||
|
class="truncate underline underline-offset-2"
|
||||||
|
>
|
||||||
|
{possibleNote(possible)}
|
||||||
|
</a>
|
||||||
|
{:else}
|
||||||
|
<span class="truncate">{possibleNote(possible)}</span>
|
||||||
|
{/if}
|
||||||
|
</span>
|
||||||
|
{:else}
|
||||||
|
<span class="block font-mono text-[10px] text-muted-foreground tabular-nums">
|
||||||
|
{formatFileSize(job.size)}
|
||||||
|
</span>
|
||||||
|
{/if}
|
||||||
</span>
|
</span>
|
||||||
|
|
||||||
<span
|
<span
|
||||||
@@ -96,6 +188,8 @@
|
|||||||
Adding
|
Adding
|
||||||
{:else if job.status === 'done'}
|
{:else if job.status === 'done'}
|
||||||
Done
|
Done
|
||||||
|
{:else if job.status === 'skipped'}
|
||||||
|
Skipped
|
||||||
{:else if job.status === 'failed'}
|
{:else if job.status === 'failed'}
|
||||||
Failed
|
Failed
|
||||||
{:else}
|
{:else}
|
||||||
|
|||||||
@@ -1,15 +1,27 @@
|
|||||||
<script lang="ts">
|
<script lang="ts">
|
||||||
import { buttonVariants } from '$lib/components/ui/button/button.svelte';
|
import { buttonVariants } from '$lib/components/ui/button/button.svelte';
|
||||||
import * as Tooltip from '$lib/components/ui/tooltip/index';
|
import * as Tooltip from '$lib/components/ui/tooltip/index';
|
||||||
import { BookOpenCheck, Download, Trash2, SquareCheckBig, X, Album, PlusIcon } from '@lucide/svelte';
|
import {
|
||||||
|
BookOpenCheck,
|
||||||
|
Download,
|
||||||
|
GitMerge,
|
||||||
|
Trash2,
|
||||||
|
SquareCheckBig,
|
||||||
|
SlidersHorizontal,
|
||||||
|
X,
|
||||||
|
Album,
|
||||||
|
PlusIcon
|
||||||
|
} from '@lucide/svelte';
|
||||||
import * as DropdownMenu from '$lib/components/ui/dropdown-menu/index';
|
import * as DropdownMenu from '$lib/components/ui/dropdown-menu/index';
|
||||||
import { getBookOperationsState } from '$lib/state/bookOperations.svelte';
|
import { getBookOperationsState } from '$lib/state/bookOperations.svelte';
|
||||||
import { getBookSelectionState } from '$lib/state/bookSelection.svelte';
|
import { getBookSelectionState } from '$lib/state/bookSelection.svelte';
|
||||||
import { getBookCollectionState } from '$lib/state/bookCollection.svelte';
|
import { getBookCollectionState } from '$lib/state/bookCollection.svelte';
|
||||||
import { getLibraryState } from '$lib/state/library.svelte';
|
import { getLibraryState } from '$lib/state/library.svelte';
|
||||||
import { getBookshelfState } from '$lib/state/bookshelf.svelte';
|
import { getBookshelfState } from '$lib/state/bookshelf.svelte';
|
||||||
import { invalidate } from '$app/navigation';
|
|
||||||
import ShelfCreateDialog from '../forms/shelf-create-dialog.svelte';
|
import ShelfCreateDialog from '../forms/shelf-create-dialog.svelte';
|
||||||
|
import MergeBooks from '../forms/merge-books/merge-books.svelte';
|
||||||
|
import { mergeInto } from '../forms/merge-books/merge';
|
||||||
|
import type { Book } from '$lib/schema';
|
||||||
|
|
||||||
const libraryState = getLibraryState();
|
const libraryState = getLibraryState();
|
||||||
const bookshelfState = getBookshelfState();
|
const bookshelfState = getBookshelfState();
|
||||||
@@ -17,10 +29,50 @@
|
|||||||
const bookOps = getBookOperationsState();
|
const bookOps = getBookOperationsState();
|
||||||
const collectionState = getBookCollectionState();
|
const collectionState = getBookCollectionState();
|
||||||
|
|
||||||
let selectedBooks = $derived(selectionState.getSelectedBooks())
|
let selectedBooks = $derived(selectionState.getSelectedBooks());
|
||||||
|
|
||||||
let createShelfDialogOpen = $state(false)
|
let createShelfDialogOpen = $state(false);
|
||||||
|
|
||||||
|
// The books the merge dialog opened on. Snapshotted rather than read live from
|
||||||
|
// the selection, so clearing the selection on success cannot empty the dialog
|
||||||
|
// underneath itself.
|
||||||
|
let mergingBooks = $state<Book[] | null>(null);
|
||||||
|
|
||||||
|
// A quick merge has no dialog to disable, so the flag is what stops a slow
|
||||||
|
// round trip being started twice.
|
||||||
|
let quickMerging = $state(false);
|
||||||
|
|
||||||
|
/** Put the library back in step with a merge that went through. */
|
||||||
|
function afterMerge(folded: Book[]) {
|
||||||
|
// Only the folded records are gone; the survivor is still in the library.
|
||||||
|
libraryState.activeLibrary!.total! -= folded.length;
|
||||||
|
bookshelfState.deletedBooks(folded);
|
||||||
|
selectionState.deselectAll();
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Merge without opening the workbench, keeping the book selected first.
|
||||||
|
*
|
||||||
|
* Selection is held in insertion order, so the first record is the one the
|
||||||
|
* reader started from — the one they were looking at when they decided the
|
||||||
|
* rest were copies of it.
|
||||||
|
*/
|
||||||
|
async function quickMerge() {
|
||||||
|
if (quickMerging) return;
|
||||||
|
|
||||||
|
// Read once: the merge clears the selection, and the bookkeeping afterwards
|
||||||
|
// still needs the records that went away.
|
||||||
|
const [survivor, ...folded] = selectedBooks;
|
||||||
|
if (!survivor || folded.length === 0) return;
|
||||||
|
|
||||||
|
quickMerging = true;
|
||||||
|
|
||||||
|
const merged = await mergeInto(libraryState.activeLibrary!.id, survivor, folded);
|
||||||
|
|
||||||
|
quickMerging = false;
|
||||||
|
|
||||||
|
if (merged) afterMerge(folded);
|
||||||
|
}
|
||||||
</script>
|
</script>
|
||||||
|
|
||||||
<!-- Mark selected as finished button -->
|
<!-- Mark selected as finished button -->
|
||||||
@@ -95,12 +147,13 @@
|
|||||||
{/each}
|
{/each}
|
||||||
</DropdownMenu.Group>
|
</DropdownMenu.Group>
|
||||||
<DropdownMenu.Separator />
|
<DropdownMenu.Separator />
|
||||||
<DropdownMenu.Item
|
<DropdownMenu.Item
|
||||||
onclick={() => createShelfDialogOpen = true}
|
onclick={() => (createShelfDialogOpen = true)}
|
||||||
class="text-muted-foreground ">
|
class="text-muted-foreground "
|
||||||
<PlusIcon class="size-4" />
|
>
|
||||||
New Shelf
|
<PlusIcon class="size-4" />
|
||||||
</DropdownMenu.Item>
|
New Shelf
|
||||||
|
</DropdownMenu.Item>
|
||||||
</DropdownMenu.Content>
|
</DropdownMenu.Content>
|
||||||
</DropdownMenu.Root>
|
</DropdownMenu.Root>
|
||||||
{/snippet}
|
{/snippet}
|
||||||
@@ -128,6 +181,44 @@
|
|||||||
</Tooltip.Root>
|
</Tooltip.Root>
|
||||||
</Tooltip.Provider>
|
</Tooltip.Provider>
|
||||||
|
|
||||||
|
<!-- Merge selected button. Two books is the smallest thing a merge can mean, so
|
||||||
|
it appears only once there are two. -->
|
||||||
|
{#if selectedBooks.length > 1}
|
||||||
|
<Tooltip.Provider>
|
||||||
|
<Tooltip.Root ignoreNonKeyboardFocus>
|
||||||
|
<Tooltip.Trigger>
|
||||||
|
{#snippet child({ props })}
|
||||||
|
<!-- A choice rather than a jump straight into the workbench: two
|
||||||
|
copies of one book have nothing to resolve, and the dialog is a
|
||||||
|
step in the way of saying so. -->
|
||||||
|
<DropdownMenu.Root>
|
||||||
|
<DropdownMenu.Trigger
|
||||||
|
{...props}
|
||||||
|
disabled={quickMerging}
|
||||||
|
class={buttonVariants({ variant: 'ghost', size: 'icon' })}
|
||||||
|
>
|
||||||
|
<GitMerge />
|
||||||
|
</DropdownMenu.Trigger>
|
||||||
|
<DropdownMenu.Content>
|
||||||
|
<DropdownMenu.Item onclick={quickMerge}>
|
||||||
|
<GitMerge class="size-4 shrink-0" />
|
||||||
|
Merge into first selected
|
||||||
|
</DropdownMenu.Item>
|
||||||
|
<DropdownMenu.Item onclick={() => (mergingBooks = selectedBooks)}>
|
||||||
|
<SlidersHorizontal class="size-4 shrink-0" />
|
||||||
|
Manual merge…
|
||||||
|
</DropdownMenu.Item>
|
||||||
|
</DropdownMenu.Content>
|
||||||
|
</DropdownMenu.Root>
|
||||||
|
{/snippet}
|
||||||
|
</Tooltip.Trigger>
|
||||||
|
<Tooltip.Content>
|
||||||
|
<p>Merge {selectedBooks.length} books</p>
|
||||||
|
</Tooltip.Content>
|
||||||
|
</Tooltip.Root>
|
||||||
|
</Tooltip.Provider>
|
||||||
|
{/if}
|
||||||
|
|
||||||
<!-- Delete selected button -->
|
<!-- Delete selected button -->
|
||||||
<Tooltip.Provider>
|
<Tooltip.Provider>
|
||||||
<Tooltip.Root ignoreNonKeyboardFocus>
|
<Tooltip.Root ignoreNonKeyboardFocus>
|
||||||
@@ -136,8 +227,8 @@
|
|||||||
bookOps.deleteDialogTitle = `Delete ${selectionState.getSelectedIds().length} books?`;
|
bookOps.deleteDialogTitle = `Delete ${selectionState.getSelectedIds().length} books?`;
|
||||||
bookOps.deleteFn = async (deleteFiles: boolean) => {
|
bookOps.deleteFn = async (deleteFiles: boolean) => {
|
||||||
await bookOps.deleteBooks(selectionState.getSelectedIds(), deleteFiles);
|
await bookOps.deleteBooks(selectionState.getSelectedIds(), deleteFiles);
|
||||||
libraryState.activeLibrary!.total! -= selectedBooks.length
|
libraryState.activeLibrary!.total! -= selectedBooks.length;
|
||||||
bookshelfState.deletedBooks(selectedBooks)
|
bookshelfState.deletedBooks(selectedBooks);
|
||||||
selectionState.deselectAll();
|
selectionState.deselectAll();
|
||||||
};
|
};
|
||||||
bookOps.deleteDialogOpen = true;
|
bookOps.deleteDialogOpen = true;
|
||||||
@@ -186,14 +277,43 @@
|
|||||||
</Tooltip.Root>
|
</Tooltip.Root>
|
||||||
</Tooltip.Provider>
|
</Tooltip.Provider>
|
||||||
|
|
||||||
|
<ShelfCreateDialog
|
||||||
<ShelfCreateDialog
|
bind:open={createShelfDialogOpen}
|
||||||
bind:open={createShelfDialogOpen}
|
|
||||||
onSubmit={async (name: string) => {
|
onSubmit={async (name: string) => {
|
||||||
|
const bookshelf = await bookshelfState.addBookshelf(
|
||||||
const bookshelf = await bookshelfState.addBookshelf(name, libraryState.activeLibrary!.id, selectionState.getSelectedIds())
|
name,
|
||||||
selectedBooks.forEach(book => book.lists.push(bookshelf))
|
libraryState.activeLibrary!.id,
|
||||||
selectionState.deselectAll()
|
selectionState.getSelectedIds()
|
||||||
createShelfDialogOpen = false
|
);
|
||||||
|
selectedBooks.forEach((book) => book.lists.push(bookshelf));
|
||||||
|
selectionState.deselectAll();
|
||||||
|
createShelfDialogOpen = false;
|
||||||
}}
|
}}
|
||||||
/>
|
/>
|
||||||
|
|
||||||
|
<!--
|
||||||
|
Keyed on the selection so opening merge on a different pair starts from those
|
||||||
|
records rather than the previous dialog's draft.
|
||||||
|
-->
|
||||||
|
{#if mergingBooks}
|
||||||
|
{#key mergingBooks.map((book) => book.id).join()}
|
||||||
|
<MergeBooks
|
||||||
|
books={mergingBooks}
|
||||||
|
libraryId={libraryState.activeLibrary!.id}
|
||||||
|
bind:open={
|
||||||
|
() => mergingBooks !== null,
|
||||||
|
(value) => {
|
||||||
|
// Bound, not passed as a bare `true`: the dialog closes itself on an
|
||||||
|
// outside click or Escape, and if that never reaches `mergingBooks`
|
||||||
|
// the snapshot stays set — the menu item then re-assigns the same
|
||||||
|
// selection, nothing changes, and the dialog never reopens.
|
||||||
|
if (!value) mergingBooks = null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
onmerged={(_survivor, folded) => {
|
||||||
|
afterMerge(folded);
|
||||||
|
mergingBooks = null;
|
||||||
|
}}
|
||||||
|
/>
|
||||||
|
{/key}
|
||||||
|
{/if}
|
||||||
|
|||||||
@@ -1,7 +1,14 @@
|
|||||||
<script lang="ts">
|
<script lang="ts">
|
||||||
import type { Book } from '$lib/schema';
|
// Typed by what is drawn rather than by `Book`: the duplicates review holds
|
||||||
|
// candidates that carry a title and author names without a whole record, and a
|
||||||
let { book, class: className = '' }: { book: Book; class?: string } = $props();
|
// `Book` satisfies this shape anyway.
|
||||||
|
let {
|
||||||
|
book,
|
||||||
|
class: className = ''
|
||||||
|
}: {
|
||||||
|
book: { title: string; authors?: { name: string }[] | null };
|
||||||
|
class?: string;
|
||||||
|
} = $props();
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Bookcloth tones rather than a point on the hue wheel.
|
* Bookcloth tones rather than a point on the hue wheel.
|
||||||
|
|||||||
@@ -6,6 +6,49 @@ export type Book = components['schemas']['BookRead'];
|
|||||||
export type BookFile = components['schemas']['FileMetadataRead'];
|
export type BookFile = components['schemas']['FileMetadataRead'];
|
||||||
export type BookProgress = components['schemas']['BookProgressRead'];
|
export type BookProgress = components['schemas']['BookProgressRead'];
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A file the library already held, so the upload did not store it again.
|
||||||
|
*
|
||||||
|
* `book_id` is null when the match was another file in the same upload — there is
|
||||||
|
* no book to point at yet.
|
||||||
|
*/
|
||||||
|
export type DuplicateFile = components['schemas']['DuplicateFileRead'];
|
||||||
|
export type BooksUploadResult = components['schemas']['BooksUploadResult'];
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A stored book that may be the same book as another one.
|
||||||
|
*
|
||||||
|
* Weaker than `DuplicateFile`, and deliberately so: that one means the library holds
|
||||||
|
* these exact bytes, this one means the metadata agrees. A second edition and a
|
||||||
|
* translation both look like this, so nothing is ever refused on the strength of it.
|
||||||
|
*/
|
||||||
|
export type DuplicateBook = components['schemas']['DuplicateBookRead'];
|
||||||
|
|
||||||
|
/** A book that was imported, together with what it might be a second copy of. */
|
||||||
|
export type PossibleDuplicate = components['schemas']['PossibleDuplicateRead'];
|
||||||
|
|
||||||
|
/** Books the library holds that all look like copies of one book. */
|
||||||
|
export type DuplicateBookGroup = components['schemas']['DuplicateBookGroupRead'];
|
||||||
|
|
||||||
|
/** The metadata a reader resolved while merging, or while reviewing a provider. */
|
||||||
|
export type BookMetadataUpdate = components['schemas']['BookMetadataUpdate'];
|
||||||
|
|
||||||
|
/** Mirrors BookMerge in backend/src/chitai/schemas/book.py */
|
||||||
|
export const bookMergeSchema = z.object({
|
||||||
|
library_id: z.coerce.number(),
|
||||||
|
survivor_id: z.coerce.number(),
|
||||||
|
merged_ids: z.array(z.coerce.number()).min(1, 'Pick at least one book to fold in'),
|
||||||
|
// Passed through untouched — the backend validates it as BookMetadataUpdate, and
|
||||||
|
// duplicating that shape here would be two places to keep in step for no gain.
|
||||||
|
metadata: z.record(z.string(), z.unknown()).optional()
|
||||||
|
});
|
||||||
|
|
||||||
|
/** Mirrors DuplicateDismissal in backend/src/chitai/schemas/book.py */
|
||||||
|
export const duplicateDismissalSchema = z.object({
|
||||||
|
book_a_id: z.coerce.number(),
|
||||||
|
book_b_id: z.coerce.number()
|
||||||
|
});
|
||||||
|
|
||||||
export const bookQuerySchema = commonQuerySchema.extend({
|
export const bookQuerySchema = commonQuerySchema.extend({
|
||||||
libraries: stringArrayCoerce,
|
libraries: stringArrayCoerce,
|
||||||
authors: stringArrayCoerce,
|
authors: stringArrayCoerce,
|
||||||
@@ -97,4 +140,6 @@ export type BookQuery = z.infer<typeof bookQuerySchema>;
|
|||||||
export type BookEditMetadata = z.infer<typeof editBookMetadataSchema>;
|
export type BookEditMetadata = z.infer<typeof editBookMetadataSchema>;
|
||||||
export type DeleteBook = z.infer<typeof deleteBooksSchema>;
|
export type DeleteBook = z.infer<typeof deleteBooksSchema>;
|
||||||
export type DeleteBookFiles = z.infer<typeof deleteBookFilesSchema>;
|
export type DeleteBookFiles = z.infer<typeof deleteBookFilesSchema>;
|
||||||
|
export type DuplicateDismissal = z.infer<typeof duplicateDismissalSchema>;
|
||||||
|
export type BookMerge = z.infer<typeof bookMergeSchema>;
|
||||||
export type UpdateBookProgress = z.infer<typeof updateBookProgressSchema>;
|
export type UpdateBookProgress = z.infer<typeof updateBookProgressSchema>;
|
||||||
|
|||||||
+319
-11
@@ -22,6 +22,23 @@ export interface paths {
|
|||||||
patch?: never;
|
patch?: never;
|
||||||
trace?: never;
|
trace?: never;
|
||||||
};
|
};
|
||||||
|
"/books/duplicate-files": {
|
||||||
|
parameters: {
|
||||||
|
query?: never;
|
||||||
|
header?: never;
|
||||||
|
path?: never;
|
||||||
|
cookie?: never;
|
||||||
|
};
|
||||||
|
get?: never;
|
||||||
|
put?: never;
|
||||||
|
/** CheckDuplicateFiles */
|
||||||
|
post: operations["BooksDuplicateFilesCheckDuplicateFiles"];
|
||||||
|
delete?: never;
|
||||||
|
options?: never;
|
||||||
|
head?: never;
|
||||||
|
patch?: never;
|
||||||
|
trace?: never;
|
||||||
|
};
|
||||||
"/books": {
|
"/books": {
|
||||||
parameters: {
|
parameters: {
|
||||||
query?: never;
|
query?: never;
|
||||||
@@ -58,6 +75,24 @@ export interface paths {
|
|||||||
patch?: never;
|
patch?: never;
|
||||||
trace?: never;
|
trace?: never;
|
||||||
};
|
};
|
||||||
|
"/books/duplicate-books/dismissals": {
|
||||||
|
parameters: {
|
||||||
|
query?: never;
|
||||||
|
header?: never;
|
||||||
|
path?: never;
|
||||||
|
cookie?: never;
|
||||||
|
};
|
||||||
|
get?: never;
|
||||||
|
put?: never;
|
||||||
|
/** DismissDuplicateBooks */
|
||||||
|
post: operations["BooksDuplicateBooksDismissalsDismissDuplicateBooks"];
|
||||||
|
/** RestoreDuplicateBooks */
|
||||||
|
delete: operations["BooksDuplicateBooksDismissalsRestoreDuplicateBooks"];
|
||||||
|
options?: never;
|
||||||
|
head?: never;
|
||||||
|
patch?: never;
|
||||||
|
trace?: never;
|
||||||
|
};
|
||||||
"/books/{book_id}": {
|
"/books/{book_id}": {
|
||||||
parameters: {
|
parameters: {
|
||||||
query?: never;
|
query?: never;
|
||||||
@@ -110,6 +145,40 @@ export interface paths {
|
|||||||
patch?: never;
|
patch?: never;
|
||||||
trace?: never;
|
trace?: never;
|
||||||
};
|
};
|
||||||
|
"/books/duplicate-books": {
|
||||||
|
parameters: {
|
||||||
|
query?: never;
|
||||||
|
header?: never;
|
||||||
|
path?: never;
|
||||||
|
cookie?: never;
|
||||||
|
};
|
||||||
|
/** ListDuplicateBooks */
|
||||||
|
get: operations["BooksDuplicateBooksListDuplicateBooks"];
|
||||||
|
put?: never;
|
||||||
|
post?: never;
|
||||||
|
delete?: never;
|
||||||
|
options?: never;
|
||||||
|
head?: never;
|
||||||
|
patch?: never;
|
||||||
|
trace?: never;
|
||||||
|
};
|
||||||
|
"/books/merge": {
|
||||||
|
parameters: {
|
||||||
|
query?: never;
|
||||||
|
header?: never;
|
||||||
|
path?: never;
|
||||||
|
cookie?: never;
|
||||||
|
};
|
||||||
|
get?: never;
|
||||||
|
put?: never;
|
||||||
|
/** MergeBooks */
|
||||||
|
post: operations["BooksMergeMergeBooks"];
|
||||||
|
delete?: never;
|
||||||
|
options?: never;
|
||||||
|
head?: never;
|
||||||
|
patch?: never;
|
||||||
|
trace?: never;
|
||||||
|
};
|
||||||
"/books/progress": {
|
"/books/progress": {
|
||||||
parameters: {
|
parameters: {
|
||||||
query?: never;
|
query?: never;
|
||||||
@@ -637,6 +706,12 @@ export interface components {
|
|||||||
cover_image?: string | null;
|
cover_image?: string | null;
|
||||||
files?: string[];
|
files?: string[];
|
||||||
};
|
};
|
||||||
|
/** BookMerge */
|
||||||
|
BookMerge: {
|
||||||
|
survivor_id: number;
|
||||||
|
merged_ids: number[];
|
||||||
|
metadata?: components["schemas"]["BookMetadataUpdate"] | null;
|
||||||
|
};
|
||||||
/** BookMetadataUpdate */
|
/** BookMetadataUpdate */
|
||||||
BookMetadataUpdate: {
|
BookMetadataUpdate: {
|
||||||
title?: string | null;
|
title?: string | null;
|
||||||
@@ -710,6 +785,46 @@ export interface components {
|
|||||||
BooksCreateFromFiles: {
|
BooksCreateFromFiles: {
|
||||||
files?: string[];
|
files?: string[];
|
||||||
};
|
};
|
||||||
|
/** BooksUploadResult */
|
||||||
|
BooksUploadResult: {
|
||||||
|
created: components["schemas"]["BookRead"][];
|
||||||
|
skipped: components["schemas"]["DuplicateFileRead"][];
|
||||||
|
possible_duplicates?: components["schemas"]["PossibleDuplicateRead"][];
|
||||||
|
};
|
||||||
|
/** DuplicateBookGroupRead */
|
||||||
|
DuplicateBookGroupRead: {
|
||||||
|
books: components["schemas"]["DuplicateBookRead"][];
|
||||||
|
};
|
||||||
|
/** DuplicateBookRead */
|
||||||
|
DuplicateBookRead: {
|
||||||
|
book_id: number;
|
||||||
|
title: string;
|
||||||
|
authors: string[];
|
||||||
|
library_id: number;
|
||||||
|
cover_image?: string | null;
|
||||||
|
matched_on: string[];
|
||||||
|
};
|
||||||
|
/** DuplicateDismissal */
|
||||||
|
DuplicateDismissal: {
|
||||||
|
book_a_id: number;
|
||||||
|
book_b_id: number;
|
||||||
|
};
|
||||||
|
/** DuplicateFileRead */
|
||||||
|
DuplicateFileRead: {
|
||||||
|
filename: string;
|
||||||
|
hash: string;
|
||||||
|
size: number;
|
||||||
|
library_id: number;
|
||||||
|
book_id?: number | null;
|
||||||
|
book_title?: string | null;
|
||||||
|
};
|
||||||
|
/** FileFingerprint */
|
||||||
|
FileFingerprint: {
|
||||||
|
hash: string;
|
||||||
|
size: number;
|
||||||
|
/** @default */
|
||||||
|
filename: string;
|
||||||
|
};
|
||||||
/** FileMetadataRead */
|
/** FileMetadataRead */
|
||||||
FileMetadataRead: {
|
FileMetadataRead: {
|
||||||
id: number;
|
id: number;
|
||||||
@@ -777,6 +892,12 @@ export interface components {
|
|||||||
refresh_token?: string | null;
|
refresh_token?: string | null;
|
||||||
expires_in?: number | null;
|
expires_in?: number | null;
|
||||||
};
|
};
|
||||||
|
/** PossibleDuplicateRead */
|
||||||
|
PossibleDuplicateRead: {
|
||||||
|
book_id: number;
|
||||||
|
title: string;
|
||||||
|
candidates: components["schemas"]["DuplicateBookRead"][];
|
||||||
|
};
|
||||||
/** PublisherRead */
|
/** PublisherRead */
|
||||||
PublisherRead: {
|
PublisherRead: {
|
||||||
id: number;
|
id: number;
|
||||||
@@ -828,6 +949,7 @@ export interface operations {
|
|||||||
parameters: {
|
parameters: {
|
||||||
query?: {
|
query?: {
|
||||||
library_id?: number | null;
|
library_id?: number | null;
|
||||||
|
allow_duplicates?: boolean;
|
||||||
};
|
};
|
||||||
header?: never;
|
header?: never;
|
||||||
path: {
|
path: {
|
||||||
@@ -908,6 +1030,47 @@ export interface operations {
|
|||||||
};
|
};
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
|
BooksDuplicateFilesCheckDuplicateFiles: {
|
||||||
|
parameters: {
|
||||||
|
query?: {
|
||||||
|
library_id?: number | null;
|
||||||
|
};
|
||||||
|
header?: never;
|
||||||
|
path?: never;
|
||||||
|
cookie?: never;
|
||||||
|
};
|
||||||
|
requestBody: {
|
||||||
|
content: {
|
||||||
|
"application/json": components["schemas"]["FileFingerprint"][];
|
||||||
|
};
|
||||||
|
};
|
||||||
|
responses: {
|
||||||
|
/** @description Request fulfilled, document follows */
|
||||||
|
200: {
|
||||||
|
headers: {
|
||||||
|
[name: string]: unknown;
|
||||||
|
};
|
||||||
|
content: {
|
||||||
|
"application/json": components["schemas"]["DuplicateFileRead"][];
|
||||||
|
};
|
||||||
|
};
|
||||||
|
/** @description Bad request syntax or unsupported method */
|
||||||
|
400: {
|
||||||
|
headers: {
|
||||||
|
[name: string]: unknown;
|
||||||
|
};
|
||||||
|
content: {
|
||||||
|
"application/json": {
|
||||||
|
status_code: number;
|
||||||
|
detail: string;
|
||||||
|
extra?: null | {
|
||||||
|
[key: string]: unknown;
|
||||||
|
} | unknown[];
|
||||||
|
};
|
||||||
|
};
|
||||||
|
};
|
||||||
|
};
|
||||||
|
};
|
||||||
BooksListBooks: {
|
BooksListBooks: {
|
||||||
parameters: {
|
parameters: {
|
||||||
query?: {
|
query?: {
|
||||||
@@ -917,7 +1080,7 @@ export interface operations {
|
|||||||
tags?: number[] | null;
|
tags?: number[] | null;
|
||||||
shelves?: number[] | null;
|
shelves?: number[] | null;
|
||||||
progress?: string[] | null;
|
progress?: string[] | null;
|
||||||
ids?: string[] | null;
|
ids?: number[] | null;
|
||||||
searchString?: string | null;
|
searchString?: string | null;
|
||||||
searchIgnoreCase?: boolean | null;
|
searchIgnoreCase?: boolean | null;
|
||||||
currentPage?: number;
|
currentPage?: number;
|
||||||
@@ -969,6 +1132,7 @@ export interface operations {
|
|||||||
parameters: {
|
parameters: {
|
||||||
query?: {
|
query?: {
|
||||||
library_id?: number | null;
|
library_id?: number | null;
|
||||||
|
allow_duplicates?: boolean;
|
||||||
};
|
};
|
||||||
header?: never;
|
header?: never;
|
||||||
path?: never;
|
path?: never;
|
||||||
@@ -1047,6 +1211,7 @@ export interface operations {
|
|||||||
parameters: {
|
parameters: {
|
||||||
query?: {
|
query?: {
|
||||||
library_id?: number | null;
|
library_id?: number | null;
|
||||||
|
allow_duplicates?: boolean;
|
||||||
};
|
};
|
||||||
header?: never;
|
header?: never;
|
||||||
path?: never;
|
path?: never;
|
||||||
@@ -1060,21 +1225,86 @@ export interface operations {
|
|||||||
responses: {
|
responses: {
|
||||||
/** @description Document created, URL follows */
|
/** @description Document created, URL follows */
|
||||||
201: {
|
201: {
|
||||||
|
headers: {
|
||||||
|
[name: string]: unknown;
|
||||||
|
};
|
||||||
|
content: {
|
||||||
|
"application/json": components["schemas"]["BooksUploadResult"];
|
||||||
|
};
|
||||||
|
};
|
||||||
|
/** @description Bad request syntax or unsupported method */
|
||||||
|
400: {
|
||||||
headers: {
|
headers: {
|
||||||
[name: string]: unknown;
|
[name: string]: unknown;
|
||||||
};
|
};
|
||||||
content: {
|
content: {
|
||||||
"application/json": {
|
"application/json": {
|
||||||
items?: components["schemas"]["BookRead"][];
|
status_code: number;
|
||||||
/** @description Maximal number of items to send. */
|
detail: string;
|
||||||
limit?: number;
|
extra?: null | {
|
||||||
/** @description Offset from the beginning of the query. */
|
[key: string]: unknown;
|
||||||
offset?: number;
|
} | unknown[];
|
||||||
/** @description Total number of items. */
|
|
||||||
total?: number;
|
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
|
};
|
||||||
|
};
|
||||||
|
BooksDuplicateBooksDismissalsDismissDuplicateBooks: {
|
||||||
|
parameters: {
|
||||||
|
query?: never;
|
||||||
|
header?: never;
|
||||||
|
path?: never;
|
||||||
|
cookie?: never;
|
||||||
|
};
|
||||||
|
requestBody: {
|
||||||
|
content: {
|
||||||
|
"application/json": components["schemas"]["DuplicateDismissal"];
|
||||||
|
};
|
||||||
|
};
|
||||||
|
responses: {
|
||||||
|
/** @description Request fulfilled, nothing follows */
|
||||||
|
204: {
|
||||||
|
headers: {
|
||||||
|
[name: string]: unknown;
|
||||||
|
};
|
||||||
|
content?: never;
|
||||||
|
};
|
||||||
|
/** @description Bad request syntax or unsupported method */
|
||||||
|
400: {
|
||||||
|
headers: {
|
||||||
|
[name: string]: unknown;
|
||||||
|
};
|
||||||
|
content: {
|
||||||
|
"application/json": {
|
||||||
|
status_code: number;
|
||||||
|
detail: string;
|
||||||
|
extra?: null | {
|
||||||
|
[key: string]: unknown;
|
||||||
|
} | unknown[];
|
||||||
|
};
|
||||||
|
};
|
||||||
|
};
|
||||||
|
};
|
||||||
|
};
|
||||||
|
BooksDuplicateBooksDismissalsRestoreDuplicateBooks: {
|
||||||
|
parameters: {
|
||||||
|
query: {
|
||||||
|
book_a_id: number;
|
||||||
|
book_b_id: number;
|
||||||
|
};
|
||||||
|
header?: never;
|
||||||
|
path?: never;
|
||||||
|
cookie?: never;
|
||||||
|
};
|
||||||
|
requestBody?: never;
|
||||||
|
responses: {
|
||||||
|
/** @description Request fulfilled, nothing follows */
|
||||||
|
204: {
|
||||||
|
headers: {
|
||||||
|
[name: string]: unknown;
|
||||||
|
};
|
||||||
|
content?: never;
|
||||||
|
};
|
||||||
/** @description Bad request syntax or unsupported method */
|
/** @description Bad request syntax or unsupported method */
|
||||||
400: {
|
400: {
|
||||||
headers: {
|
headers: {
|
||||||
@@ -1254,6 +1484,84 @@ export interface operations {
|
|||||||
};
|
};
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
|
BooksDuplicateBooksListDuplicateBooks: {
|
||||||
|
parameters: {
|
||||||
|
query?: {
|
||||||
|
library_id?: number | null;
|
||||||
|
};
|
||||||
|
header?: never;
|
||||||
|
path?: never;
|
||||||
|
cookie?: never;
|
||||||
|
};
|
||||||
|
requestBody?: never;
|
||||||
|
responses: {
|
||||||
|
/** @description Request fulfilled, document follows */
|
||||||
|
200: {
|
||||||
|
headers: {
|
||||||
|
[name: string]: unknown;
|
||||||
|
};
|
||||||
|
content: {
|
||||||
|
"application/json": components["schemas"]["DuplicateBookGroupRead"][];
|
||||||
|
};
|
||||||
|
};
|
||||||
|
/** @description Bad request syntax or unsupported method */
|
||||||
|
400: {
|
||||||
|
headers: {
|
||||||
|
[name: string]: unknown;
|
||||||
|
};
|
||||||
|
content: {
|
||||||
|
"application/json": {
|
||||||
|
status_code: number;
|
||||||
|
detail: string;
|
||||||
|
extra?: null | {
|
||||||
|
[key: string]: unknown;
|
||||||
|
} | unknown[];
|
||||||
|
};
|
||||||
|
};
|
||||||
|
};
|
||||||
|
};
|
||||||
|
};
|
||||||
|
BooksMergeMergeBooks: {
|
||||||
|
parameters: {
|
||||||
|
query?: {
|
||||||
|
library_id?: number | null;
|
||||||
|
};
|
||||||
|
header?: never;
|
||||||
|
path?: never;
|
||||||
|
cookie?: never;
|
||||||
|
};
|
||||||
|
requestBody: {
|
||||||
|
content: {
|
||||||
|
"application/json": components["schemas"]["BookMerge"];
|
||||||
|
};
|
||||||
|
};
|
||||||
|
responses: {
|
||||||
|
/** @description Document created, URL follows */
|
||||||
|
201: {
|
||||||
|
headers: {
|
||||||
|
[name: string]: unknown;
|
||||||
|
};
|
||||||
|
content: {
|
||||||
|
"application/json": components["schemas"]["BookRead"];
|
||||||
|
};
|
||||||
|
};
|
||||||
|
/** @description Bad request syntax or unsupported method */
|
||||||
|
400: {
|
||||||
|
headers: {
|
||||||
|
[name: string]: unknown;
|
||||||
|
};
|
||||||
|
content: {
|
||||||
|
"application/json": {
|
||||||
|
status_code: number;
|
||||||
|
detail: string;
|
||||||
|
extra?: null | {
|
||||||
|
[key: string]: unknown;
|
||||||
|
} | unknown[];
|
||||||
|
};
|
||||||
|
};
|
||||||
|
};
|
||||||
|
};
|
||||||
|
};
|
||||||
BooksProgressSetBookProgressBatch: {
|
BooksProgressSetBookProgressBatch: {
|
||||||
parameters: {
|
parameters: {
|
||||||
query: {
|
query: {
|
||||||
@@ -1931,7 +2239,7 @@ export interface operations {
|
|||||||
tags?: number[] | null;
|
tags?: number[] | null;
|
||||||
shelves?: number[] | null;
|
shelves?: number[] | null;
|
||||||
progress?: string[] | null;
|
progress?: string[] | null;
|
||||||
ids?: string[] | null;
|
ids?: number[] | null;
|
||||||
searchString?: string | null;
|
searchString?: string | null;
|
||||||
searchIgnoreCase?: boolean | null;
|
searchIgnoreCase?: boolean | null;
|
||||||
currentPage?: number;
|
currentPage?: number;
|
||||||
@@ -2018,7 +2326,7 @@ export interface operations {
|
|||||||
OpdsLibraryLibraryIdCollectionTypeGetLibraryCollectionFeed: {
|
OpdsLibraryLibraryIdCollectionTypeGetLibraryCollectionFeed: {
|
||||||
parameters: {
|
parameters: {
|
||||||
query?: {
|
query?: {
|
||||||
ids?: string[] | null;
|
ids?: number[] | null;
|
||||||
searchString?: string | null;
|
searchString?: string | null;
|
||||||
searchIgnoreCase?: boolean | null;
|
searchIgnoreCase?: boolean | null;
|
||||||
currentPage?: number;
|
currentPage?: number;
|
||||||
@@ -2147,7 +2455,7 @@ export interface operations {
|
|||||||
tags?: number[] | null;
|
tags?: number[] | null;
|
||||||
shelves?: number[] | null;
|
shelves?: number[] | null;
|
||||||
progress?: string[] | null;
|
progress?: string[] | null;
|
||||||
ids?: string[] | null;
|
ids?: number[] | null;
|
||||||
searchString?: string | null;
|
searchString?: string | null;
|
||||||
searchIgnoreCase?: boolean | null;
|
searchIgnoreCase?: boolean | null;
|
||||||
currentPage?: number;
|
currentPage?: number;
|
||||||
|
|||||||
@@ -1,9 +1,9 @@
|
|||||||
import { getContext, setContext } from 'svelte';
|
import { getContext, setContext } from 'svelte';
|
||||||
import { invalidate } from '$app/navigation';
|
import { invalidate } from '$app/navigation';
|
||||||
|
|
||||||
import type { Book, PaginatedResponse } from '$lib/schema';
|
import type { Book, BooksUploadResult, DuplicateBook, DuplicateFile } from '$lib/schema';
|
||||||
|
|
||||||
export type UploadStatus = 'queued' | 'uploading' | 'done' | 'failed';
|
export type UploadStatus = 'queued' | 'uploading' | 'done' | 'skipped' | 'failed';
|
||||||
|
|
||||||
export interface UploadJob {
|
export interface UploadJob {
|
||||||
id: string;
|
id: string;
|
||||||
@@ -15,10 +15,23 @@ export interface UploadJob {
|
|||||||
status: UploadStatus;
|
status: UploadStatus;
|
||||||
error?: string;
|
error?: string;
|
||||||
book?: Book;
|
book?: Book;
|
||||||
|
/**
|
||||||
|
* Files the library already held, which were not stored again. A job with
|
||||||
|
* nothing left over lands as `skipped`; one that had something new is `done`
|
||||||
|
* and still carries these, since the reader asked for those files too.
|
||||||
|
*/
|
||||||
|
duplicates?: DuplicateFile[];
|
||||||
|
/**
|
||||||
|
* Books already in the library that the one this job created might be a second
|
||||||
|
* copy of. Much weaker than `duplicates`: the bytes are new and only the metadata
|
||||||
|
* agrees, which a second edition and a translation both do. The book was stored.
|
||||||
|
*/
|
||||||
|
possibleDuplicates?: DuplicateBook[];
|
||||||
}
|
}
|
||||||
|
|
||||||
export interface UploadSummary {
|
export interface UploadSummary {
|
||||||
created: number;
|
created: number;
|
||||||
|
skipped: number;
|
||||||
failed: number;
|
failed: number;
|
||||||
firstBook?: Book;
|
firstBook?: Book;
|
||||||
}
|
}
|
||||||
@@ -70,12 +83,30 @@ export class UploadQueueState {
|
|||||||
|
|
||||||
readonly total = $derived(this.jobs.length);
|
readonly total = $derived(this.jobs.length);
|
||||||
readonly done = $derived(this.jobs.filter((job) => job.status === 'done').length);
|
readonly done = $derived(this.jobs.filter((job) => job.status === 'done').length);
|
||||||
|
readonly skipped = $derived(this.jobs.filter((job) => job.status === 'skipped').length);
|
||||||
readonly failed = $derived(this.jobs.filter((job) => job.status === 'failed').length);
|
readonly failed = $derived(this.jobs.filter((job) => job.status === 'failed').length);
|
||||||
readonly active = $derived(
|
readonly active = $derived(
|
||||||
this.jobs.some((job) => job.status === 'queued' || job.status === 'uploading')
|
this.jobs.some((job) => job.status === 'queued' || job.status === 'uploading')
|
||||||
);
|
);
|
||||||
readonly current = $derived(this.jobs.find((job) => job.status === 'uploading'));
|
readonly current = $derived(this.jobs.find((job) => job.status === 'uploading'));
|
||||||
readonly settled = $derived(this.done + this.failed);
|
readonly settled = $derived(this.done + this.skipped + this.failed);
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Whether the run left something the reader still has to see.
|
||||||
|
*
|
||||||
|
* A skipped book is not a failure, but it is the only place that says the file
|
||||||
|
* was already here — and the only place to override it from. A book that may
|
||||||
|
* already be in the library counts too: it is a note the reader has to actually
|
||||||
|
* read, and a tray that clears itself after six seconds is one they never will.
|
||||||
|
*/
|
||||||
|
readonly needsAttention = $derived(
|
||||||
|
this.jobs.some(
|
||||||
|
(job) =>
|
||||||
|
job.status === 'failed' ||
|
||||||
|
(job.duplicates?.length ?? 0) > 0 ||
|
||||||
|
(job.possibleDuplicates?.length ?? 0) > 0
|
||||||
|
)
|
||||||
|
);
|
||||||
|
|
||||||
#running = false;
|
#running = false;
|
||||||
#dismissTimer: ReturnType<typeof setTimeout> | undefined;
|
#dismissTimer: ReturnType<typeof setTimeout> | undefined;
|
||||||
@@ -129,20 +160,20 @@ export class UploadQueueState {
|
|||||||
|
|
||||||
release() {
|
release() {
|
||||||
this.#held = false;
|
this.#held = false;
|
||||||
if (!this.active && this.failed === 0 && this.total > 0) this.#scheduleDismiss();
|
if (!this.active && !this.needsAttention && this.total > 0) this.#scheduleDismiss();
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Clears itself only when everything worked. A run with failures stays until
|
* Clears itself only when everything went in cleanly. A run with failures or
|
||||||
* dismissed — it is the only record of what did not make it in, and the only
|
* skipped files stays until dismissed — it is the only record of what did not
|
||||||
* place to retry from.
|
* make it in, and the only place to retry or override from.
|
||||||
*/
|
*/
|
||||||
#scheduleDismiss() {
|
#scheduleDismiss() {
|
||||||
clearTimeout(this.#dismissTimer);
|
clearTimeout(this.#dismissTimer);
|
||||||
if (this.#held) return;
|
if (this.#held) return;
|
||||||
|
|
||||||
this.#dismissTimer = setTimeout(() => {
|
this.#dismissTimer = setTimeout(() => {
|
||||||
if (!this.active && this.failed === 0) this.jobs = [];
|
if (!this.active && !this.needsAttention) this.jobs = [];
|
||||||
}, DISMISS_AFTER_MS);
|
}, DISMISS_AFTER_MS);
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -169,26 +200,35 @@ export class UploadQueueState {
|
|||||||
const index = this.jobs.findIndex((job) => job.status === 'queued');
|
const index = this.jobs.findIndex((job) => job.status === 'queued');
|
||||||
if (index === -1) break;
|
if (index === -1) break;
|
||||||
|
|
||||||
|
const job = this.jobs[index];
|
||||||
this.#patch(index, { status: 'uploading' });
|
this.#patch(index, { status: 'uploading' });
|
||||||
|
|
||||||
try {
|
try {
|
||||||
const body = new FormData();
|
const body = new FormData();
|
||||||
for (const file of this.jobs[index].files) body.append('files', file);
|
for (const file of job.files) body.append('files', file);
|
||||||
|
|
||||||
const response = await fetch(
|
const response = await fetch(
|
||||||
`/api/books/fromFiles?library_id=${encodeURIComponent(String(this.jobs[index].libraryId))}`,
|
`/api/books/fromFiles?library_id=${encodeURIComponent(String(job.libraryId))}`,
|
||||||
{ method: 'POST', body }
|
{ method: 'POST', body }
|
||||||
);
|
);
|
||||||
|
|
||||||
if (!response.ok) throw new Error(`The server returned ${response.status}`);
|
if (!response.ok) throw new Error(`The server returned ${response.status}`);
|
||||||
|
|
||||||
const result: PaginatedResponse<Book> = await response.json();
|
const result: BooksUploadResult = await response.json();
|
||||||
const book = result.items[0];
|
const book = result.created[0];
|
||||||
|
|
||||||
created += result.total ?? result.items.length;
|
created += result.created.length;
|
||||||
firstBook ??= book;
|
firstBook ??= book;
|
||||||
|
|
||||||
this.#patch(index, { status: 'done', book });
|
// Nothing created means every file in this folder was already here.
|
||||||
|
// That is not a failure, and it is not something to hide either.
|
||||||
|
this.#patch(index, {
|
||||||
|
status: result.created.length === 0 ? 'skipped' : 'done',
|
||||||
|
book,
|
||||||
|
duplicates: result.skipped,
|
||||||
|
// One job is one book, so it has at most one set of candidates.
|
||||||
|
possibleDuplicates: result.possible_duplicates?.[0]?.candidates
|
||||||
|
});
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
// One bad book must not take the rest of the queue with it.
|
// One bad book must not take the rest of the queue with it.
|
||||||
console.error(`Failed to upload ${this.jobs[index].label}`, error);
|
console.error(`Failed to upload ${this.jobs[index].label}`, error);
|
||||||
@@ -205,9 +245,9 @@ export class UploadQueueState {
|
|||||||
|
|
||||||
if (created > 0) await invalidate('app:books');
|
if (created > 0) await invalidate('app:books');
|
||||||
|
|
||||||
if (this.failed === 0) this.#scheduleDismiss();
|
if (!this.needsAttention) this.#scheduleDismiss();
|
||||||
|
|
||||||
onFinished?.({ created, failed: this.failed, firstBook });
|
onFinished?.({ created, skipped: this.skipped, failed: this.failed, firstBook });
|
||||||
}
|
}
|
||||||
|
|
||||||
#patch(index: number, changes: Partial<UploadJob>) {
|
#patch(index: number, changes: Partial<UploadJob>) {
|
||||||
|
|||||||
@@ -1,9 +1,14 @@
|
|||||||
<script lang="ts">
|
<script lang="ts">
|
||||||
import { page } from '$app/state';
|
import { page } from '$app/state';
|
||||||
import { resolve } from '$app/paths';
|
import { resolve } from '$app/paths';
|
||||||
|
import { getLibraryState } from '$lib/state/library.svelte';
|
||||||
|
|
||||||
let { children } = $props();
|
let { children } = $props();
|
||||||
|
|
||||||
|
// Set in (root)/+layout.svelte, above this group, so listing the libraries in
|
||||||
|
// the nav needs no load function.
|
||||||
|
const libraryState = getLibraryState();
|
||||||
|
|
||||||
// Route ids rather than paths: resolve() is called in the markup so it stays a
|
// Route ids rather than paths: resolve() is called in the markup so it stays a
|
||||||
// direct call the lint rule can see, and the active check compares route ids.
|
// direct call the lint rule can see, and the active check compares route ids.
|
||||||
// A pathname comparison would miss during SSR, where resolve() returns a
|
// A pathname comparison would miss during SSR, where resolve() returns a
|
||||||
@@ -14,12 +19,28 @@
|
|||||||
{ title: 'Libraries', routeId: '/(root)/settings/libraries' },
|
{ title: 'Libraries', routeId: '/(root)/settings/libraries' },
|
||||||
{ title: 'Devices', routeId: '/(root)/settings/devices' }
|
{ title: 'Devices', routeId: '/(root)/settings/devices' }
|
||||||
] as const;
|
] as const;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Whether a nested library item is the one being looked at.
|
||||||
|
*
|
||||||
|
* Both halves are needed: the route id alone is shared by every library, so
|
||||||
|
* matching on it would light all of them up at once.
|
||||||
|
*/
|
||||||
|
function isActiveLibrary(id: number) {
|
||||||
|
return (
|
||||||
|
page.route.id?.startsWith('/(root)/settings/libraries/[libraryId]') === true &&
|
||||||
|
page.params.libraryId === String(id)
|
||||||
|
);
|
||||||
|
}
|
||||||
</script>
|
</script>
|
||||||
|
|
||||||
<div class="flex h-[calc(100vh-var(--header-height)-2rem)] flex-col">
|
<div class="flex h-[calc(100vh-var(--header-height)-2rem)] flex-col">
|
||||||
<h1 class="font-serif text-xl font-medium tracking-tight">Settings</h1>
|
<h1 class="font-serif text-xl font-medium tracking-tight">Settings</h1>
|
||||||
<div class="mt-6 flex flex-1 gap-8 overflow-hidden">
|
<div class="mt-6 flex flex-1 gap-8 overflow-hidden">
|
||||||
<nav class="flex w-48 shrink-0 flex-col gap-1">
|
<!-- Scrolls on its own now that it holds a list that grows with the number
|
||||||
|
of libraries; the row above it is overflow-hidden, so without this a
|
||||||
|
long list is clipped rather than reachable. -->
|
||||||
|
<nav class="flex w-48 shrink-0 flex-col gap-1 overflow-y-auto">
|
||||||
{#each items as item}
|
{#each items as item}
|
||||||
<a
|
<a
|
||||||
href={resolve(item.routeId)}
|
href={resolve(item.routeId)}
|
||||||
@@ -30,6 +51,26 @@
|
|||||||
>
|
>
|
||||||
{item.title}
|
{item.title}
|
||||||
</a>
|
</a>
|
||||||
|
|
||||||
|
<!-- Libraries is the one item with children: a library's own settings
|
||||||
|
hang off it, so every library and every section it has stay one
|
||||||
|
click from here. -->
|
||||||
|
{#if item.routeId === '/(root)/settings/libraries'}
|
||||||
|
{#each libraryState.libraries as library (library.id)}
|
||||||
|
<a
|
||||||
|
href={resolve('/(root)/settings/libraries/[libraryId]/duplicates', {
|
||||||
|
libraryId: String(library.id)
|
||||||
|
})}
|
||||||
|
class="ml-3 truncate rounded-md border-l px-3 py-1.5 text-sm transition-colors hover:bg-muted {isActiveLibrary(
|
||||||
|
library.id
|
||||||
|
)
|
||||||
|
? 'bg-muted font-medium'
|
||||||
|
: 'text-muted-foreground'}"
|
||||||
|
>
|
||||||
|
{library.name}
|
||||||
|
</a>
|
||||||
|
{/each}
|
||||||
|
{/if}
|
||||||
{/each}
|
{/each}
|
||||||
</nav>
|
</nav>
|
||||||
<div class="flex-1 overflow-auto">{@render children()}</div>
|
<div class="flex-1 overflow-auto">{@render children()}</div>
|
||||||
|
|||||||
@@ -0,0 +1,48 @@
|
|||||||
|
<script lang="ts">
|
||||||
|
import { page } from '$app/state';
|
||||||
|
import { resolve } from '$app/paths';
|
||||||
|
import { getLibraryState } from '$lib/state/library.svelte';
|
||||||
|
|
||||||
|
let { children } = $props();
|
||||||
|
|
||||||
|
// Set in (root)/+layout.svelte, above the settings group, so the libraries are
|
||||||
|
// already here — this pane needs no load function of its own.
|
||||||
|
const libraryState = getLibraryState();
|
||||||
|
|
||||||
|
const library = $derived(
|
||||||
|
libraryState.libraries.find((lib) => String(lib.id) === page.params.libraryId)
|
||||||
|
);
|
||||||
|
|
||||||
|
// Duplicates is the only section today. The strip exists so General and a
|
||||||
|
// danger zone have an obvious place to land; neither is built yet, and an
|
||||||
|
// empty tab is worse than no tab.
|
||||||
|
//
|
||||||
|
// Active state is matched on route id, not pathname, for the reason spelled
|
||||||
|
// out in settings/+layout.svelte.
|
||||||
|
const sections = [
|
||||||
|
{ title: 'Duplicates', routeId: '/(root)/settings/libraries/[libraryId]/duplicates' }
|
||||||
|
] as const;
|
||||||
|
</script>
|
||||||
|
|
||||||
|
<div class="flex flex-col gap-4">
|
||||||
|
<div>
|
||||||
|
<h2 class="text-lg font-semibold">{library?.name ?? 'Library'}</h2>
|
||||||
|
<p class="text-sm text-muted-foreground">Settings for this library</p>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<nav class="flex gap-1 border-b">
|
||||||
|
{#each sections as section (section.routeId)}
|
||||||
|
<a
|
||||||
|
href={resolve(section.routeId, { libraryId: page.params.libraryId! })}
|
||||||
|
class="-mb-px border-b-2 px-3 py-2 text-sm font-medium transition-colors hover:text-foreground {page
|
||||||
|
.route.id === section.routeId
|
||||||
|
? 'border-foreground'
|
||||||
|
: 'border-transparent text-muted-foreground'}"
|
||||||
|
>
|
||||||
|
{section.title}
|
||||||
|
</a>
|
||||||
|
{/each}
|
||||||
|
</nav>
|
||||||
|
|
||||||
|
{@render children()}
|
||||||
|
</div>
|
||||||
@@ -0,0 +1,10 @@
|
|||||||
|
import { resolve } from '$app/paths';
|
||||||
|
import { redirect } from '@sveltejs/kit';
|
||||||
|
|
||||||
|
export async function load({ params }) {
|
||||||
|
// The library pane is its sections; land on the first one.
|
||||||
|
redirect(
|
||||||
|
303,
|
||||||
|
resolve('/(root)/settings/libraries/[libraryId]/duplicates', { libraryId: params.libraryId })
|
||||||
|
);
|
||||||
|
}
|
||||||
@@ -0,0 +1,18 @@
|
|||||||
|
import { listBooks, listDuplicateBooks } from '$lib/api/book.remote.js';
|
||||||
|
|
||||||
|
export async function load({ params, depends }) {
|
||||||
|
// Dismissing or merging a group re-runs this, so the card leaves the screen.
|
||||||
|
depends('app:duplicate-books');
|
||||||
|
|
||||||
|
const groups = await listDuplicateBooks(params.libraryId);
|
||||||
|
|
||||||
|
// The groups carry only enough to render a card. Merging needs the whole record —
|
||||||
|
// identifiers, description, publisher — so fetch them in one go rather than per
|
||||||
|
// card, and let the dialog pick out the books for its own group.
|
||||||
|
const ids = [...new Set(groups.flatMap((group) => group.books.map((book) => book.book_id)))];
|
||||||
|
const books = ids.length
|
||||||
|
? await listBooks({ ids, pageSize: ids.length })
|
||||||
|
: { items: [] };
|
||||||
|
|
||||||
|
return { groups, books: books.items };
|
||||||
|
}
|
||||||
@@ -0,0 +1,249 @@
|
|||||||
|
<script lang="ts">
|
||||||
|
import { page } from '$app/state';
|
||||||
|
import { resolve } from '$app/paths';
|
||||||
|
import { invalidate } from '$app/navigation';
|
||||||
|
import { toast } from 'svelte-sonner';
|
||||||
|
import { CopyCheck, Fingerprint, GitMerge, SlidersHorizontal, Type } from '@lucide/svelte';
|
||||||
|
|
||||||
|
import * as Card from '$lib/components/ui/card/index.js';
|
||||||
|
import * as Empty from '$lib/components/ui/empty/index.js';
|
||||||
|
import * as DropdownMenu from '$lib/components/ui/dropdown-menu/index.js';
|
||||||
|
import { Badge } from '$lib/components/ui/badge/index.js';
|
||||||
|
import { Button, buttonVariants } from '$lib/components/ui/button/index.js';
|
||||||
|
import BookImage from '$lib/components/view/book-image.svelte';
|
||||||
|
import GeneratedCover from '$lib/components/view/generated-cover.svelte';
|
||||||
|
import { dismissDuplicateBooks } from '$lib/api/book.remote';
|
||||||
|
import MergeBooks from '$lib/components/forms/merge-books/merge-books.svelte';
|
||||||
|
import { mergeInto } from '$lib/components/forms/merge-books/merge';
|
||||||
|
import type { Book, DuplicateBookGroup } from '$lib/schema';
|
||||||
|
|
||||||
|
let { data }: { data: { groups: DuplicateBookGroup[]; books: Book[] } } = $props();
|
||||||
|
|
||||||
|
// The group open in the workbench, or null when it is closed.
|
||||||
|
let merging = $state<DuplicateBookGroup | null>(null);
|
||||||
|
|
||||||
|
/** The full records behind a group, in the order the group lists them. */
|
||||||
|
function recordsFor(group: DuplicateBookGroup): Book[] {
|
||||||
|
return group.books
|
||||||
|
.map(({ book_id }) => data.books.find((book) => book.id === book_id))
|
||||||
|
.filter((book): book is Book => book !== undefined);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Which groups have a request in flight — a dismissal or a quick merge — so a
|
||||||
|
// slow round trip cannot be started twice.
|
||||||
|
let dismissing = $state<number[]>([]);
|
||||||
|
let quickMerging = $state<number[]>([]);
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Merge without opening the workbench, keeping the first book as it stands.
|
||||||
|
*
|
||||||
|
* The common case by far: the group is the same book twice, one record is as
|
||||||
|
* good as the other, and the point is to end up with one. Resolving metadata
|
||||||
|
* field by field is the other button.
|
||||||
|
*/
|
||||||
|
async function quickMerge(group: DuplicateBookGroup) {
|
||||||
|
const key = keyOf(group);
|
||||||
|
if (quickMerging.includes(key)) return;
|
||||||
|
|
||||||
|
const [survivor, ...folded] = recordsFor(group);
|
||||||
|
if (!survivor || folded.length === 0) return;
|
||||||
|
|
||||||
|
quickMerging = [...quickMerging, key];
|
||||||
|
|
||||||
|
try {
|
||||||
|
await mergeInto(page.params.libraryId!, survivor, folded);
|
||||||
|
} finally {
|
||||||
|
quickMerging = quickMerging.filter((id) => id !== key);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** A group is named by its lowest book id, which the backend orders it by. */
|
||||||
|
function keyOf(group: DuplicateBookGroup) {
|
||||||
|
return group.books[0].book_id;
|
||||||
|
}
|
||||||
|
|
||||||
|
function reasonLabel(reason: string) {
|
||||||
|
if (reason === 'identifier') return 'Same identifier';
|
||||||
|
if (reason === 'title-author') return 'Same title and author';
|
||||||
|
return reason;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Dismiss every pairing in a group at once.
|
||||||
|
*
|
||||||
|
* A group is held together pair by pair, so saying "these are not duplicates"
|
||||||
|
* about three books means saying it about all three pairs — dismissing only the
|
||||||
|
* first would leave the rest of the group standing and the screen unchanged.
|
||||||
|
*/
|
||||||
|
async function notDuplicates(group: DuplicateBookGroup) {
|
||||||
|
const key = keyOf(group);
|
||||||
|
if (dismissing.includes(key)) return;
|
||||||
|
|
||||||
|
dismissing = [...dismissing, key];
|
||||||
|
|
||||||
|
const ids = group.books.map((book) => book.book_id);
|
||||||
|
const pairs = ids.flatMap((a, index) =>
|
||||||
|
ids.slice(index + 1).map((b) => ({ book_a_id: a, book_b_id: b }))
|
||||||
|
);
|
||||||
|
|
||||||
|
try {
|
||||||
|
await Promise.all(pairs.map((pair) => dismissDuplicateBooks(pair)));
|
||||||
|
await invalidate('app:duplicate-books');
|
||||||
|
toast.success('Marked as different books');
|
||||||
|
} catch {
|
||||||
|
toast.error('Could not mark these as different books');
|
||||||
|
} finally {
|
||||||
|
dismissing = dismissing.filter((id) => id !== key);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
</script>
|
||||||
|
|
||||||
|
<!-- No max width and no padding of its own: the settings pane sets the width and
|
||||||
|
is the thing that scrolls, so a wrapper that centres inside it only makes the
|
||||||
|
cards narrower than they need to be. -->
|
||||||
|
<div class="flex flex-col gap-6 pb-4">
|
||||||
|
<p class="text-sm text-muted-foreground">
|
||||||
|
Books whose metadata matches. A second edition, a translation and a different scan of one book
|
||||||
|
all look like this, so nothing here has been changed or removed — this is a list to read, not a
|
||||||
|
problem to fix.
|
||||||
|
</p>
|
||||||
|
|
||||||
|
{#if data.groups.length === 0}
|
||||||
|
<Empty.Root>
|
||||||
|
<Empty.Header>
|
||||||
|
<Empty.Media variant="icon">
|
||||||
|
<CopyCheck />
|
||||||
|
</Empty.Media>
|
||||||
|
<Empty.Title>Nothing looks duplicated</Empty.Title>
|
||||||
|
<Empty.Description>
|
||||||
|
No two books in this library share an identifier, or a title and an author.
|
||||||
|
</Empty.Description>
|
||||||
|
</Empty.Header>
|
||||||
|
</Empty.Root>
|
||||||
|
{:else}
|
||||||
|
{#each data.groups as group (keyOf(group))}
|
||||||
|
{@const busy = dismissing.includes(keyOf(group)) || quickMerging.includes(keyOf(group))}
|
||||||
|
{@const records = recordsFor(group)}
|
||||||
|
<Card.Root>
|
||||||
|
<Card.Header>
|
||||||
|
<Card.Title>{group.books.length} books look like the same book</Card.Title>
|
||||||
|
<Card.Action class="flex gap-2">
|
||||||
|
<Button
|
||||||
|
variant="outline"
|
||||||
|
size="sm"
|
||||||
|
disabled={busy}
|
||||||
|
onclick={() => notDuplicates(group)}
|
||||||
|
>
|
||||||
|
Not duplicates
|
||||||
|
</Button>
|
||||||
|
<!-- A choice rather than a jump straight into the workbench:
|
||||||
|
most groups are one book twice, where there is nothing to
|
||||||
|
resolve and the dialog is a step in the way.
|
||||||
|
|
||||||
|
Disabled until every record in the group came back, so
|
||||||
|
neither action can run on a partial group. -->
|
||||||
|
<DropdownMenu.Root>
|
||||||
|
<DropdownMenu.Trigger
|
||||||
|
class={buttonVariants({ size: 'sm' })}
|
||||||
|
disabled={busy || records.length !== group.books.length}
|
||||||
|
>
|
||||||
|
Merge…
|
||||||
|
</DropdownMenu.Trigger>
|
||||||
|
<DropdownMenu.Content align="end">
|
||||||
|
<DropdownMenu.Item onclick={() => quickMerge(group)}>
|
||||||
|
<GitMerge class="size-4 shrink-0" />
|
||||||
|
Merge into first selected
|
||||||
|
</DropdownMenu.Item>
|
||||||
|
<DropdownMenu.Item onclick={() => (merging = group)}>
|
||||||
|
<SlidersHorizontal class="size-4 shrink-0" />
|
||||||
|
Manual merge…
|
||||||
|
</DropdownMenu.Item>
|
||||||
|
</DropdownMenu.Content>
|
||||||
|
</DropdownMenu.Root>
|
||||||
|
</Card.Action>
|
||||||
|
</Card.Header>
|
||||||
|
|
||||||
|
<Card.Content>
|
||||||
|
<ul class="flex flex-wrap gap-4">
|
||||||
|
{#each group.books as book (book.book_id)}
|
||||||
|
<li class="w-36">
|
||||||
|
<a
|
||||||
|
href={resolve('/(root)/(library)/book/[bookId]', {
|
||||||
|
bookId: String(book.book_id)
|
||||||
|
})}
|
||||||
|
class="group flex flex-col gap-2"
|
||||||
|
>
|
||||||
|
<span
|
||||||
|
class="block aspect-9/12 w-full overflow-hidden rounded-sm bg-muted shadow-md transition-all group-hover:brightness-75"
|
||||||
|
>
|
||||||
|
{#if book.cover_image}
|
||||||
|
<BookImage src="/api/{book.cover_image}" class="h-full w-full object-cover" />
|
||||||
|
{:else}
|
||||||
|
<!-- A blank swatch here is worse than anywhere else: this screen
|
||||||
|
is a side-by-side comparison, and two of them are impossible
|
||||||
|
to tell apart. The candidate carries author names rather than
|
||||||
|
records, which is all the cover draws. -->
|
||||||
|
<GeneratedCover
|
||||||
|
book={{
|
||||||
|
title: book.title,
|
||||||
|
authors: book.authors.map((name) => ({ name }))
|
||||||
|
}}
|
||||||
|
/>
|
||||||
|
{/if}
|
||||||
|
</span>
|
||||||
|
|
||||||
|
<span class="line-clamp-2 font-serif text-sm group-hover:underline">
|
||||||
|
{book.title}
|
||||||
|
</span>
|
||||||
|
</a>
|
||||||
|
|
||||||
|
<p class="line-clamp-2 text-xs text-muted-foreground">
|
||||||
|
{book.authors.join(', ')}
|
||||||
|
</p>
|
||||||
|
|
||||||
|
<!-- Why this book is in the group, so the reader can judge the
|
||||||
|
evidence rather than take the grouping on trust. -->
|
||||||
|
<p class="mt-1 flex flex-wrap gap-1">
|
||||||
|
{#each book.matched_on as reason (reason)}
|
||||||
|
<Badge variant="secondary" class="gap-1 text-[10px] font-normal">
|
||||||
|
{#if reason === 'identifier'}
|
||||||
|
<Fingerprint class="size-3" />
|
||||||
|
{:else}
|
||||||
|
<Type class="size-3" />
|
||||||
|
{/if}
|
||||||
|
{reasonLabel(reason)}
|
||||||
|
</Badge>
|
||||||
|
{/each}
|
||||||
|
</p>
|
||||||
|
</li>
|
||||||
|
{/each}
|
||||||
|
</ul>
|
||||||
|
</Card.Content>
|
||||||
|
</Card.Root>
|
||||||
|
{/each}
|
||||||
|
{/if}
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!--
|
||||||
|
Keyed on the group so a second merge starts from that group's records rather
|
||||||
|
than the previous one's draft, the same way the edit dialog keys on the book.
|
||||||
|
-->
|
||||||
|
{#if merging}
|
||||||
|
{#key merging.books[0].book_id}
|
||||||
|
<MergeBooks
|
||||||
|
books={recordsFor(merging)}
|
||||||
|
libraryId={page.params.libraryId!}
|
||||||
|
bind:open={
|
||||||
|
() => merging !== null,
|
||||||
|
(value) => {
|
||||||
|
// Bound, not passed as a bare `true`: the dialog closes itself on an
|
||||||
|
// outside click or Escape, and if that never reaches `merging` the
|
||||||
|
// group stays set — the menu item then re-assigns the same group,
|
||||||
|
// nothing changes, and the workbench never reopens.
|
||||||
|
if (!value) merging = null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
onmerged={() => (merging = null)}
|
||||||
|
/>
|
||||||
|
{/key}
|
||||||
|
{/if}
|
||||||
Reference in New Issue
Block a user