Commit graph

29 commits

Author SHA1 Message Date
Phlogi
8c6adf7157
feat(server): resolve duplicates (#25316)
* feat(web): Synchronize information from deduplicated images

* Added new settings menu to the the deduplication tab.
* The toggable options in the settings are synchronization of: albums, favorites, ratings, description, visibility and location.
* When synchronizing the albums, the resolved images will be added to all albums of the duplicates.
* When synchronizing the favorite status, the resolved images will be marked as favorite, if at least one selectable image is marked as favorite.
* When synchronizing the ratings, the highest rating from the selectable images will be applied to the resolved image.
* When synchronizing the description, all descriptions from the selectable images will be merged into one description for the resolved image.
* When synchronizing the visibility, the most restrictive visibility setting from the selectable images will be applied to the resolved image.
* When synchronizing the location, if exactly one unique location exists among the selectable images, this location will be applied to the resolved image.
* There is no additional UI for these settings to keep the visual clutter minimal. The settings are applied automatically based on the user's preferences.

* Replace addAssetToAlbums with copyAsset

* fix linter

* feat(web): add duplicate sync fields and fix typo

* feat(web): add tag sync and enhance duplicate resolution

This update introduces tag synchronization for duplicate resolution,
ensuring all unique tag IDs from duplicates are applied to kept assets.
The visibility sync logic is updated to use a simplified ordering, as the hidden status items will never show up in a duplicate set.
Album synchronization now merges albums directly via addAssetsToAlbums; as the approach with copyAsset API endpoint was ineffiecient.
Description, rating, and location sync logic is improved for correctness.
and deduplication. i18n strings were added / updated.

* feat(server): move duplicate resolution to backend with sync and stacking

Moves duplicate metadata synchronization from frontend to backend, enabling robust
batch operations and proper validation. This is an improved refactor of PR #13851.

New endpoints:
- POST /duplicates/resolve - batch resolve with configurable metadata sync
- POST /duplicates/stack - create stacks from duplicate groups
- GET /duplicates - now includes suggestedKeepAssetIds based on file size and EXIF

Key changes:
- Move sync logic (albums, tags, favorites, ratings, descriptions, location, visibility) to server
- Add server-side metadata merge policies with proper conflict resolution
- Replace client-side resolution logic with new backend endpoints
- Add comprehensive E2E tests (70+ test cases) and unit tests
- Update OpenAPI specs and TypeScript SDK

No breaking changes - only additions to existing API.

* feat(preferences): enable all duplicate sync settings by default

* chore: clean up

* chore: clean up

* refactor: rename & clean up

* fix: preference upgrade

* chore: linting

* refactor(e2e): use updateAssets API for setAssetDuplicateId

* fix: visibility sync logic in duplicate resolution

* fix(duplicate): write description to exifUpdate

Previously the duplicate resolution populated assetUpdate.description even
though description belongs to exif info.

* fix(duplicate): remove redundant updateLockedColumns wrapper

updateAllExif already computes lockedProperties via distinctLocked
using Object.keys(options). The wrapper added a lockedProperties key
to the options object, causing the spurious string 'lockedProperties'
to be stored in the lockedProperties array.

* fix(duplicate): write merged tags to asset_exif to survive metadata re-extraction

During duplicate resolution, replaceAssetTags correctly wrote merged tag
IDs to the tag_asset table, but never updated asset_exif.tags or locked
the tags property. The subsequent SidecarWrite → AssetExtractMetadata
chain calls applyTagList, which destructively replaces tag_asset rows
with whatever is in asset_exif.tags — still the original per-asset tags,
not the merged set.

Write merged tag values to asset_exif.tags via updateAllExif (which also
locks the property via distinctLocked), and queue SidecarWrite when tags
change so they persist to the sidecar file.

* docs(duplicates): clarify location and tag sync behavior

* refactor(duplicate): remove sync settings, always sync all metadata on resolve

Remove DuplicateSyncSettingsDto and the per-field sync toggles
(albums, favorites, rating, description, visibility, location, tags).
Duplicate resolution now unconditionally syncs all metadata from
trashed assets to kept assets.

- Remove DuplicateSyncSettingsDto and settings field from DuplicateResolveDto
- Update DuplicateService to always run all sync logic without conditionals
- Delete DuplicateSettingsModal.svelte and settings gear button from UI
- Remove DuplicateSettings type and duplicateSettings persisted store
- Update unit and e2e tests to remove settings from resolve requests

* docs: update duplicates utility to reflect automatic metadata sync

* docs(web): replace duplicates info modal with link to documentation

* chore: clean up

* fix: add missing type cast to jsonAgg in duplicate repository getAll

* fix: skip persisting rating=0 in duplicate merge to avoid unnecessary sidecar write

---------

Co-authored-by: Toni <51962051+EinToni@users.noreply.github.com>
Co-authored-by: Jason Rasmussen <jason@rasm.me>
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2026-03-26 18:33:55 +00:00
Daniel Dietzler
34ce68095d
chore: upgrade to kysely 0.28.11 (#26744) 2026-03-11 16:17:31 +01:00
Daniel Dietzler
7cb355279e
chore: remove asset stubs (#26187) 2026-02-13 11:00:31 -05:00
Daniel Dietzler
7e0356e227
refactor: more small tests (#26159) 2026-02-12 08:34:32 -05:00
Daniel Dietzler
e54678e0d6
refactor: small tests (#26141) 2026-02-11 11:49:00 -05:00
Jason Rasmussen
bcb968e3d1
refactor: job names (#19949) 2025-07-15 18:39:00 -04:00
Jason Rasmussen
e73abe0762
refactor: enum casing (#19946) 2025-07-15 14:50:13 -04:00
Jason Rasmussen
910661e75c
chore: remove unused mocks (#19299) 2025-06-19 10:35:09 -05:00
Jason Rasmussen
5cd186d3d4
refactor: duplicate queries (#19136) 2025-06-12 14:23:02 -04:00
Mert
a6a4dfcfd3
fix(server): queueing for duplicate detection (#18380)
* fix queueing

* update tests
2025-05-20 08:44:39 -05:00
Alex
d33ce13561
feat(server): visibility column (#17939)
* feat: private view

* pr feedback

* sql generation

* feat: visibility column

* fix: set visibility value as the same as the still part after unlinked live photos

* fix: test

* pr feedback
2025-05-06 12:12:48 -05:00
Jason Rasmussen
8c5116bc1d
refactor: stream search duplicates (#17991) 2025-04-30 10:40:32 -04:00
Jason Rasmussen
17e720440d
refactor: new asset-job repository (#17622)
* refactor: new asset-job repository

* fix: broken medium tests on main
2025-04-15 10:24:51 -04:00
Jason Rasmussen
fdbe6d649f
refactor: remove smart search entity (#17447)
refactor: smart search entity
2025-04-08 09:56:45 -04:00
Mert
a708649504
fix(server): skip stacked assets in duplicate detection (#16380)
* skip stacked assets in duplicate detection

* update sql

* handle stacking after duplicate detection runs
2025-02-27 10:16:13 -06:00
Jason Rasmussen
fa5aeaf539
refactor: last repository (#16042) 2025-02-11 22:15:56 +00:00
Jason Rasmussen
9d85272c2b
refactor: repositories (#16036) 2025-02-11 14:08:13 -05:00
Jason Rasmussen
735f8d661e
refactor: test mocks (#16008) 2025-02-11 00:47:42 +01:00
Jason Rasmussen
03948a69e2
refactor: system metadata repository (#15954) 2025-02-07 17:26:49 -05:00
Jason Rasmussen
d3446f3092
refactor: logging repository (#15540) 2025-01-23 08:31:30 -05:00
Mert
2e12c46980
refactor(server): use kysely (#12857) 2025-01-09 11:15:41 -05:00
Pranay Pandey
380fc06979
fix: remove duplicateIds on unique assets (#13752) 2024-11-04 10:03:03 -05:00
Daniel Dietzler
9d0f03808c
chore: finishing unit tests for a couple of services (#13292) 2024-10-08 17:08:49 -04:00
Jason Rasmussen
4ea281f854
refactor: service dependencies (#13108)
refactor(server): simplify service dependency management
2024-10-02 10:54:35 -04:00
Jason Rasmussen
63437529e1
refactor(server): config file env (#13100) 2024-10-01 16:03:55 -04:00
bo0tzz
e54c18367b
chore: Lower default duplicate detection distance (#10703) 2024-06-30 11:36:02 -04:00
Alex
6879bcb7a4
chore(server): duplication default settings (#9781) 2024-05-26 20:51:41 -04:00
Mert
e7c8501930
fix(server): search duplicates of the same asset type (#9747)
* search by type

* make sql

---------

Co-authored-by: Alex <alex.tran1502@gmail.com>
2024-05-26 22:04:23 +00:00
Mert
ff52300624
refactor(server): duplicate controller and service (#9542)
* duplicate controller and service

* change endpoint name

* fix search tests

* remove unused import

* add to index
2024-05-16 19:39:33 -04:00