immich/server/src/interfaces/storage.interface.ts

55 lines
2 KiB
TypeScript
Raw Normal View History

2024-02-13 08:48:47 -05:00
import { WatchOptions } from 'chokidar';
import { Stats } from 'node:fs';
import { FileReadOptions } from 'node:fs/promises';
import { Readable } from 'node:stream';
fix(server): don't crash when refreshing large libraries (#7934) * add job to check for offline files * fix lint * only check for offline when using checkForOffline * improve tests * remove old test * wip * remove trie * refactor batches * also check offline status * fix spelling * don't do offline scan * rename scan to check * fix job statuses * fix lint * cleanup * add test * open-api * fix test * fix spinner * reset text * don't double batch * fix comments from mert * remove tries * fix tests * fix e2e * fix test * fix test * add tests * fix lint * fix e2e * interweave scans * fix errors * fix messages * fix test * add mock * fix sql * fix e2e * use library batch size * save -> update * add file extensions * update specs * test for import paths * check import paths when testing offline * fix lint * normalize import path * remove console logs * decrease batch size to 1000 * add test for import path * add test for already-online assets * fix merge * fix lint * add library job back * add offline job to correct queue * library spec compiles now * move one test to new e2e * fix comments * fix comments * fix lint * refactor path validation * fix loop bug * remove logging * expect responses * fix asset mock * take the straightforward approach * use generator correctly * fix vitest on file edit * bump vitest to 1.6.0 * test for offline check * add e2e tests for offlining assets depending on import path * cleanup e2e test after finish * cleanup library service * paginate the walk generator * fix tests * fix typo * refactoring handleOfflineCheck * better testing of handleOfflineCheck * fix lint * handle large library deletions * dont check if library is deleted * fix mock * add a 100k page size to library * fix loading animation * better log messages * Better logging for offline asset removal * fix sql and tests * fix number format * Remove submodule * fix format * chore: cleanup * chore: fix tests --------- Co-authored-by: Alex <alex.tran1502@gmail.com> Co-authored-by: Jason Rasmussen <jason@rasm.me>
2024-08-28 19:05:48 +02:00
import { CrawlOptionsDto, WalkOptionsDto } from 'src/dtos/library.dto';
export interface ImmichReadStream {
stream: Readable;
type?: string;
length?: number;
}
export interface ImmichZipStream extends ImmichReadStream {
addFile: (inputPath: string, filename: string) => void;
finalize: () => Promise<void>;
}
2023-03-21 22:49:19 -04:00
export interface DiskUsage {
available: number;
free: number;
total: number;
}
export const IStorageRepository = 'IStorageRepository';
2024-02-13 08:48:47 -05:00
export interface WatchEvents {
onReady(): void;
onAdd(path: string): void;
onChange(path: string): void;
onUnlink(path: string): void;
onError(error: Error): void;
}
feat(server): Automatic watching of library folders (#6192) * feat: initial watch support * allow offline files * chore: ignore query errors when resetting e2e db * revert db query * add savepoint * guard the user query * chore: openapi and db migration * wip * support multiple libraries * fix tests * wip * can now cleanup chokidar watchers * fix unit tests * add library watch queue * add missing init from merge * wip * can now filter file extensions * remove watch api from non job client * Fix e2e test * watch library with updated import path and exclusion pattern * add library watch frontend ui * case sensitive watching extensions * can auto watch libraries * move watcher e2e tests to separate file * don't watch libraries from a queue * use event emitters * shorten e2e test timeout * refactor chokidar code to filesystem provider * expose chokidar parameters to config file * fix storage mock * set default config for library watching * add fs provider mocks * cleanup * add more unit tests for watcher * chore: fix format + sql * add more tests * move unwatch feature back to library service * add file event unit tests * chore: formatting * add documentation * fix e2e tests * chore: fix e2e tests * fix library updating * test cleanup * fix typo * cleanup * fixing as per pr comments * reduce library watch config file * update storage config and mocks * move negative event tests to unit tests * fix library watcher e2e * make watch configuration global * remove the feature flag * refactor watcher teardown * fix microservices init * centralize asset scan job queue * improve docs * add more tests * chore: open api * initialize app service * fix docs * fix library watch feature flag * Update docs/docs/features/libraries.md Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com> * fix: import right app service * don't be truthy * fix test speling * stricter library update tests * move fs watcher mock to external file * subscribe to config changes * docker does not need polling * make library watch() private * feat: add configuration ui --------- Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com> Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2024-01-31 09:15:54 +01:00
export interface IStorageRepository {
createZipStream(): ImmichZipStream;
createReadStream(filepath: string, mimeType?: string | null): Promise<ImmichReadStream>;
readFile(filepath: string, options?: FileReadOptions<Buffer>): Promise<Buffer>;
writeFile(filepath: string, buffer: Buffer): Promise<void>;
realpath(filepath: string): Promise<string>;
unlink(filepath: string): Promise<void>;
unlinkDir(folder: string, options?: { recursive?: boolean; force?: boolean }): Promise<void>;
removeEmptyDirs(folder: string, self?: boolean): Promise<void>;
checkFileExists(filepath: string, mode?: number): Promise<boolean>;
mkdirSync(filepath: string): void;
2023-03-21 22:49:19 -04:00
checkDiskUsage(folder: string): Promise<DiskUsage>;
readdir(folder: string): Promise<string[]>;
stat(filepath: string): Promise<Stats>;
fix(server): don't crash when refreshing large libraries (#7934) * add job to check for offline files * fix lint * only check for offline when using checkForOffline * improve tests * remove old test * wip * remove trie * refactor batches * also check offline status * fix spelling * don't do offline scan * rename scan to check * fix job statuses * fix lint * cleanup * add test * open-api * fix test * fix spinner * reset text * don't double batch * fix comments from mert * remove tries * fix tests * fix e2e * fix test * fix test * add tests * fix lint * fix e2e * interweave scans * fix errors * fix messages * fix test * add mock * fix sql * fix e2e * use library batch size * save -> update * add file extensions * update specs * test for import paths * check import paths when testing offline * fix lint * normalize import path * remove console logs * decrease batch size to 1000 * add test for import path * add test for already-online assets * fix merge * fix lint * add library job back * add offline job to correct queue * library spec compiles now * move one test to new e2e * fix comments * fix comments * fix lint * refactor path validation * fix loop bug * remove logging * expect responses * fix asset mock * take the straightforward approach * use generator correctly * fix vitest on file edit * bump vitest to 1.6.0 * test for offline check * add e2e tests for offlining assets depending on import path * cleanup e2e test after finish * cleanup library service * paginate the walk generator * fix tests * fix typo * refactoring handleOfflineCheck * better testing of handleOfflineCheck * fix lint * handle large library deletions * dont check if library is deleted * fix mock * add a 100k page size to library * fix loading animation * better log messages * Better logging for offline asset removal * fix sql and tests * fix number format * Remove submodule * fix format * chore: cleanup * chore: fix tests --------- Co-authored-by: Alex <alex.tran1502@gmail.com> Co-authored-by: Jason Rasmussen <jason@rasm.me>
2024-08-28 19:05:48 +02:00
crawl(options: CrawlOptionsDto): Promise<string[]>;
walk(options: WalkOptionsDto): AsyncGenerator<string[]>;
copyFile(source: string, target: string): Promise<void>;
rename(source: string, target: string): Promise<void>;
watch(paths: string[], options: WatchOptions, events: Partial<WatchEvents>): () => Promise<void>;
utimes(filepath: string, atime: Date, mtime: Date): Promise<void>;
}