Commit Graph

269 Commits

Author SHA1 Message Date
WithoutPants
046fd1c0be Blob fixes (#3599)
* Fix error if movie back image blob was not found
* Don't error out if scene cover get fails
* Don't error out on image get fails
* Add debug logging for fs blobs
* Remove old blob data when no longer referenced
2023-03-26 10:56:32 +11:00
WithoutPants
5a41001246 Fix batch performer tagging with multiple endpoints (#3548)
* Set stash ids correctly during performer batch add
* Refactor performer tagger dialogs
2023-03-17 15:07:53 +11:00
WithoutPants
7cff71c35f Add filesystem based blob storage (#3187)
* Refactor transaction hooks. Add preCommit
* Add BlobStore
* Use blobStore for tag images
* Use blobStore for studio images
* Use blobStore for performer images
* Use blobStore for scene covers
* Don't generate screenshots in legacy directory
* Run post-hooks outside original transaction
* Use blobStore for movie images
* Remove unnecessary DestroyImage methods
* Add missing filter for scene cover
* Add covers to generate options
* Add generate cover option to UI
* Add screenshot migration
* Delete thumb files as part of screenshot migration
2023-03-17 10:52:49 +11:00
yoshnopa
7e8f941155 Add Chapters for Galleries (#3289)
Co-authored-by: WithoutPants <53250216+WithoutPants@users.noreply.github.com>
2023-03-16 15:04:54 +11:00
WithoutPants
58852f86fe Handle large and all entity queries (#3544)
* Remove upper page size limit
* Batch GetMany function
* Remove upper query limit from UI
2023-03-16 09:08:21 +11:00
DingDongSoLong4
51469cfc7f Optimize allData queries (#3452)
* Add specific fields to allData queries
* Add additional allData endpoints
2023-02-20 09:24:47 +11:00
DingDongSoLong4
32e8496314 Add studio performer count (#3362)
* Add studio performer count

* Add mocks
2023-01-28 19:12:47 -05:00
HijackHornet
0b4b100ecc [Feature] Images new fields : URL & Date (#3015)
Co-authored-by: WithoutPants <53250216+WithoutPants@users.noreply.github.com>
2022-12-20 11:13:41 +11:00
WithoutPants
4daf0a14a2 Performer disambiguation and aliases (#3113)
* Refactor performer relationships
* Remove checksum from performer
* Add disambiguation, overhaul aliases
* Add disambiguation filter criterion
* Improve name matching during import
* Add disambiguation filtering in UI
* Include aliases in performer select
2022-12-01 13:54:08 +11:00
CJ
0664c5b974 Track watch activity for scenes. (#3055)
* track watchtime and view time
* add view count sorting, added continue position filter
* display metrics in file info
* add toggle for tracking activity
* save activity every 10 seconds
* reset resume when video is nearly complete
* start from beginning when playing scene in queue

Co-authored-by: WithoutPants <53250216+WithoutPants@users.noreply.github.com>
2022-11-21 12:55:15 +11:00
WithoutPants
f39fa416a9 Fix database locked errors (#3153)
* Make read-only operations use WithReadTxn
* Allow one database write thread
* Add unit test for concurrent transactions
* Perform some actions after commit to release txn
* Suppress some errors from cancelled context
2022-11-21 06:49:10 +11:00
stg-annon
3660bf2d1a Support filtering by StashID endpoint (#3005)
* Add endpoint to stash_id filter in UI

Co-authored-by: WithoutPants <53250216+WithoutPants@users.noreply.github.com>
2022-11-17 10:08:15 +11:00
skier233
7eae751d1c Stash rating system (#2830)
* add rating100 fields to represent rating range 1-100
* deprecate existing (1-5) rating fields
* add half- and quarter-star options for rating system
* add decimal rating system option

Co-authored-by: WithoutPants <53250216+WithoutPants@users.noreply.github.com>
2022-11-16 09:31:44 +11:00
gitgiggety
f66333bac9 Add date filters (#2834)
* graphql: support date and timestamp filter types
* sql: add support for date & timestamp criterions
* ui: add support for date and timestamp criterions
* scenes: add support for filtering by date, created at and updated at
* image: support filtering by created at and updated at
* gallery: support filtering by date, created at and updated at
* movie: support filtering by date, created at and updated at
* studio: support filtering by date, created at and updated at
* tag: support filtering by date, created at and updated at
* performer: support filtering by bitrh & death date and created & updated at
* marker: support filtering by created & updated at and scene date, created & updated at
2022-11-15 11:52:05 +11:00
WithoutPants
ce17230c13 Refactor autotag to use individual transactions (#3106)
* Add id filtering to scenes, images, and galleries
* Perform tagging in batches
* One transaction per object tagged
2022-11-14 17:07:24 +11:00
WithoutPants
4a054ab081 Support file-less scenes. Add scene split, merge and reassign file (#3006)
* Reassign scene file functionality
* Implement scene create
* Add scene create UI
* Add sceneMerge backend support
* Add merge scene to UI
* Populate split create with scene details
* Add merge button to duplicate checker
* Handle file-less scenes in marker preview generate
* Make unique file name for file-less scene exports
* Add o-counter to scene update input
* Hide rescan for file-less scenes
* Generate heatmap if no speed set on file
* Fix count in scene/image queries
2022-11-14 16:35:09 +11:00
WithoutPants
9df66024d1 Make title optional for non-user created galleries (#3110) 2022-11-10 14:19:13 +11:00
WithoutPants
d2743cf5fb Change performer height to be numeric (#3060)
* Make height an int. Add height_cm field
* Change UI to use height_cm
* Use number fields for height/weight
* Add migration note
2022-11-08 14:09:03 +11:00
HappyAxolotl
eff86bf2f8 [Feature] Add fields director and (studio) code to scenes (#3051)
* added schema migration and updated data models
* added code and director to UI
* new fields are exported and imported
* added filters
* Add changelog entry
2022-11-07 18:16:52 +11:00
WithoutPants
270bc317cb Performer refactor (#3057)
* Separate performer model from sqlite model
* Use GenderEnum for gender
2022-10-31 14:58:01 +11:00
stg-annon
a60afc162f Add Tag description filter (#3011)
* init tag description filter

issue with rendering UI filter dialog

* move description to string block
* Add 0.18.0 changelog page
2022-10-26 18:06:52 +11:00
WithoutPants
47395ce13f Use basename as title if empty when scraping by fragment (#3040)
* Fallback to file basename if title empty in scrape
* Populate dialog from basename if title empty
2022-10-24 14:36:22 +11:00
WithoutPants
ef9e138a2d [Files Refactor] Object file management (#2790)
* Add Make Primary file function
* Add delete file functionality
2022-10-06 14:50:06 +11:00
pickleahead
4c73f2f845 Add descriptions to tags and display tag cards on hover (#2708)
* add descriptions to tags
* display tag description and tag image on hover

Co-authored-by: WithoutPants <53250216+WithoutPants@users.noreply.github.com>
2022-10-06 10:01:06 +11:00
WithoutPants
dce90a3ed9 Handle file rescan (#2951)
* Fire handlers when file updated or moved
* Create galleries as needed
* Clean empty galleries
* Handle cleaning zip folders when path changed
* Fix gallery association on duplicate images
* Re-create missing folder-based galleries
2022-09-28 16:08:00 +10:00
WithoutPants
98e3610ade Fix gallery checksum resolver error (#2929) 2022-09-19 16:50:04 +10:00
WithoutPants
9e08edc76f [Files Refactor] Don't require fingerprint calculation post-migrate (#2892) 2022-09-07 14:21:10 +10:00
WithoutPants
13bdba5b24 Fix json filename generation (#2887) 2022-09-06 07:04:52 +00:00
WithoutPants
276f14cdcb Fix filename generation in export (#2883) 2022-09-06 07:04:52 +00:00
WithoutPants
6b0bcdea88 [Files Refactor] Set primary flag when cleaning (#2880)
* Ensure single primary per object
* Set primary file during clean
* Only show migration notes for actual migrations
2022-09-06 07:04:52 +00:00
WithoutPants
94d39da706 [Files Refactor] Bug fixes (#2868)
* Return error if multiple rows returned for id
* Add missing LoadFiles calls
* Show id if path is empty
2022-09-06 07:04:52 +00:00
WithoutPants
273cf0383d [Files Refactor] Performance tuning (#2865)
* Don't load image files by default
* Don't load gallery files by default
* Don't load scene files by default
* Retry locked transactions forever
* Don't show release notes if config not loaded
* Don't translate path slashes in export
2022-09-06 07:04:52 +00:00
WithoutPants
0b534d89c6 [Files Refactor] Import export fixup (#2763)
* Adjust json schema
* Remove mappings file from export
* Import file/folder support
* Update documentation
* Make gallery filenames unique
2022-09-06 07:04:52 +00:00
WithoutPants
00608c167a [Files Refactor] Performance tuning (#2819)
* Load scene relationships on demand
* Load image relationships on demand
* Load gallery relationships on demand
* Add dataloaden
* Use dataloaders
* Use where in for other find many functions
2022-09-06 07:03:42 +00:00
WithoutPants
9b31b20fed [Files Refactor] Performance tuning (#2813)
* Do database txn in same thread. Retry on locked db
* Remove captions from slimscenedata
* Fix tracing
* Use where in instead of individual selects
* Remove scenes_query view
* Remove image query view
* Remove gallery query view
* Use where in for FindMany
* Don't interrupt scanning zip files
* Fix image filesize sort
2022-09-06 07:03:42 +00:00
WithoutPants
bc47932343 [Files Refactor] Performance tuning (#2784)
* Improve image query performance
* Tune queries
* Fix db generator
* Don't show release notes in setup
* Further tune indexes
* Log when creating screenshot
2022-09-06 07:03:42 +00:00
WithoutPants
461068462c [Files Refactor] Filter and sort by file count (#2744)
* Add filtering on file count
* Add sorting by file count
2022-09-06 07:03:42 +00:00
WithoutPants
5495d72849 File storage rewrite (#2676)
* Restructure data layer part 2 (#2599)
* Refactor and separate image model
* Refactor image query builder
* Handle relationships in image query builder
* Remove relationship management methods
* Refactor gallery model/query builder
* Add scenes to gallery model
* Convert scene model
* Refactor scene models
* Remove unused methods
* Add unit tests for gallery
* Add image tests
* Add scene tests
* Convert unnecessary scene value pointers to values
* Convert unnecessary pointer values to values
* Refactor scene partial
* Add scene partial tests
* Refactor ImagePartial
* Add image partial tests
* Refactor gallery partial update
* Add partial gallery update tests
* Use zero/null package for null values
* Add files and scan system
* Add sqlite implementation for files/folders
* Add unit tests for files/folders
* Image refactors
* Update image data layer
* Refactor gallery model and creation
* Refactor scene model
* Refactor scenes
* Don't set title from filename
* Allow galleries to freely add/remove images
* Add multiple scene file support to graphql and UI
* Add multiple file support for images in graphql/UI
* Add multiple file for galleries in graphql/UI
* Remove use of some deprecated fields
* Remove scene path usage
* Remove gallery path usage
* Remove path from image
* Move funscript to video file
* Refactor caption detection
* Migrate existing data
* Add post commit/rollback hook system
* Lint. Comment out import/export tests
* Add WithDatabase read only wrapper
* Prepend tasks to list
* Add 32 pre-migration
* Add warnings in release and migration notes
2022-09-06 07:03:42 +00:00
WithoutPants
964b559309 Restructure data layer (#2532)
* Add new txn manager interface
* Add txn management to sqlite
* Rename get to getByID
* Add contexts to repository methods
* Update query builders
* Add context to reader writer interfaces
* Use repository in resolver
* Tighten interfaces
* Tighten interfaces in dlna
* Tighten interfaces in match package
* Tighten interfaces in scraper package
* Tighten interfaces in scan code
* Tighten interfaces on autotag package
* Remove ReaderWriter usage
* Merge database package into sqlite
2022-09-06 07:03:40 +00:00
WithoutPants
7b5bd80515 Separate graphql API from rest of the system (#2503)
* Move graphql generated files to api
* Refactor identify options
* Remove models.StashBoxes
* Move ScraperSource to scraper package
* Rename field strategy enums
* Rename identify.TaskOptions to Options
2022-09-06 07:03:40 +00:00
bnkai
6cfb7fe79d Fix Synopsis json string in movie jsonschema (#2664)
* Fix Synopsis json string in movie jsonschema
* backwards compatible movie synopsis import
2022-06-22 10:59:39 +10:00
CJ
9264c15540 Customize recommendations (#2592)
* refactored common code in recommendation row
* Implement front page options in config
* Allow customisation from front page
* Rename recommendations to front page
* Add generic UI settings
* Support adding premade filters

Co-authored-by: WithoutPants <53250216+WithoutPants@users.noreply.github.com>
2022-06-14 10:34:04 +10:00
cj
c1a096a1a6 Caption support (#2462)
Co-authored-by: WithoutPants <53250216+WithoutPants@users.noreply.github.com>
2022-05-06 11:59:28 +10:00
WithoutPants
aacf07feef Restructure ffmpeg (#2392)
* Refactor transcode generation
* Move phash generation into separate package
* Refactor image thumbnail generation
* Move JSONTime to separate package
* Ffmpeg refactoring
* Refactor live transcoding
* Refactor scene marker preview generation
* Refactor preview generation
* Refactor screenshot generation
* Refactor sprite generation
* Change ffmpeg.IsStreamable to return error
* Move frame rate calculation into ffmpeg
* Refactor file locking
* Refactor title set during scan
* Add missing lockmanager instance
* Return error instead of logging in MatchContainer
2022-04-18 10:50:10 +10:00
WithoutPants
61d9f57ce9 Add ignore autotag flag (#2439)
* Add autoTagIgnored to database schema
* Graphql changes
* UI changes
* Add field to edit performers dialog
* Apply flag to autotag behaviour
2022-04-04 20:03:39 +10:00
WithoutPants
f69bd8a94f Restructure go project (#2356)
* Move main to cmd
* Move api to internal
* Move logger and manager to internal
* Move shell hiding code to separate package
* Decouple job from desktop and utils
* Decouple session from config
* Move static into internal
* Decouple config from dlna
* Move desktop to internal
* Move dlna to internal
* Decouple remaining packages from config
* Move config into internal
* Move jsonschema and paths to models
* Make ffmpeg functions private
* Move file utility methods into fsutil package
* Move symwalk into fsutil
* Move single-use util functions into client package
* Move slice functions to separate packages
* Add env var to suppress windowsgui arg
* Move hash functions into separate package
* Move identify to internal
* Move autotag to internal
* Touch UI when generating backend
2022-03-17 11:33:59 +11:00
agentfisk
e8447c520a Heatmap and speed sorting/filtering for interactive scenes [rewrite] (#2096)
* add InteractiveSpeed to scene model
* add InteractiveHeatmapSpeedGenerator
* add GenerateInteractiveHeatmapSpeedTask
* add InteractiveHeatmapSpeedTask to GenerateJob
* add InteractiveHeatmap on sceneRoutes
* delete heatmap when scene is destroyed
* render interactive heatmap in GridCard
* render InteractiveSpeed on SceneCard
* render InteractiveSpeed in SceneFileInfoPanel
* InteractiveSpeed filters
2021-12-13 13:41:07 +11:00
SmallCoccinelle
09c4c4173d Support a minModTime input on metadata scans. (#1951)
* Support a maxAge input on metadata scans.

Extend the GraphQL world with a Duration scalar. It is parsed as a
typical Go duration, i.e., "4h" is 4 hours. Alternatively, one can
pass an integer which is interpreted as seconds.

Extend Mutation.metadataScan(input: $input) to support a new optional
value, maxAge. If set, the scanner will exit early if the file it
is looking at has an mtime older than the cutOff point generated by

now() - maxAge

This speeds up scanning in the case where the user knows how old the
changes on disk are, by exiting the scan early if that is the case.

* Change maxAge into minModTime

Introduce a `Timestamp` scalar, so we have a scalar we control. Let
it accept three formats:

* RFC3339Nano
* @UNIX where UNIX is a unix-timestamp: seconds after 01-01-1970
* '<4h': a timestamp relative to the current server time

This scalar parses to a time.Time.

Use MinModTime in the scanner to filter out a large number of scan
analyzes by exiting the scan operation early.

* Heed the linter, perform errcheck

* Rename test vars for consistency.

* Code review: move minModTime into queuefiles

* Remove the ability to input Unix timestamps

Test failures on the CI-system explains why this is undesirable. It is
not clear what timezone one is operating in when entering a unix
timestamp. We could go with UTC, but it is so much easier to require an
RFC3339 timestamp, which avoids this problem entirely.

* Move the minModTime field into filters

Create a new filter input object for metadata scans, and push the
minModTime field in there. If we come up with new filters, they can
be added to that input object rather than cluttering the main input
object.

* Use utils.ParseDateStringAsTime

Replace time.Parse with utils.ParseDateStringAsTime

While here, add some more test cases for that parser.
2021-11-26 11:48:31 +11:00
WithoutPants
c98ad7a62c Validate SQLiteDate (#2052)
* Reject invalid SQLiteDate strings
* Add unit tests
* Handle null values
2021-11-23 08:18:43 +11:00
WithoutPants
2277d0a919 Add search string parsing (#1982)
* Add search string parsing
* Add manual page
2021-11-22 14:59:22 +11:00