Infinite "FixThumbnailsWithoutMetadata" scan with "Fixed 0 ThumbnailBook"

[noiret]

Infinite "FixThumbnailsWithoutMetadata" scan with "Fixed 0 ThumbnailBook"

[noiret]

You can donwload the book here : https://wormhole.app/0lpap#rQdfYhJxj1ZwOB-u3RpScw

[gotson]

Oh that's funny. We'll you probably shouldn't have that kind of files in your komga library.

I'll have a look at how I can fix your issue with the never ending tasks.

[gotson]

The logs are from 1.7.2. Please run the latest version and provide log file.

[noiret]

Hello, logs with 1.8.4 (latest) komga-1-8-4.log

[gotson]

thanks.

Difficult to see what's going on, i would need to have more information on what's in your DB.

Can you execute the following query in the DB via sqlite3 or any DB explorer and provide the results ?

select BOOK_ID,
       length(THUMBNAIL) as thumbnail_size,
       url,
       type,
       WIDTH,
       HEIGHT,
       MEDIA_TYPE,
       FILE_SIZE
from THUMBNAIL_BOOK
where FILE_SIZE = 0
   or MEDIA_TYPE = ''
   or WIDTH = 0
   or HEIGHT = 0
;

[noiret]

The CSV export from database.sqlite (semicolon separated) : Query-database-komga.csv

[gotson]

All the thumbnails listed are generated, but they don't have any data.

Can you try to request the thumbnail for one of those books, see what's the api returns?

[noiret]

With the first id in the list i get a 404 (using GET api/v1/books/{bookid}/thumbnail route) :

 cache-control: max-age=0,must-revalidate,no-transform,private 
 content-length: 0 
 date: Mon,11 Dec 2023 12:36:16 GMT 
 strict-transport-security: max-age=31536000 ; includeSubDomains 
 vary: Origin,Access-Control-Request-Method,Access-Control-Request-Headers 
 x-content-type-options: nosniff 
 x-frame-options: SAMEORIGIN 
 x-xss-protection: 0 

(HTTP 200 with book id not listed in the previous query)

[gotson]

Those books probably need to be analyzed again. Are they all part of the same library?

[noiret]

Yes, I only have one large library.
I have scheduled analysis on it.

[noiret]

The analysis is complete, but the problem seems to persist

2023-12-12T09:27:17.867Z  INFO 1 --- [taskProcessor-2] o.g.komga.application.tasks.TaskHandler  : Task FixThumbnailsWithoutMetadata(priority='0') executed in 243.737688ms
2023-12-12T09:27:17.871Z  INFO 1 --- [taskProcessor-4] o.g.komga.application.tasks.TaskHandler  : Executing task: FixThumbnailsWithoutMetadata(priority='0')
2023-12-12T09:27:18.119Z  INFO 1 --- [taskProcessor-4] o.g.k.domain.service.ThumbnailLifecycle  : Fetched 1000 ThumbnailBook to fix, total: 3405
2023-12-12T09:27:18.119Z  INFO 1 --- [taskProcessor-4] o.g.k.domain.service.ThumbnailLifecycle  : Fixed 0 ThumbnailBook in 247.927913ms
2023-12-12T09:27:18.119Z  INFO 1 --- [taskProcessor-4] o.g.komga.application.tasks.TaskEmitter  : Sending task: FixThumbnailsWithoutMetadata(priority='0')
2023-12-12T09:27:18.122Z  INFO 1 --- [taskProcessor-4] o.g.komga.application.tasks.TaskHandler  : Task FixThumbnailsWithoutMetadata(priority='0') executed in 250.661296ms
2023-12-12T09:27:18.125Z  INFO 1 --- [taskProcessor-1] o.g.komga.application.tasks.TaskHandler  : Executing task: FixThumbnailsWithoutMetadata(priority='0')
2023-12-12T09:27:18.436Z  INFO 1 --- [taskProcessor-1] o.g.k.domain.service.ThumbnailLifecycle  : Fetched 1000 ThumbnailBook to fix, total: 3405
2023-12-12T09:27:18.436Z  INFO 1 --- [taskProcessor-1] o.g.k.domain.service.ThumbnailLifecycle  : Fixed 0 ThumbnailBook in 310.778248ms
2023-12-12T09:27:18.436Z  INFO 1 --- [taskProcessor-1] o.g.komga.application.tasks.TaskEmitter  : Sending task: FixThumbnailsWithoutMetadata(priority='0')
2023-12-12T09:27:18.439Z  INFO 1 --- [taskProcessor-1] o.g.komga.application.tasks.TaskHandler  : Task FixThumbnailsWithoutMetadata(priority='0') executed in 313.307328ms
2023-12-12T09:27:18.442Z  INFO 1 --- [taskProcessor-6] o.g.komga.application.tasks.TaskHandler  : Executing task: FixThumbnailsWithoutMetadata(priority='0')
2023-12-12T09:27:18.688Z  INFO 1 --- [taskProcessor-6] o.g.k.domain.service.ThumbnailLifecycle  : Fetched 1000 ThumbnailBook to fix, total: 3405
2023-12-12T09:27:18.688Z  INFO 1 --- [taskProcessor-6] o.g.k.domain.service.ThumbnailLifecycle  : Fixed 0 ThumbnailBook in 246.405242ms
2023-12-12T09:27:18.689Z  INFO 1 --- [taskProcessor-6] o.g.komga.application.tasks.TaskEmitter  : Sending task: FixThumbnailsWithoutMetadata(priority='0')
2023-12-12T09:27:18.691Z  INFO 1 --- [taskProcessor-6] o.g.komga.application.tasks.TaskHandler  : Task FixThumbnailsWithoutMetadata(priority='0') executed in 248.853406ms
2023-12-12T09:27:18.694Z  INFO 1 --- [taskProcessor-5] o.g.komga.application.tasks.TaskHandler  : Executing task: FixThumbnailsWithoutMetadata(priority='0')
2023-12-12T09:27:18.938Z  INFO 1 --- [taskProcessor-5] o.g.k.domain.service.ThumbnailLifecycle  : Fetched 1000 ThumbnailBook to fix, total: 3405
2023-12-12T09:27:18.938Z  INFO 1 --- [taskProcessor-5] o.g.k.domain.service.ThumbnailLifecycle  : Fixed 0 ThumbnailBook in 244.136625ms
2023-12-12T09:27:18.939Z  INFO 1 --- [taskProcessor-5] o.g.komga.application.tasks.TaskEmitter  : Sending task: FixThumbnailsWithoutMetadata(priority='0')
2023-12-12T09:27:18.941Z  INFO 1 --- [taskProcessor-5] o.g.komga.application.tasks.TaskHandler  : Task FixThumbnailsWithoutMetadata(priority='0') executed in 246.762927ms
2023-12-12T09:27:18.945Z  INFO 1 --- [taskProcessor-3] o.g.komga.application.tasks.TaskHandler  : Executing task: FixThumbnailsWithoutMetadata(priority='0')
2023-12-12T09:27:19.198Z  INFO 1 --- [taskProcessor-3] o.g.k.domain.service.ThumbnailLifecycle  : Fetched 1000 ThumbnailBook to fix, total: 3405
2023-12-12T09:27:19.198Z  INFO 1 --- [taskProcessor-3] o.g.k.domain.service.ThumbnailLifecycle  : Fixed 0 ThumbnailBook in 252.760988ms
2023-12-12T09:27:19.198Z  INFO 1 --- [taskProcessor-3] o.g.komga.application.tasks.TaskEmitter  : Sending task: FixThumbnailsWithoutMetadata(priority='0')
2023-12-12T09:27:19.200Z  INFO 1 --- [taskProcessor-3] o.g.komga.application.tasks.TaskHandler  : Task FixThumbnailsWithoutMetadata(priority='0') executed in 255.517088ms
2023-12-12T09:27:19.204Z  INFO 1 --- [taskProcessor-4] o.g.komga.application.tasks.TaskHandler  : Executing task: FixThumbnailsWithoutMetadata(priority='0')
< ... >

[gotson]

Can you locate one of the book for which it does not work, analyze it, and see if there's anything in the logs ?

[noiret]

Yes, here is the log : analyse-book-komga.log

[gotson]

I suspect the books do not have a cover at all. Could you send me one of those books ?

[noiret]

For information, i also have ZIP archives in the exported list

[gotson]

For information, i also have ZIP archives in the exported list

I'm not sure to understand that statement.

[noiret]

For example, the book with the ID "0CDYKE6554E8P" in the list ( #1338 (comment)) is not an EPUB file but an archive : https://wormhole.app/5zR8R#ayTN1Hw3dbBRm7eOhS8TFQ

[noiret]

For me, the problem doesn't seem to be linked to these files (these files have been there for years without issue).
What's new is that Komga tries to generate thumbnails on these files in a loop and produce high CPU usage.

[gotson]

For example, the book with the ID "0CDYKE6554E8P" in the list ( #1338 (comment)) is not an EPUB file but an archive : https://wormhole.app/5zR8R#ayTN1Hw3dbBRm7eOhS8TFQ

That file has no images in it, thus should show as in error. Can you confirm?

[noiret]

That file has no images in it, thus should show as in error. Can you confirm?

Just to be clear, the archive does not contain an image at the root, but a PDF with images.

[noiret]

And no, the file is not listed in media analysis page

[gotson]

That file has no images in it, thus should show as in error. Can you confirm?

Just to be clear, the archive does not contain an image at the root, but a PDF with images.

Pdf inside archives are not supported.

And no, the file is not listed in media analysis page

Can you check the book media status in the API for that book?

[gotson]

Actually for that book if you could provide the json from /api/v1/books/{bookId} that would help

[noiret]

Actually for that book if you could provide the json from /api/v1/books/{bookId} that would help

{
  "id": "0CDYKE6554E8P",
  "seriesId": "0CDYKE6554E8N",
  "seriesTitle": "ARTeam",
  "libraryId": "0CDYKDSJN4BKY",
  "name": "ARTeam_eZine_Number1",
  "url": "/mnt/livres/Zines/ARTeam/ARTeam_eZine_Number1.rar",
  "number": 3,
  "created": "2023-05-21T09:47:38Z",
  "lastModified": "2023-12-11T16:18:51Z",
  "fileLastModified": "2020-12-23T16:43:16Z",
  "sizeBytes": 3129556,
  "size": "3 MiB",
  "media": {
    "status": "READY",
    "mediaType": "application/x-rar-compressed; version=4",
    "pagesCount": 1,
    "comment": "",
    "mediaProfile": "DIVINA"
  },
  "metadata": {
    "title": "ARTeam_eZine_Number1",
    "titleLock": false,
    "summary": "",
    "summaryLock": false,
    "number": "3",
    "numberLock": false,
    "numberSort": 3,
    "numberSortLock": false,
    "releaseDate": null,
    "releaseDateLock": false,
    "authors": [],
    "authorsLock": false,
    "tags": [],
    "tagsLock": false,
    "isbn": "",
    "isbnLock": false,
    "links": [],
    "linksLock": false,
    "created": "2023-05-21T09:47:38Z",
    "lastModified": "2023-12-11T16:18:51Z"
  },
  "readProgress": null,
  "deleted": false,
  "fileHash": "9c94c119e39f67bb262e97b50fc08160",
  "oneshot": false
}

[gotson]

How about /api/v1/books/{bookId}/pages?

[noiret]

How about /api/v1/books/{bookId}/pages?

[
  {
    "number": 1,
    "fileName": "ARTeam.eZine.Number1\\Supplements\\Breaking Protocol\\Src\\Bruteus\\Bruteus\\icon.ico",
    "mediaType": "image/vnd.microsoft.icon",
    "width": null,
    "height": null,
    "sizeBytes": 1150,
    "size": "1.1 KiB"
  }
]

[gotson]

I've been looking at this, turns out an empty thumnail is created and saved when no poster can be found. This should not happen and i'll fix this.

I will also do a database cleanup in the next release to remove those empty thumbnails, which should prevent the fixThumbnails task to loop through them.

[noiret]

Thanks for your support 👍

[github-actions]

🎉 This issue has been resolved in 1.9.0 (Release Notes)