Artwork/Cache: Difference between revisions

From Official Kodi Wiki
Jump to navigation Jump to search
mNo edit summary
(Updated for v19)
(13 intermediate revisions by 3 users not shown)
Line 1: Line 1:
{{mininav|[[Development]]}}
{{Artwork_Contents}}
{{mininav|[[Music_library|Music Library]] {{l2|[[Video_library|Video Library]]}} | [[Artwork]] }}


; {{see also|artwork}}


{{cleanup}}
= Introduction =
<section begin="intro" />{{Kodi}} maintains a texture cache on local device storage for all artwork displayed in the GUI, whether from an online source or the local file system. These textures are optimized and stored locally for faster loading.


XBMC maintains a local texture cache for thumbs and fanart, to allow fast loading of these images so that skins can show off your media in the best way possible.
When artwork is added to the library, {{Kodi}} initially only has a URL or path to the artwork. The artwork itself is only downloaded (if from an online source) and then cached when the image is displayed in the GUI or other interface such as the Chorus web interface.<section end="intro" />


All textures that Kodi loads, with the exception of textures that are provided directly by the skin, are cached to the userdata/Thumbnails folder.<br />
All images that {{Kodi}} loads, with the exception of textures that are provided directly by the skin, are cached to the userdata/Thumbnails folder.<br />


{{#lst:Artwork|CacheMaxResolution}}
'''Note:''' It is not possible to disable the artwork cache and operate exclusively from locally saved artwork as described in the related artwork pages.


== Art for library items ==
As of v12:


The art URL's associated with library items are stored in the art table inside the video or music databases.  These store the original URL to the art (e.g. from http://themoviedb.org, http://thetvdb.com, or http://theaudiodb.com).  This means that central databases store where the art is obtained from, and multiple clients then just keep a local thumbnail cache as described below.


== The caching procedure ==
= Artwork for library items =
The caching procedure utilizes both copies (possibly scaled down) of the original texture, saved in .jpg or .png format, an (optional) fast loading .dds version of those images, and an SQL database that maintains the local cache.
The artwork URLs associated with library items are stored in the art table inside the video or music databases. These store the original URL to the artwork (e.g. from local artwork files or from web sources such as https://themoviedb.org, https://thetvdb.com, or https://theaudiodb.com). This means that central databases store where the art is obtained from, and multiple devices then just keep a local thumbnail cache as described below.


The cache operates primarily on an "as and when needed" basis.  i.e. textures are cached when they're first requested to be loaded.  In addition, however, in some places textures may be cached before they're immediately needed, such as in the scanning routines.  In either case, the main caching procedure starts with an image URL.  We cache as follows:


1.  We generate a CRC (hash) of the URL. <br />
2.  We cache the image, reducing it in size if required, to a local file with the given CRC.  It's cached as a PNG or JPG based on whether transparency is required.<br />
3.  We generate a hash of the image, based on size and modified date, that we can use to check whether the original image has changed or not.<br />
4.  We store these details, along with information on when the image was cached in the SQL database Textures.db.


== Loading textures ==
= The caching procedure =
The caching procedure utilizes an optimized copy of the original image saved in .jpg or .png format and an SQL database that maintains the local cache.


When loading a texture, Kodi basically just has a URL to the image. Given a particular URL, we load the texture as follows:
The cache operates primarily on an "as and when needed" basis; i.e. textures are cached when they're first requested for display in a skin or web interface. In some cases textures may be cached before they're immediately needed, such as thumbnails generated from video files. In either case, the main caching procedure starts with an image URL. It caches as follows:


1. We first check to see whether the image is provided by the skin.  If so, we load it directly.<br />
# {{Kodi}} generates a hash (CRC) of the URL, like "6a643bff".
2. We then check to see whether the URL is from our thumbnail cache.  If it is, we skip to step 5.<br />
# The texture is optimized (see [[#Texture optimization]]) and cached to a device-local file with the given URL hash. It is cached as a PNG if transparency is required and JPG otherwise.
3. If not, we look up to see whether this image is in our texture cache by querying the SQL database.  If it is, we swap the URL to the cached version and skip to step 5.<br />
# For local files, another hash of the image is generated based on size and modified date, which can be used to check if the original image has changed.
4.  If it's not in the cache, we cache the image as outlined above.<br />
# These details are stored in the SQL database Textures.db, along with information on when the image was cached.
5.  We then additionally check if a "fast loading" version (.dds) of the texture exists. If it does, we swap the URL to the fast loading version.<br />
6. We load the image.<br />
7. We (optionally) generate a "fast loading" version (.dds) of the texture, if it doesn't already exist.<br />
8.  We update the texture database with access times to the texture (allowing us to clear the texture cache out of images that are rarely used).<br />
9.  If it's been at least 24 hours since the texture was last loaded, we check whether or not the cached version needs to be refreshed from the original image.  If so, we cache the image as outlined above.


== DDS artwork ==


XBMC uses the compressed format DXT1 or DXT5 in the form of a .dds file.  These are GPU-friendly compressed formats which allow for very fast loading of textures.  It comes at the disadvantage of a loss in quality - DXT compression can in some cases be very bad quality.  Kodi uses libsquish which is reasonably optimal at compressing the images.  In the future Kodi may use DXT5 in the YCoCg colour space which is the same size as DXT5 (i.e. 8bits/pixel) but is visually close to being lossless.  For now, if Kodi deems the quality isn't quite good enough to use DXT1 or DXT5, we use ARGB (i.e. 32bits/pixel) which can result in some very large images on disk.  On systems with slow disk access, these may take longer to load than .jpg or .png versions.


The generation of .dds versions of the artwork is automatic if you have set the <useddsfanart> advancedsetting.  DDS versions are generated in step 7 above.  Even if you haven't set the advanced setting, Kodi will still load .dds versions of the textures if they're available, so on slower systems you may wish to generate .dds versions offline and place them in the texture cache directly.
= Loading textures =
When loading a texture for display in a skin or web interface, {{Kodi}} basically just has a URL to the image. Given a particular URL, it loads the texture as follows:


== The future ==
# If the image is provided by the skin, skip to step 5.
# If the URL is directly to a cached version of a texture in the cache, skip to step 5.
# If the URL is for an image already cached in the thumbnail cache, swap the URL to the cached version and skip to step 5.
# Cache the image as outlined in the section above.
# Load the image. If the image is provided by the skin, skip steps 6 and 7.
# The access time for this texture is saved in the texture database.
# If the image is not from a web service (HTTP/S) and at least 24 hours have passed since it was last loaded, check the hash generated in step 3 of "The caching procedure" for changes to the image. If the hash has changed, cache the new image as outlined in the section above.


The texture cache in the future will be auto-cleaning, i.e. textures that are stale will automatically be cleaned out to save disk space.  Also, the music and video artwork will be moved to the new system.  This will allow the cached versions of these images to be updated whenever the original image changes (eg you replace the fanart with a different version).


In addition, the texture cache will be updated to automatically tell the image control that it needs to refresh itself.  Currently if the texturecache detects that a new version of an image is available (whether it be a .dds version or an updated copy of the original image that has since changed) then all that happens is the new version gets re-cached.  Thus, you only see the new version of an image once you view it a second time after any change (this allows the image to be free'd from memory and reloaded from the freshly updated version).


== How to refresh the cached image of an icon ==
= Texture optimization =
# Images are limited to a maximum pixel dimension to fit in a 16:9 box, with a default configuration of 1920x1080 for images that are 16:9 (like fanart) and larger than this, and 1280x720 for all other images. This can be changed with '''[[Advancedsettings.xml#Library artwork| advancedsettings.xml]]'''.
#* If you are experiencing slow loading images or lagging GUI navigation on low powered equipment and/or your {{Kodi}} device has minimal storage available, these defaults can be lowered to noticeably reduce the cost of artwork. For higher powered equipment these settings can be increased to maintain the original artwork size. (eg 3840x2160 fanart)
# Images are re-encoded in a decoder-friendly way so that the image can be displayed as quickly as possible.


There is an image like ''/home/x/.xbmc/addons/my.lovely.addon/icon.png'' . After changing that file, the old image is still visible in XBMC. To solve this we need to remove the actual thumbnail file from the filesystem and its reference in the DB:
This optimized image is exported with {{Kodi}}'s '''[[Import-export_library|Library Export]]''' feature, rather than the original full-quality image, so it is best to avoid exporting artwork if at all possible.<ref>https://forum.kodi.tv/showthread.php?tid=315806&pid=2599318#pid2599318</ref>
 
 
 
= Advanced: How to refresh the cached image of an icon manually =
The [[Texture Cache Maintenance utility]] can do this in a safer way, but this is the manual option.
There is an image like ''/home/x/.kodi/addons/my.lovely.addon/icon.png'' . After changing that file, the old image is still visible in {{kodi}}. To solve this we need to remove the actual thumbnail file from the filesystem and its reference in the DB:


  # Get the thumbnail path to remove the file from the filesystem
  # Get the thumbnail path to remove the file from the filesystem
  $ sqlite3 .xbmc/userdata/Database/Textures13.db "SELECT cachedurl FROM texture WHERE url = '/home/x/.xbmc/addons/my.lovely.addon/icon.png';"
  $ sqlite3 .kodi/userdata/Database/Textures13.db "SELECT cachedurl FROM texture WHERE url = '/home/x/.kodi/addons/my.lovely.addon/icon.png';"
  6/6a643bff.png
  6/6a643bff.png


This ''6/6a643bff.png'' is the cached image under ''/home/x/.xbmc/userdata/Thumbnails''. So: you need to remove from your filesystem ''/home/x/.xbmc/userdata/Thumbnails/6/6a43bff.png"
This ''6/6a643bff.png'' is the cached image under ''/home/x/.kodi/userdata/Thumbnails''. So: you need to remove from your filesystem ''/home/x/.kodi/userdata/Thumbnails/6/6a43bff.png''


  # Remove the image->cached image link in the DB
  # Remove the image->cached image link in the DB
  $ sqlite3 .xbmc/userdata/Database/Textures13.db "DELETE FROM texture WHERE url = '/home/x/.xbmc/addons/my.lovely.addon/icon.png';"
  $ sqlite3 .kodi/userdata/Database/Textures13.db "DELETE FROM texture WHERE url = '/home/x/.kodi/addons/my.lovely.addon/icon.png';"


Next time Kodi needs the image, it will recreate the thumbnail.
Next time Kodi needs the image, it will recreate the thumbnail.


:{{note|The <code>"Textures'''13'''.db"</code> '''number/version''' Increases with newer Kodi versions.}}
:{{note|The <code>"Textures'''13'''.db"</code> '''[[Databases#Database_Versions|number/version can increase]]''' with later Kodi versions.}}
 
'''''See also: [[Databases#Database_Versions|Database Versions]]'''''
 
 


{{see also|Database_versions}}
= Advanced: delete textures13.db =
This last resort procedure is one sure way to repair any image caching problem: delete the ''Textures13.db'' database and the thumbnail directory. This procedure does not affect the entries in the Video and Music libraries, so Kodi will not find more images after, this is just a brute-force solution to a caching problem. It affects the artwork database only. [[Texture Cache Maintenance utility]] can do very targeted fixes of the texture cache, so consider it before doing this.


== Cache manipulation tools ==
'''Note'''
#Be aware that this should be a last resort procedure. Deleting this database file and the thumbnail directory will remove all cache entries and links, requiring remote URLs to web services to be downloaded again.
#As the URL link to the original artwork location is stored in the video or music databases, Kodi will use the existing URLs to rebuild the Texture database and thumbnail cache, rather than finding new images locally or from a scraper.
#{{red|Using this procedure, you cannot change the scraping location for artwork. ie- If you originally scraped from online sources, you cannot expect that Kodi will now scrape from your newly added local artwork and vice-versa. It will retrieve artwork from the same source as it was before, so the result will be the same image. To bulk change artwork, refer to ''[[Artwork#Importing-Scraping Artwork| Importing-Scraping Artwork]]''}}<br /><br />


You will need to perform the following for a clean start with respect to the artwork texture cache.
#Exit Kodi
#Delete ''texturesxx.db'' located in the ''\userdata\database\'' folder.
#Delete the ''Thumbnails'' directory located at ''\userdata\'' which stores all the cached images,
Once the above steps are performed
#Restart Kodi
#Kodi will automatically rebuild the Textures database and repopulate the Thumbnails cache as the images are displayed in the GUI or other Kodi interface.
= Cache manipulation tools =
* [[Texture Cache Maintenance utility]]
* [[Texture Cache Maintenance utility]]
** {{#lst:Texture_Cache_Maintenance_utility|intro}}
= Kodi version updates =
<section begin="CacheMaxResolution" />From v18, the 1920x1080 resolution limitation has been removed. Using the following code in your advancedsettings.xml will allow caching artwork at original size. <ref>https://github.com/xbmc/xbmc/pull/13599</ref>
<syntaxhighlight lang="xml">
<imageres>9999</imageres>
<fanartres>9999</fanartres>
</syntaxhighlight>
<section end="CacheMaxResolution" />
= References =
<references />


{{Top}}
{{updated|19}}


[[Category:Development]]
[[Category:Karellen]]
[[Category:Manual]]
[[Category:Music library]]
[[Category:Video library]]

Revision as of 03:48, 23 June 2021

Artwork contents

Artwork Main Page



See also:


Return to:

Home icon grey.png   ▶ Music Library
▶ Video Library
▶ Artwork ▶ Cache


Introduction

Kodi maintains a texture cache on local device storage for all artwork displayed in the GUI, whether from an online source or the local file system. These textures are optimized and stored locally for faster loading.

When artwork is added to the library, Kodi initially only has a URL or path to the artwork. The artwork itself is only downloaded (if from an online source) and then cached when the image is displayed in the GUI or other interface such as the Chorus web interface.

All images that Kodi loads, with the exception of textures that are provided directly by the skin, are cached to the userdata/Thumbnails folder.

Note: It is not possible to disable the artwork cache and operate exclusively from locally saved artwork as described in the related artwork pages.


Artwork for library items

The artwork URLs associated with library items are stored in the art table inside the video or music databases. These store the original URL to the artwork (e.g. from local artwork files or from web sources such as https://themoviedb.org, https://thetvdb.com, or https://theaudiodb.com). This means that central databases store where the art is obtained from, and multiple devices then just keep a local thumbnail cache as described below.


The caching procedure

The caching procedure utilizes an optimized copy of the original image saved in .jpg or .png format and an SQL database that maintains the local cache.

The cache operates primarily on an "as and when needed" basis; i.e. textures are cached when they're first requested for display in a skin or web interface. In some cases textures may be cached before they're immediately needed, such as thumbnails generated from video files. In either case, the main caching procedure starts with an image URL. It caches as follows:

  1. Kodi generates a hash (CRC) of the URL, like "6a643bff".
  2. The texture is optimized (see #Texture optimization) and cached to a device-local file with the given URL hash. It is cached as a PNG if transparency is required and JPG otherwise.
  3. For local files, another hash of the image is generated based on size and modified date, which can be used to check if the original image has changed.
  4. These details are stored in the SQL database Textures.db, along with information on when the image was cached.


Loading textures

When loading a texture for display in a skin or web interface, Kodi basically just has a URL to the image. Given a particular URL, it loads the texture as follows:

  1. If the image is provided by the skin, skip to step 5.
  2. If the URL is directly to a cached version of a texture in the cache, skip to step 5.
  3. If the URL is for an image already cached in the thumbnail cache, swap the URL to the cached version and skip to step 5.
  4. Cache the image as outlined in the section above.
  5. Load the image. If the image is provided by the skin, skip steps 6 and 7.
  6. The access time for this texture is saved in the texture database.
  7. If the image is not from a web service (HTTP/S) and at least 24 hours have passed since it was last loaded, check the hash generated in step 3 of "The caching procedure" for changes to the image. If the hash has changed, cache the new image as outlined in the section above.


Texture optimization

  1. Images are limited to a maximum pixel dimension to fit in a 16:9 box, with a default configuration of 1920x1080 for images that are 16:9 (like fanart) and larger than this, and 1280x720 for all other images. This can be changed with advancedsettings.xml.
    • If you are experiencing slow loading images or lagging GUI navigation on low powered equipment and/or your Kodi device has minimal storage available, these defaults can be lowered to noticeably reduce the cost of artwork. For higher powered equipment these settings can be increased to maintain the original artwork size. (eg 3840x2160 fanart)
  2. Images are re-encoded in a decoder-friendly way so that the image can be displayed as quickly as possible.

This optimized image is exported with Kodi's Library Export feature, rather than the original full-quality image, so it is best to avoid exporting artwork if at all possible.[1]


Advanced: How to refresh the cached image of an icon manually

The Texture Cache Maintenance utility can do this in a safer way, but this is the manual option. There is an image like /home/x/.kodi/addons/my.lovely.addon/icon.png . After changing that file, the old image is still visible in Kodi. To solve this we need to remove the actual thumbnail file from the filesystem and its reference in the DB:

# Get the thumbnail path to remove the file from the filesystem
$ sqlite3 .kodi/userdata/Database/Textures13.db "SELECT cachedurl FROM texture WHERE url = '/home/x/.kodi/addons/my.lovely.addon/icon.png';"
6/6a643bff.png

This 6/6a643bff.png is the cached image under /home/x/.kodi/userdata/Thumbnails. So: you need to remove from your filesystem /home/x/.kodi/userdata/Thumbnails/6/6a43bff.png

# Remove the image->cached image link in the DB
$ sqlite3 .kodi/userdata/Database/Textures13.db "DELETE FROM texture WHERE url = '/home/x/.kodi/addons/my.lovely.addon/icon.png';"

Next time Kodi needs the image, it will recreate the thumbnail.

Note: The "Textures13.db" number/version can increase with later Kodi versions.

See also: Database Versions


Advanced: delete textures13.db

This last resort procedure is one sure way to repair any image caching problem: delete the Textures13.db database and the thumbnail directory. This procedure does not affect the entries in the Video and Music libraries, so Kodi will not find more images after, this is just a brute-force solution to a caching problem. It affects the artwork database only. Texture Cache Maintenance utility can do very targeted fixes of the texture cache, so consider it before doing this.

Note

  1. Be aware that this should be a last resort procedure. Deleting this database file and the thumbnail directory will remove all cache entries and links, requiring remote URLs to web services to be downloaded again.
  2. As the URL link to the original artwork location is stored in the video or music databases, Kodi will use the existing URLs to rebuild the Texture database and thumbnail cache, rather than finding new images locally or from a scraper.
  3. Using this procedure, you cannot change the scraping location for artwork. ie- If you originally scraped from online sources, you cannot expect that Kodi will now scrape from your newly added local artwork and vice-versa. It will retrieve artwork from the same source as it was before, so the result will be the same image. To bulk change artwork, refer to Importing-Scraping Artwork

You will need to perform the following for a clean start with respect to the artwork texture cache.

  1. Exit Kodi
  2. Delete texturesxx.db located in the \userdata\database\ folder.
  3. Delete the Thumbnails directory located at \userdata\ which stores all the cached images,

Once the above steps are performed

  1. Restart Kodi
  2. Kodi will automatically rebuild the Textures database and repopulate the Thumbnails cache as the images are displayed in the GUI or other Kodi interface.


Cache manipulation tools

  • Texture Cache Maintenance utility
    • Texture Cache Maintenance utility is a python script created by MillhouseVH, that can do a number of artwork related maintenance on the Kodi library. The script can be used to pre-generate thumbnails, clean up old thumbnail files, find corrupted files, and much more. The script can work on any OS that can run Python 2.6 or higher.


Kodi version updates

From v18, the 1920x1080 resolution limitation has been removed. Using the following code in your advancedsettings.xml will allow caching artwork at original size. [2]

<imageres>9999</imageres>
<fanartres>9999</fanartres>



References



Return to top