Artwork/Cache: Difference between revisions

From Official Kodi Wiki
Jump to navigation Jump to search
m (Robot: Changing Category:XBMC Manual to Category:Manual)
(17 intermediate revisions by 8 users not shown)
Line 1: Line 1:
{{mininav|[[Development]]}}
{{cleanup}}
{{cleanup}}
{{XBMC wiki toc}}
 
XBMC maintains a local texture cache for thumbs and fanart, to allow fast loading of these images so that skins can show off your media in the best way possible.
XBMC maintains a local texture cache for thumbs and fanart, to allow fast loading of these images so that skins can show off your media in the best way possible.


All textures that XBMC loads, with the exception of textures that are provided directly by the skin, are cached to the userdata/Thumbnails folder.
All textures that XBMC loads, with the exception of textures that are provided directly by the skin, are cached to the userdata/Thumbnails folder.<br />
 
 
== Art for library items ==
As of XBMC v12:


{{frodo box|1=
The art URL's associated with library items are stored in the art table inside the video or music databases.  These store the original URL to the art (e.g. from http://themoviedb.org, http://thetvdb.com, or http://theaudiodb.com).  This means that central databases store where the art is obtained from, and multiple clients then just keep a local thumbnail cache as described below.
==Art for library items==
The art URLs associated with library items are stored in the art table inside the video or music databases.  These store the original URL to the art (eg from http://themoviedb.org, http://thetvdb.com, or http://theaudiodb.org).  This means that central databases store where the art is obtained from, and multiple clients then just keep a local thumbnail cache as described below.
}}


==The caching procedure==
== The caching procedure ==
The caching procedure utilizes both copies (possibly scaled down) of the original texture, saved in .jpg or .png format, an (optional) fast loading .dds version of those images, and an SQL database that maintains the local cache.
The caching procedure utilizes both copies (possibly scaled down) of the original texture, saved in .jpg or .png format, an (optional) fast loading .dds version of those images, and an SQL database that maintains the local cache.


The cache operates primarily on an "as and when needed" basis.  i.e. textures are cached when they're first requested to be loaded.  In addition, however, in some places textures may be cached before they're immediately needed, such as in the scanning routines.  In either case, the main caching procedure starts with an image URL.  We cache as follows:
The cache operates primarily on an "as and when needed" basis.  i.e. textures are cached when they're first requested to be loaded.  In addition, however, in some places textures may be cached before they're immediately needed, such as in the scanning routines.  In either case, the main caching procedure starts with an image URL.  We cache as follows:


1.  We generate a CRC (hash) of the URL. (see [[Thumbnails#Sample Code|Code Samples]] on how to calculate CRC values)<br>
1.  We generate a CRC (hash) of the URL. (see [[Thumbnails#Sample Code|Code Samples]] on how to calculate CRC values)<br />
2.  We cache the image, reducing it in size if required, to a local file with the given CRC.  It's cached as a PNG or JPG based on whether transparency is required.<br>
2.  We cache the image, reducing it in size if required, to a local file with the given CRC.  It's cached as a PNG or JPG based on whether transparency is required.<br />
3.  We generate a hash of the image, based on size and modified date, that we can use to check whether the original image has changed or not.<br>
3.  We generate a hash of the image, based on size and modified date, that we can use to check whether the original image has changed or not.<br />
4.  We store these details, along with information on when the image was cached in the SQL database Textures.db.
4.  We store these details, along with information on when the image was cached in the SQL database Textures.db.


==Loading textures==
== Loading textures ==


When loading a texture, XBMC basically just has a URL to the image.  Given a particular URL, we load the texture as follows:
When loading a texture, XBMC basically just has a URL to the image.  Given a particular URL, we load the texture as follows:


1.  We first check to see whether the image is provided by the skin.  If so, we load it directly.<br>
1.  We first check to see whether the image is provided by the skin.  If so, we load it directly.<br />
2.  We then check to see whether the URL is from our thumbnail cache.  If it is, we skip to step 5.<br>
2.  We then check to see whether the URL is from our thumbnail cache.  If it is, we skip to step 5.<br />
3.  If not, we look up to see whether this image is in our texture cache by querying the SQL database.  If it is, we swap the URL to the cached version and skip to step 5.<br>
3.  If not, we look up to see whether this image is in our texture cache by querying the SQL database.  If it is, we swap the URL to the cached version and skip to step 5.<br />
4.  If it's not in the cache, we cache the image as outlined above.<br>
4.  If it's not in the cache, we cache the image as outlined above.<br />
5.  We then additionally check if a "fast loading" version (.dds) of the texture exists.  If it does, we swap the URL to the fast loading version.<br>
5.  We then additionally check if a "fast loading" version (.dds) of the texture exists.  If it does, we swap the URL to the fast loading version.<br />
6.  We load the image.<br>
6.  We load the image.<br />
7.  We (optionally) generate a "fast loading" version (.dds) of the texture, if it doesn't already exist.<br>
7.  We (optionally) generate a "fast loading" version (.dds) of the texture, if it doesn't already exist.<br />
8.  We update the texture database with access times to the texture (allowing us to clear the texture cache out of images that are rarely used).<br>
8.  We update the texture database with access times to the texture (allowing us to clear the texture cache out of images that are rarely used).<br />
9.  If it's been at least 24 hours since the texture was last loaded, we check whether or not the cached version needs to be refreshed from the original image.  If so, we cache the image as outlined above.
9.  If it's been at least 24 hours since the texture was last loaded, we check whether or not the cached version needs to be refreshed from the original image.  If so, we cache the image as outlined above.


==DDS artwork==
== DDS artwork ==


XBMC uses the compressed format DXT1 or DXT5 in the form of a .dds file.  These are GPU-friendly compressed formats which allow for very fast loading of textures.  It comes at the disadvantage of a loss in quality - DXT compression can in some cases be very bad quality.  XBMC uses libsquish which is reasonably optimal at compressing the images.  In the future XBMC may use DXT5 in the YCoCg colour space which is the same size as DXT5 (i.e. 8bits/pixel) but is visually close to being lossless.  For now, if XBMC deems the quality isn't quite good enough to use DXT1 or DXT5, we use ARGB (i.e. 32bits/pixel) which can result in some very large images on disk.  On systems with slow disk access, these may take longer to load than .jpg or .png versions.
XBMC uses the compressed format DXT1 or DXT5 in the form of a .dds file.  These are GPU-friendly compressed formats which allow for very fast loading of textures.  It comes at the disadvantage of a loss in quality - DXT compression can in some cases be very bad quality.  XBMC uses libsquish which is reasonably optimal at compressing the images.  In the future XBMC may use DXT5 in the YCoCg colour space which is the same size as DXT5 (i.e. 8bits/pixel) but is visually close to being lossless.  For now, if XBMC deems the quality isn't quite good enough to use DXT1 or DXT5, we use ARGB (i.e. 32bits/pixel) which can result in some very large images on disk.  On systems with slow disk access, these may take longer to load than .jpg or .png versions.
Line 40: Line 43:
The generation of .dds versions of the artwork is automatic if you have set the <useddsfanart> advancedsetting.  DDS versions are generated in step 7 above.  Even if you haven't set the advanced setting, XBMC will still load .dds versions of the textures if they're available, so on slower systems you may wish to generate .dds versions offline and place them in the texture cache directly.
The generation of .dds versions of the artwork is automatic if you have set the <useddsfanart> advancedsetting.  DDS versions are generated in step 7 above.  Even if you haven't set the advanced setting, XBMC will still load .dds versions of the textures if they're available, so on slower systems you may wish to generate .dds versions offline and place them in the texture cache directly.


==The future==
== The future ==


The texture cache in the future will be auto-cleaning, i.e. textures that are stale will automatically be cleaned out to save disk space.  Also, the music and video artwork will be moved to the new system.  This will allow the cached versions of these images to be updated whenever the original image changes (eg you replace the fanart with a different version).
The texture cache in the future will be auto-cleaning, i.e. textures that are stale will automatically be cleaned out to save disk space.  Also, the music and video artwork will be moved to the new system.  This will allow the cached versions of these images to be updated whenever the original image changes (eg you replace the fanart with a different version).


In addition, the texture cache will be updated to automatically tell the image control that it needs to refresh itself.  Currently if the texturecache detects that a new version of an image is available (whether it be a .dds version or an updated copy of the original image that has since changed) then all that happens is the new version gets re-cached.  Thus, you only see the new version of an image once you view it a second time after any change (this allows the image to be free'd from memory and reloaded from the freshly updated version).
In addition, the texture cache will be updated to automatically tell the image control that it needs to refresh itself.  Currently if the texturecache detects that a new version of an image is available (whether it be a .dds version or an updated copy of the original image that has since changed) then all that happens is the new version gets re-cached.  Thus, you only see the new version of an image once you view it a second time after any change (this allows the image to be free'd from memory and reloaded from the freshly updated version).
Lastly, DXT5YCoCg is likely to be supported, as is further compression for the on-disk images in the form of lzo.  If we do lzo compression then the cached versions will likely be 30-50% smaller, but ofcourse they will then be not loadable in any other software other than XBMC.


== How to refresh the cached image of an icon ==
== How to refresh the cached image of an icon ==
Line 53: Line 54:


  # Get the thumbnail path to remove the file from the filesystem
  # Get the thumbnail path to remove the file from the filesystem
  $ sqlite3 .xbmc/userdata/Database/Textures6.db "SELECT cachedurl FROM texture WHERE url = '/home/x/.xbmc/addons/my.lovely.addon/icon.png';"
  $ sqlite3 .xbmc/userdata/Database/Textures13.db "SELECT cachedurl FROM texture WHERE url = '/home/x/.xbmc/addons/my.lovely.addon/icon.png';"
  6/6a643bff.png
  6/6a643bff.png


Line 59: Line 60:


  # Remove the image->cached image link in the DB
  # Remove the image->cached image link in the DB
  $ sqlite3 .xbmc/userdata/Database/Textures6.db "DELETE FROM texture WHERE url = '/home/x/.xbmc/addons/my.lovely.addon/icon.png';"
  $ sqlite3 .xbmc/userdata/Database/Textures13.db "DELETE FROM texture WHERE url = '/home/x/.xbmc/addons/my.lovely.addon/icon.png';"


Next time XBMC needs the image, it will recreate the thumbnail.
Next time XBMC needs the image, it will recreate the thumbnail.


Remarks:
:{{note|The <code>"Textures'''13'''.db"</code> '''number/version''' Increases with newer XBMC versions.}}
* Not sure if the Textures DB is called always "Textures6.db".
 
{{see also|XBMC_databases/Versions}}
 
== Cache manipulation tools ==
 
* [[Texture Cache Maintenance utility]]


[[Category:Thumbnails]]
[[Category:Thumbnails]]
[[Category:Development]]
[[Category:Development]]
[[Category:Manual]]

Revision as of 21:45, 17 November 2014

Home icon grey.png   ▶ Development ▶ Cache
Cleanup.png This page or section may require cleanup, updating, spellchecking, reformatting and/or updated images. Please improve this page if you can. The discussion page may contain suggestions.


XBMC maintains a local texture cache for thumbs and fanart, to allow fast loading of these images so that skins can show off your media in the best way possible.

All textures that XBMC loads, with the exception of textures that are provided directly by the skin, are cached to the userdata/Thumbnails folder.


Art for library items

As of XBMC v12:

The art URL's associated with library items are stored in the art table inside the video or music databases. These store the original URL to the art (e.g. from http://themoviedb.org, http://thetvdb.com, or http://theaudiodb.com). This means that central databases store where the art is obtained from, and multiple clients then just keep a local thumbnail cache as described below.

The caching procedure

The caching procedure utilizes both copies (possibly scaled down) of the original texture, saved in .jpg or .png format, an (optional) fast loading .dds version of those images, and an SQL database that maintains the local cache.

The cache operates primarily on an "as and when needed" basis. i.e. textures are cached when they're first requested to be loaded. In addition, however, in some places textures may be cached before they're immediately needed, such as in the scanning routines. In either case, the main caching procedure starts with an image URL. We cache as follows:

1. We generate a CRC (hash) of the URL. (see Code Samples on how to calculate CRC values)
2. We cache the image, reducing it in size if required, to a local file with the given CRC. It's cached as a PNG or JPG based on whether transparency is required.
3. We generate a hash of the image, based on size and modified date, that we can use to check whether the original image has changed or not.
4. We store these details, along with information on when the image was cached in the SQL database Textures.db.

Loading textures

When loading a texture, XBMC basically just has a URL to the image. Given a particular URL, we load the texture as follows:

1. We first check to see whether the image is provided by the skin. If so, we load it directly.
2. We then check to see whether the URL is from our thumbnail cache. If it is, we skip to step 5.
3. If not, we look up to see whether this image is in our texture cache by querying the SQL database. If it is, we swap the URL to the cached version and skip to step 5.
4. If it's not in the cache, we cache the image as outlined above.
5. We then additionally check if a "fast loading" version (.dds) of the texture exists. If it does, we swap the URL to the fast loading version.
6. We load the image.
7. We (optionally) generate a "fast loading" version (.dds) of the texture, if it doesn't already exist.
8. We update the texture database with access times to the texture (allowing us to clear the texture cache out of images that are rarely used).
9. If it's been at least 24 hours since the texture was last loaded, we check whether or not the cached version needs to be refreshed from the original image. If so, we cache the image as outlined above.

DDS artwork

XBMC uses the compressed format DXT1 or DXT5 in the form of a .dds file. These are GPU-friendly compressed formats which allow for very fast loading of textures. It comes at the disadvantage of a loss in quality - DXT compression can in some cases be very bad quality. XBMC uses libsquish which is reasonably optimal at compressing the images. In the future XBMC may use DXT5 in the YCoCg colour space which is the same size as DXT5 (i.e. 8bits/pixel) but is visually close to being lossless. For now, if XBMC deems the quality isn't quite good enough to use DXT1 or DXT5, we use ARGB (i.e. 32bits/pixel) which can result in some very large images on disk. On systems with slow disk access, these may take longer to load than .jpg or .png versions.

The generation of .dds versions of the artwork is automatic if you have set the <useddsfanart> advancedsetting. DDS versions are generated in step 7 above. Even if you haven't set the advanced setting, XBMC will still load .dds versions of the textures if they're available, so on slower systems you may wish to generate .dds versions offline and place them in the texture cache directly.

The future

The texture cache in the future will be auto-cleaning, i.e. textures that are stale will automatically be cleaned out to save disk space. Also, the music and video artwork will be moved to the new system. This will allow the cached versions of these images to be updated whenever the original image changes (eg you replace the fanart with a different version).

In addition, the texture cache will be updated to automatically tell the image control that it needs to refresh itself. Currently if the texturecache detects that a new version of an image is available (whether it be a .dds version or an updated copy of the original image that has since changed) then all that happens is the new version gets re-cached. Thus, you only see the new version of an image once you view it a second time after any change (this allows the image to be free'd from memory and reloaded from the freshly updated version).

How to refresh the cached image of an icon

There is an image like /home/x/.xbmc/addons/my.lovely.addon/icon.png . After changing that file, the old image is still visible in XBMC. To solve this we need to remove the actual thumbnail file from the filesystem and its reference in the DB:

# Get the thumbnail path to remove the file from the filesystem
$ sqlite3 .xbmc/userdata/Database/Textures13.db "SELECT cachedurl FROM texture WHERE url = '/home/x/.xbmc/addons/my.lovely.addon/icon.png';"
6/6a643bff.png

This 6/6a643bff.png is the cached image under /home/x/.xbmc/userdata/Thumbnails. So: you need to remove from your filesystem /home/x/.xbmc/userdata/Thumbnails/6/6a43bff.png"

# Remove the image->cached image link in the DB
$ sqlite3 .xbmc/userdata/Database/Textures13.db "DELETE FROM texture WHERE url = '/home/x/.xbmc/addons/my.lovely.addon/icon.png';"

Next time XBMC needs the image, it will recreate the thumbnail.

Note: The "Textures13.db" number/version Increases with newer XBMC versions.

Cache manipulation tools