Scraping Music

From Official Kodi Wiki
Revision as of 06:35, 10 October 2017 by Karellen (talk | contribs) (New Page)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search
Steps to create your Music Library
Nuvola apps kscd.png
1. Guide Main Page
2. Music Settings
3. Music File Tagging
4. Scanning Music Into Library
5. Artist information folder
6. Scraping Additional Music data

7. Update Music Library
8. NFO Files
8.1 Music NFO Files
9. Artwork
9.1 Music Artwork


10. Import-export library
10.1 Music Import-Export
11. Backup & Recover
Zappy.png
Home icon grey.png   ▶ Music library ▶ Scraping Music


Introduction

Music Scraping downloads additional information about the Artists and Albums in your library, as well as downloading any missing artwork for the Artists and Albums. This step is optional, but if you choose to scrape, it must be performed after your music is Scanned into the library.

It should be noted that the source of information can be either an online source, local nfo file or data in a Single File Export. For this guide, it is assumed you have no local files as this is a first setup of your library, and the scraped data is being retrieved from an online source.

Scraping of additional artist and album information does not happen when you run a library update (including when first adding a source) unless "Fetch additional info on update" is enabled. Currently this defaults to disabled. It is strongly recommended to keep this disabled for several reasons:

  • It allows you to perform the initial tag scan quickly. In v17 and below, when Fetch additional info on update is enabled, the two steps happen immediately each album and artist is added. This slows down populating the library. It is a background task, but still gives the impression that it is taking a considerable time to populate the library. In reality it is slowed down by the online scraping, and not the local tag scanning. This will be changed in v18 where Kodi will perform the initial tag scan then, once complete, will commence the online scrape.
  • The strategy of immediately scraping each album and artist as it is added can also have unwanted consequences. A common example involves compilation albums. Say for example you have both Fetch additional info on update and Show song and album artists as enabled, and a folder called "Compilations" containing all your Various Artist compliation albums. As the processing takes place in alphabetical order, this folder gets processed before the albums of artists in folders beginning with a letter later in the alphabet. The additional information for the song artists from these compilation albums gets added to the library and scraped before the other albums by that artist had been scanned, and so any artist.nfo file is never found and applied.
  • Bad tagging makes a messy library and then the online scrape compounds errors and makes the library difficult to unscramble. It is far better for the user to perform the initial scan, find and fix the bad tags, then perform the on-line scrape knowing there will be no further errors.
  • Scraping online also often leaves many gaps on first pass, then the user has to manually query for all anyway, often several times, to fill these gaps. This was because lack of correct throttling, and general server overload has meant that online scraping often failed (seen in the log as 503 server errors). This is expected to be fixed for v18.

This page will detail

  • What additional information is downloaded
  • Instructions to conduct the scrape


What is Scraped

The scraping process will download the following:
The additional information as listed in the following two sections. Note the following:


The following types of artwork will be downloaded:

  • Artist- Poster and Fanart
  • Album- Poster (album cover image)


Artists

The below table details the additional information Kodi scrapes into the library. This data cannot be scanned from tags as there are no music file tags for these items.

To see which tags are overwritten, refer to the previous section.

Be aware that not all skins display any or all of the additional data. The default Estuary skin will display only the Artist Bigraphy. Other skins will display more or all of the additional data.

Artist biography Artist birthday date and place Artist date and place died Artist fanart Artist genres Artist instruments
Artist moods Artist poster Artist styles Artist years active Band disbanded Band formation


Albums

The below table details the additional information Kodi scrapes into the library. This data cannot be scanned from tags as there are no music file tags for these items.

To see which tags are overwritten, refer to the previous section.

Be aware that not all skins display any or all of the additional data. The default Estuary skin will display only the Album Review. Other skins will display more or all of the additional data.

Album moods Album poster Album rating Album review
Album styles Album theme Album votes


Performing the Scrape

The following two sections will guide you through the scrape process. Be aware that this will need to be conducted twice- once for Artists and once for Albums.


Artists

To scrape additional information for Artists, follow these steps:

  1. Select Music from the main menu (Image 1)
  2. Select Artists from the category list (Image 2)
  3. Highlight any Artist (Image 3)
  4. Call up the Context Menu for the selected Artist (Image 3)
  5. Select Query info for all artists (Image 3)

Depending upon the size of your library, this process could take many hours. As an example, a library of around 900 artists required over three hours to complete the scrape. This is dependent on your internet connection and the load on the servers hosting the scraper database. It is best to perform the scrape overnight when Kodi is not in use.

Once complete, it should be run a second time to ensure "busy" responses were not received from the scraper site due to overloaded servers.


Albums

To scrape additional information for Albums, follow these steps:

  1. Select Music from the main menu (Image 1)
  2. Select Albums from the category list (Image 2)
  3. Highlight any Album (Image 3)
  4. Call up the Context Menu for the selected Album (Image 3)
  5. Select Query info for all albums (Image 3)

Depending upon the size of your library, this process could take many hours. As an example, a library of around 300 albums required almost two hours to complete the scrape. This is dependent on your internet connection and the load on the servers hosting the scraper database. It is best to perform the scrape overnight when Kodi is not in use.

Once complete, the scrape should be run a second time to ensure "busy" responses were not received from the scraper site due to overloaded servers.


End of Setup

Congratulations! You have reached the end of the second module of the Setup Guide. You should now have a fully functioning Music library full of information and artwork.

The following steps in the guide will help you tweak and modify your library. There is no requirement for you to read on at this stage. Become comfortable with your new library. When you feel there is need for a change, come back to this guide and continue with the remaining modules. We do recommend you take care to backup your library.


Display of scraped information using alternate skins:



Return to top


Next step: NFO Files