Official Kodi Wiki - User contributions [en]

Python tv scraper development

2023-01-21T22:38:35Z

Roman V M:

{{mininav|[[Scrapers]] {{l2|[[Development]]}} }}

=Introduction=

Historically, Kodi has been supporting [https://kodi.wiki/view/HOW-TO:Write_media_scrapers XML scraping addons] that allow parsing online information sources
about movies, TV shows, music and so on. However, this approach has its limitations. First, XML parsing definitions with regular expressions are difficult to write and maintain.
Second, many information sources implemented REST APIs for getting information and regular expressions are not suitable for parsing JSON data. That is why Python
scrapers have been introduced.

Python scrapers are written in Python language and have similar structure to [https://kodi.wiki/view/HOW-TO:Video_addon media addons] (media plugins).
They are also called by special <code>plugin://</code> URLs with query parameters. The main query parameter is <code>action</code> that defines a scraping stage.
An example of a URL query sting passed to a scraper plugin: <code>?action=getdetails&url=foo&pathSettings=%7B%22foo%22%3A+%22bar%22%7D</code>.
Like other plugins, scrapers are also interact with Kodi via the functions of <code>xbmcplugin</code> Python API module and pass information to Kodi through <code>xbmcgui.ListItem</code> instances.
Basically, a Python scraper is a media plugin that passes information to the Kodi database instead of presenting lists of playable media items.

A TV shows scraper must support "action" calls that are described below.

=Actions=

==Find==

The <code>find</code> action is used for searching for a specific TV show by title and optionally a year that are passed as additional query parameters of the plugin call.
This action should use <code>xbmcplugin.addDirectoryItem</code> or <code>xbmcplugin.addDirectoryItems</code> to pass <code>xbmcgui.ListItem</code> instances to Kodi.
If only one instance is passed then it is considered as a perfect match. Otherwise the media file won't be matched and will need to be resolved manually by selecting
a necessary item from a list.
The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''label''': passed as <code>label</code> parameter to the class constructor. This is the label that is presented to a user during scraping.

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary TV show info from data provider's
website or API. It can be, for example, a link to a TV show page on a TV information website or a some unique ID to request TV show information from a REST API.

* '''thumb''' (optional): passed via <code>setArtwork()</code> of <code>xbmcgui.ListItem</code> class instance method. This should be a URL of a TV show poster, for example.

==NfoUrl (TV show)==

The <code>NfoUrl</code> action is called as an alternative to <code>find</code> if <code>tvshow.nfo</code> file is present in a TV show directory.
The entire .NFO file contents are passed as <code>nfo</code>parameter. This action should use <code>xbmcplugin.addDirectoryItem</code> to pass a single <code>xbmcgui.ListItem</code> instance to Kodi.

The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary TV show info from data provider's
website or API. It can be, for example, a link to a TV show page on a TV show information website or a some unique ID to request TV show information from a REST API.

* '''Unique IDs''': set via <code>ListItem.setUniqueIDs()</code> instance method. A scraper should set at least the default unique ID from the TV information site it works with.
Optionally it can set unique IDs from other online TV show databases. This is needed to correctly get TV show's artwork.

* '''episodeguide''': set via <code>ListItem.setInfo()</code> instance method. This should be some unique string that can be used to retrieve the list of TV show episoded with all the necessary info.

If <code>tvshow.nfo</code> contains all the necessary info about the respective TV show in XML format then this actions may not pass any <code>xbmcgui.ListItem</code> instance to Kodi.
In this case Kodi will use all the information parsed from the XML file and will not call '''getdetails''' action.

==NfoUrl (episode)==

The <code>NfoUrl</code> action is also called as for each episode .NFO file that is present in a TV show directory.
The entire .NFO file contents are passed as <code>nfo</code>parameter. This action should use <code>xbmcplugin.addDirectoryItem</code> to pass a single <code>xbmcgui.ListItem</code> instance to Kodi.

The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary episode info from data provider's
website or API. It can be, for example, a link to the episode page on a TV show information website or some unique ID to request episode information from a REST API.

If an episode .NFO file contains all the necessary info about the respective episode in XML format then this action may not pass any <code>xbmcgui.ListItem</code> instance to Kodi.
In this case Kodi will use all the information parsed from the XML file and will not call '''getepisodedetails''' action.

==getdetails==

The <code>getdetails</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''url''' query parameter
from the previous stages and should set as much information to the <code>xbmcgui.ListItem</code> instance as possible using appropriate methods.
One of the necessary properties that are set via <code>ListItem.setInfo</code> method is <code>episodeguide</code>. This should be some unique string that can be used to retrieve
the list of TV show episoded with all the necessary info.

==getepisodelist==

The <code>getepisodelist</code> action should use <code>xbmcplugin.addDirectoryItem</code> or <code>xbmcplugin.addDirectoryItems</code> to pass <code>xbmcgui.ListItem</code> instances to Kodi
with information about available TV show episodes. This action receives '''url''' query parameter that is <code>episodeguide</code> property set by <code>getdetails</code> or <code>NfoUrl</code>action.

The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''url''': passed as <code>url</code> parameter to the class constructor. This is some unique string that can be used to retrieve information about a specific episode.

* '''season''': season number that is passed via <code>ListItem.setInfo</code> method.

* '''episode''': episode number that is passed <code>ListItem.setInfo</code> method.

* '''aired''': episode air date (if available) that is passed <code>ListItem.setInfo</code> method.

==getepisodedetails==

The <code>getepisodedetails</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''url''' query parameter
from the previous stage and should set as much information to the <code>xbmcgui.ListItem</code> instance as possible using appropriate methods.

==getartwork==

The <code>getartwork</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''id''' query parameter
that is the default unique ID set by '''NfoUrl''' or '''getdetails''' actions or parsed form an XML <code>tvshow.nfo</code> file if the scraper haven't set any. This action should set available artwork using <code>ListItem.addAvailableArtwork()</code> and <code>ListItem.setAvailableFanart()</code> instance methods.

'''Warning''': if your .NFO file contains the default unique ID that is different from the ID of the TV show database your scraper works with,
you must set the correct default unique ID in <code>NfoUrl</code> action. Otherwise <code>getdetails</code> will receive the default unique ID from your .NFO file
and your scraper may not be able to find any artwork using this ID.

=episodeguide=

The <code>episodeguide</code> string is passed to <code>getepisodedetails</code> action to retrieve the list of episodes. In legacy XML scrapers it was an actual URL of a page that was supposed to contain
the list of TV show episodes. Now in Python scrapers it can be any string that allows to correctly identify a TV show and retrieve the list of its episodes. However, the commonly accepted convention for <code>episodeguide</code> is that it should be a JSON-encoded string containing IDs of the specific TV show in various online databases. As a minimum it should include the ID of a show at the database
the scraper works with. An example of a JSON-encoded <code>episodeguide</code> string:

<code>{"tvmaze": "60", "tvrage": "4628", "tvdb": "72108", "imdb": "tt0364845"}</code>

Note that all IDs should be strings.

=Path Settings=

A scraper receives the current path settings via <code>pathSettings</code> query parameters as a JSON-encoded string with each action call.

=Example TV Shows Scraper=

A very high-level example of a Python TV show scraper can be found in the Kodi source code as [https://github.com/xbmc/xbmc/tree/master/addons/metadata.demo.tv metadata.demo.tv addon].

[[Category:Scraper]]

Python tv scraper development

2023-01-21T22:33:05Z

Roman V M:

{{mininav|[[Scrapers]] {{l2|[[Development]]}} }}

=Introduction=

Historically, Kodi has been supporting [https://kodi.wiki/view/HOW-TO:Write_media_scrapers XML scraping addons] that allow parsing online information sources
about movies, TV shows, music and so on. However, this approach has its limitations. First, XML parsing definitions with regular expressions are difficult to write and maintain.
Second, many information sources implemented REST APIs for getting information and regular expressions are not suitable for parsing JSON data. That is why Python
scrapers have been introduced.

Python scrapers are written in Python language and have similar structure to [https://kodi.wiki/view/HOW-TO:Video_addon media addons] (media plugins).
They are also called by special <code>plugin://</code> URLs with query parameters. The main query parameter is <code>action</code> that defines a scraping stage.
An example of a URL query sting passed to a scraper plugin: <code>?action=getdetails&url=foo&pathSettings=%7B%22foo%22%3A+%22bar%22%7D</code>.
Like other plugins, scrapers are also interact with Kodi via the functions of <code>xbmcplugin</code> Python API module and pass information to Kodi through <code>xbmcgui.ListItem</code> instances.
Basically, a Python scraper is a media plugin that passes information to the Kodi database instead of presenting lists of playable media items.

A TV shows scraper must support "action" calls that are described below.

=Actions=

==Find==

The <code>find</code> action is used for searching for a specific TV show by title and optionally a year that are passed as additional query parameters of the plugin call.
This action should use <code>xbmcplugin.addDirectoryItem</code> or <code>xbmcplugin.addDirectoryItems</code> to pass <code>xbmcgui.ListItem</code> instances to Kodi.
If only one instance is passed then it is considered as a perfect match. Otherwise the media file won't be matched and will need to be resolved manually by selecting
a necessary item from a list.
The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''label''': passed as <code>label</code> parameter to the class constructor. This is the label that is presented to a user during scraping.

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary TV show info from data provider's
website or API. It can be, for example, a link to a TV show page on a TV information website or a some unique ID to request TV show information from a REST API.

* '''thumb''' (optional): passed via <code>setArtwork()</code> of <code>xbmcgui.ListItem</code> class instance method. This should be a URL of a TV show poster, for example.

==NfoUrl (TV show)==

The <code>NfoUrl</code> action is called as an alternative to <code>find</code> if <code>tvshow.nfo</code> file is present in a TV show directory.
The entire .NFO file contents are passed as <code>nfo</code>parameter. This action should use <code>xbmcplugin.addDirectoryItem</code> to pass a single <code>xbmcgui.ListItem</code> instance to Kodi.

The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary TV show info from data provider's
website or API. It can be, for example, a link to a TV show page on a TV show information website or a some unique ID to request TV show information from a REST API.

* '''Unique IDs''': set via <code>ListItem.setUniqueIDs()</code> instance method. A scraper should set at least the default unique ID from the TV information site it works with.
Optionally it can set unique IDs from other online TV show databases. This is needed to correctly get TV show's artwork.

If <code>tvshow.nfo</code> contains all the necessary info about the respective TV show in XML format then this actions may not pass any <code>xbmcgui.ListItem</code> instance to Kodi.
In this case Kodi will use all the information parsed from the XML file and will not call '''getdetails''' action.

==NfoUrl (episode)==

The <code>NfoUrl</code> action is also called as for each episode .NFO file that is present in a TV show directory.
The entire .NFO file contents are passed as <code>nfo</code>parameter. This action should use <code>xbmcplugin.addDirectoryItem</code> to pass a single <code>xbmcgui.ListItem</code> instance to Kodi.

The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary episode info from data provider's
website or API. It can be, for example, a link to the episode page on a TV show information website or some unique ID to request episode information from a REST API.

If an episode .NFO file contains all the necessary info about the respective episode in XML format then this action may not pass any <code>xbmcgui.ListItem</code> instance to Kodi.
In this case Kodi will use all the information parsed from the XML file and will not call '''getepisodedetails''' action.

==getdetails==

The <code>getdetails</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''url''' query parameter
from the previous stages and should set as much information to the <code>xbmcgui.ListItem</code> instance as possible using appropriate methods.
One of the necessary properties that are set via <code>ListItem.setInfo</code> method is <code>episodeguide</code>. This should be some unique string that can be used to retrieve
the list of TV show episoded with all the necessary info.

==getepisodelist==

The <code>getepisodelist</code> action should use <code>xbmcplugin.addDirectoryItem</code> or <code>xbmcplugin.addDirectoryItems</code> to pass <code>xbmcgui.ListItem</code> instances to Kodi
with information about available TV show episodes. This action receives '''url''' query parameter that is <code>episodeguide</code> property set by <code>getdetails</code> or <code>NfoUrl</code>action.

The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''url''': passed as <code>url</code> parameter to the class constructor. This is some unique string that can be used to retrieve information about a specific episode.

* '''season''': season number that is passed via <code>ListItem.setInfo</code> method.

* '''episode''': episode number that is passed <code>ListItem.setInfo</code> method.

* '''aired''': episode air date (if available) that is passed <code>ListItem.setInfo</code> method.

==getepisodedetails==

The <code>getepisodedetails</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''url''' query parameter
from the previous stage and should set as much information to the <code>xbmcgui.ListItem</code> instance as possible using appropriate methods.

==getartwork==

The <code>getartwork</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''id''' query parameter
that is the default unique ID set by '''NfoUrl''' or '''getdetails''' actions or parsed form an XML <code>tvshow.nfo</code> file if the scraper haven't set any. This action should set available artwork using <code>ListItem.addAvailableArtwork()</code> and <code>ListItem.setAvailableFanart()</code> instance methods.

'''Warning''': if your .NFO file contains the default unique ID that is different from the ID of the TV show database your scraper works with,
you must set the correct default unique ID in <code>NfoUrl</code> action. Otherwise <code>getdetails</code> will receive the default unique ID from your .NFO file
and your scraper may not be able to find any artwork using this ID.

=episodeguide=

The <code>episodeguide</code> string is passed to <code>getepisodedetails</code> action to retrieve the list of episodes. In legacy XML scrapers it was an actual URL of a page that was supposed to contain
the list of TV show episodes. Now in Python scrapers it can be any string that allows to correctly identify a TV show and retrieve the list of its episodes. However, the commonly accepted convention for <code>episodeguide</code> is that it should be a JSON-encoded string containing IDs of the specific TV show in various online databases. As a minimum it should include the ID of a show at the database
the scraper works with. An example of a JSON-encoded <code>episodeguide</code> string:

<code>{"tvmaze": "60", "tvrage": "4628", "tvdb": "72108", "imdb": "tt0364845"}</code>

Note that all IDs should be strings.

=Path Settings=

A scraper receives the current path settings via <code>pathSettings</code> query parameters as a JSON-encoded string with each action call.

=Example TV Shows Scraper=

A very high-level example of a Python TV show scraper can be found in the Kodi source code as [https://github.com/xbmc/xbmc/tree/master/addons/metadata.demo.tv metadata.demo.tv addon].

[[Category:Scraper]]

Python tv scraper development

2023-01-21T22:30:42Z

Roman V M:

{{mininav|[[Scrapers]] {{l2|[[Development]]}} }}

=Introduction=

Historically, Kodi has been supporting [https://kodi.wiki/view/HOW-TO:Write_media_scrapers XML scraping addons] that allow parsing online information sources
about movies, TV shows, music and so on. However, this approach has its limitations. First, XML parsing definitions with regular expressions are difficult to write and maintain.
Second, many information sources implemented REST APIs for getting information and regular expressions are not suitable for parsing JSON data. That is why Python
scrapers have been introduced.

Python scrapers are written in Python language and have similar structure to [https://kodi.wiki/view/HOW-TO:Video_addon media addons] (media plugins).
They are also called by special <code>plugin://</code> URLs with query parameters. The main query parameter is <code>action</code> that defines a scraping stage.
An example of a URL query sting passed to a scraper plugin: <code>?action=getdetails&url=foo&pathSettings=%7B%22foo%22%3A+%22bar%22%7D</code>.
Like other plugins, scrapers are also interact with Kodi via the functions of <code>xbmcplugin</code> Python API module and pass information to Kodi through <code>xbmcgui.ListItem</code> instances.
Basically, a Python scraper is a media plugin that passes information to the Kodi database instead of presenting lists of playable media items.

A TV shows scraper must support "action" calls that are described below.

=Actions=

==Find==

The <code>find</code> action is used for searching for a specific TV show by title and optionally a year that are passed as additional query parameters of the plugin call.
This action should use <code>xbmcplugin.addDirectoryItem</code> or <code>xbmcplugin.addDirectoryItems</code> to pass <code>xbmcgui.ListItem</code> instances to Kodi.
If only one instance is passed then it is considered as a perfect match. Otherwise the media file won't be matched and will need to be resolved manually by selecting
a necessary item from a list.
The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''label''': passed as <code>label</code> parameter to the class constructor. This is the label that is presented to a user during scraping.

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary TV show info from data provider's
website or API. It can be, for example, a link to a TV show page on a TV information website or a some unique ID to request TV show information from a REST API.

* '''thumb''' (optional): passed via <code>setArtwork()</code> of <code>xbmcgui.ListItem</code> class instance method. This should be a URL of a TV show poster, for example.

==NfoUrl (TV show)==

The <code>NfoUrl</code> action is called as an alternative to <code>find</code> if <code>tvshow.nfo</code> file is present in a TV show directory.
The entire .NFO file contents are passed as <code>nfo</code>parameter. This action should use <code>xbmcplugin.addDirectoryItem</code> to pass a single <code>xbmcgui.ListItem</code> instance to Kodi.

The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary TV show info from data provider's
website or API. It can be, for example, a link to a TV show page on a TV show information website or a some unique ID to request TV show information from a REST API.

* '''Unique IDs''': set via <code>ListItem.setUniqueIDs()</code> instance method. A scraper should set at least the default unique ID from the TV information site it works with.
Optionally it can set unique IDs from other online TV show databases. This is needed to correctly get TV show's artwork.

* '''episodeguide''': set via <code>ListItem.setInfo()</code> instance method. A scraper should set '''episodeguide''' string that will allow to get the list of episodes from data provider's website or API.

If <code>tvshow.nfo</code> contains all the necessary info about the respective TV show in XML format then this actions may not pass any <code>xbmcgui.ListItem</code> instance to Kodi.
In this case Kodi will use all the information parsed from the XML file and will not call '''getdetails''' action.

==NfoUrl (episode)==

The <code>NfoUrl</code> action is also called as for each episode .NFO file that is present in a TV show directory.
The entire .NFO file contents are passed as <code>nfo</code>parameter. This action should use <code>xbmcplugin.addDirectoryItem</code> to pass a single <code>xbmcgui.ListItem</code> instance to Kodi.

The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary episode info from data provider's
website or API. It can be, for example, a link to the episode page on a TV show information website or some unique ID to request episode information from a REST API.

If an episode .NFO file contains all the necessary info about the respective episode in XML format then this action may not pass any <code>xbmcgui.ListItem</code> instance to Kodi.
In this case Kodi will use all the information parsed from the XML file and will not call '''getepisodedetails''' action.

==getdetails==

The <code>getdetails</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''url''' query parameter
from the previous stages and should set as much information to the <code>xbmcgui.ListItem</code> instance as possible using appropriate methods.
One of the necessary properties that are set via <code>ListItem.setInfo</code> method is <code>episodeguide</code>. This should be some unique string that can be used to retrieve
the list of TV show episoded with all the necessary info.

==getepisodelist==

The <code>getepisodelist</code> action should use <code>xbmcplugin.addDirectoryItem</code> or <code>xbmcplugin.addDirectoryItems</code> to pass <code>xbmcgui.ListItem</code> instances to Kodi
with information about available TV show episodes. This action receives '''url''' query parameter that is <code>episodeguide</code> property set by <code>getdetails</code> or <code>NfoUrl</code>action.

The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''url''': passed as <code>url</code> parameter to the class constructor. This is some unique string that can be used to retrieve information about a specific episode.

* '''season''': season number that is passed via <code>ListItem.setInfo</code> method.

* '''episode''': episode number that is passed <code>ListItem.setInfo</code> method.

* '''aired''': episode air date (if available) that is passed <code>ListItem.setInfo</code> method.

==getepisodedetails==

The <code>getepisodedetails</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''url''' query parameter
from the previous stage and should set as much information to the <code>xbmcgui.ListItem</code> instance as possible using appropriate methods.

==getartwork==

The <code>getartwork</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''id''' query parameter
that is the default unique ID set by '''NfoUrl''' or '''getdetails''' actions or parsed form an XML <code>tvshow.nfo</code> file if the scraper haven't set any. This action should set available artwork using <code>ListItem.addAvailableArtwork()</code> and <code>ListItem.setAvailableFanart()</code> instance methods.

'''Warning''': if your .NFO file contains the default unique ID that is different from the ID of the TV show database your scraper works with,
you must set the correct default unique ID in <code>NfoUrl</code> action. Otherwise <code>getdetails</code> will receive the default unique ID from your .NFO file
and your scraper may not be able to find any artwork using this ID.

=episodeguide=

The <code>episodeguide</code> string is passed to <code>getepisodedetails</code> action to retrieve the list of episodes. In legacy XML scrapers it was an actual URL of a page that was supposed to contain
the list of TV show episodes. Now in Python scrapers it can be any string that allows to correctly identify a TV show and retrieve the list of its episodes. However, the commonly accepted convention for <code>episodeguide</code> is that it should be a JSON-encoded string containing IDs of the specific TV show in various online databases. As a minimum it should include the ID of a show at the database
the scraper works with. An example of a JSON-encoded <code>episodeguide</code> string:

<code>{"tvmaze": "60", "tvrage": "4628", "tvdb": "72108", "imdb": "tt0364845"}</code>

Note that all IDs should be strings.

=Path Settings=

A scraper receives the current path settings via <code>pathSettings</code> query parameters as a JSON-encoded string with each action call.

=Example TV Shows Scraper=

A very high-level example of a Python TV show scraper can be found in the Kodi source code as [https://github.com/xbmc/xbmc/tree/master/addons/metadata.demo.tv metadata.demo.tv addon].

[[Category:Scraper]]

Python tv scraper development

2023-01-21T22:29:51Z

Roman V M:

{{mininav|[[Scrapers]] {{l2|[[Development]]}} }}

=Introduction=

Historically, Kodi has been supporting [https://kodi.wiki/view/HOW-TO:Write_media_scrapers XML scraping addons] that allow parsing online information sources
about movies, TV shows, music and so on. However, this approach has its limitations. First, XML parsing definitions with regular expressions are difficult to write and maintain.
Second, many information sources implemented REST APIs for getting information and regular expressions are not suitable for parsing JSON data. That is why Python
scrapers have been introduced.

Python scrapers are written in Python language and have similar structure to [https://kodi.wiki/view/HOW-TO:Video_addon media addons] (media plugins).
They are also called by special <code>plugin://</code> URLs with query parameters. The main query parameter is <code>action</code> that defines a scraping stage.
An example of a URL query sting passed to a scraper plugin: <code>?action=getdetails&url=foo&pathSettings=%7B%22foo%22%3A+%22bar%22%7D</code>.
Like other plugins, scrapers are also interact with Kodi via the functions of <code>xbmcplugin</code> Python API module and pass information to Kodi through <code>xbmcgui.ListItem</code> instances.
Basically, a Python scraper is a media plugin that passes information to the Kodi database instead of presenting lists of playable media items.

A TV shows scraper must support "action" calls that are described below.

=Actions=

==Find==

The <code>find</code> action is used for searching for a specific TV show by title and optionally a year that are passed as additional query parameters of the plugin call.
This action should use <code>xbmcplugin.addDirectoryItem</code> or <code>xbmcplugin.addDirectoryItems</code> to pass <code>xbmcgui.ListItem</code> instances to Kodi.
If only one instance is passed then it is considered as a perfect match. Otherwise the media file won't be matched and will need to be resolved manually by selecting
a necessary item from a list.
The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''label''': passed as <code>label</code> parameter to the class constructor. This is the label that is presented to a user during scraping.

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary TV show info from data provider's
website or API. It can be, for example, a link to a TV show page on a TV information website or a some unique ID to request TV show information from a REST API.

* '''thumb''' (optional): passed via <code>setArtwork()</code> of <code>xbmcgui.ListItem</code> class instance method. This should be a URL of a TV show poster, for example.

==NfoUrl (TV show)==

The <code>NfoUrl</code> action is called as an alternative to <code>find</code> if <code>tvshow.nfo</code> file is present in a TV show directory.
The entire .NFO file contents are passed as <code>nfo</code>parameter. This action should use <code>xbmcplugin.addDirectoryItem</code> to pass a single <code>xbmcgui.ListItem</code> instance to Kodi.

The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary TV show info from data provider's
website or API. It can be, for example, a link to a TV show page on a TV show information website or a some unique ID to request TV show information from a REST API.

* '''Unique IDs''': set via <code>ListItem.setUniqueIDs()</code> instance method. A scraper should set at least the default unique ID from the TV information site it works with.
Optionally it can set unique IDs from other online TV show databases. This is needed to correctly get TV show's artwork.

* '''episodeguide''': set via <code>ListItem.setInfo()</code> instance method. A scraper should set '''episodeguide''' string that will allow to get the list of episodes from data provider's
website or API.

If <code>tvshow.nfo</code> contains all the necessary info about the respective TV show in XML format then this actions may not pass any <code>xbmcgui.ListItem</code> instance to Kodi.
In this case Kodi will use all the information parsed from the XML file and will not call '''getdetails''' action.

==NfoUrl (episode)==

The <code>NfoUrl</code> action is also called as for each episode .NFO file that is present in a TV show directory.
The entire .NFO file contents are passed as <code>nfo</code>parameter. This action should use <code>xbmcplugin.addDirectoryItem</code> to pass a single <code>xbmcgui.ListItem</code> instance to Kodi.

The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary episode info from data provider's
website or API. It can be, for example, a link to the episode page on a TV show information website or some unique ID to request episode information from a REST API.

If an episode .NFO file contains all the necessary info about the respective episode in XML format then this action may not pass any <code>xbmcgui.ListItem</code> instance to Kodi.
In this case Kodi will use all the information parsed from the XML file and will not call '''getepisodedetails''' action.

==getdetails==

The <code>getdetails</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''url''' query parameter
from the previous stages and should set as much information to the <code>xbmcgui.ListItem</code> instance as possible using appropriate methods.
One of the necessary properties that are set via <code>ListItem.setInfo</code> method is <code>episodeguide</code>. This should be some unique string that can be used to retrieve
the list of TV show episoded with all the necessary info.

==getepisodelist==

The <code>getepisodelist</code> action should use <code>xbmcplugin.addDirectoryItem</code> or <code>xbmcplugin.addDirectoryItems</code> to pass <code>xbmcgui.ListItem</code> instances to Kodi
with information about available TV show episodes. This action receives '''url''' query parameter that is <code>episodeguide</code> property set by <code>getdetails</code> or <code>NfoUrl</code>action.

The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''url''': passed as <code>url</code> parameter to the class constructor. This is some unique string that can be used to retrieve information about a specific episode.

* '''season''': season number that is passed via <code>ListItem.setInfo</code> method.

* '''episode''': episode number that is passed <code>ListItem.setInfo</code> method.

* '''aired''': episode air date (if available) that is passed <code>ListItem.setInfo</code> method.

==getepisodedetails==

The <code>getepisodedetails</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''url''' query parameter
from the previous stage and should set as much information to the <code>xbmcgui.ListItem</code> instance as possible using appropriate methods.

==getartwork==

The <code>getartwork</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''id''' query parameter
that is the default unique ID set by '''NfoUrl''' or '''getdetails''' actions or parsed form an XML <code>tvshow.nfo</code> file if the scraper haven't set any. This action should set available artwork using <code>ListItem.addAvailableArtwork()</code> and <code>ListItem.setAvailableFanart()</code> instance methods.

'''Warning''': if your .NFO file contains the default unique ID that is different from the ID of the TV show database your scraper works with,
you must set the correct default unique ID in <code>NfoUrl</code> action. Otherwise <code>getdetails</code> will receive the default unique ID from your .NFO file
and your scraper may not be able to find any artwork using this ID.

=episodeguide=

The <code>episodeguide</code> string is passed to <code>getepisodedetails</code> action to retrieve the list of episodes. In legacy XML scrapers it was an actual URL of a page that was supposed to contain
the list of TV show episodes. Now in Python scrapers it can be any string that allows to correctly identify a TV show and retrieve the list of its episodes. However, the commonly accepted convention for <code>episodeguide</code> is that it should be a JSON-encoded string containing IDs of the specific TV show in various online databases. As a minimum it should include the ID of a show at the database
the scraper works with. An example of a JSON-encoded <code>episodeguide</code> string:

<code>{"tvmaze": "60", "tvrage": "4628", "tvdb": "72108", "imdb": "tt0364845"}</code>

Note that all IDs should be strings.

=Path Settings=

A scraper receives the current path settings via <code>pathSettings</code> query parameters as a JSON-encoded string with each action call.

=Example TV Shows Scraper=

A very high-level example of a Python TV show scraper can be found in the Kodi source code as [https://github.com/xbmc/xbmc/tree/master/addons/metadata.demo.tv metadata.demo.tv addon].

[[Category:Scraper]]

Python tv scraper development

2023-01-03T15:20:26Z

Roman V M:

{{mininav|[[Scrapers]] {{l2|[[Development]]}} }}

=Introduction=

Historically, Kodi has been supporting [https://kodi.wiki/view/HOW-TO:Write_media_scrapers XML scraping addons] that allow parsing online information sources
about movies, TV shows, music and so on. However, this approach has its limitations. First, XML parsing definitions with regular expressions are difficult to write and maintain.
Second, many information sources implemented REST APIs for getting information and regular expressions are not suitable for parsing JSON data. That is why Python
scrapers have been introduced.

Python scrapers are written in Python language and have similar structure to [https://kodi.wiki/view/HOW-TO:Video_addon media addons] (media plugins).
They are also called by special <code>plugin://</code> URLs with query parameters. The main query parameter is <code>action</code> that defines a scraping stage.
An example of a URL query sting passed to a scraper plugin: <code>?action=getdetails&url=foo&pathSettings=%7B%22foo%22%3A+%22bar%22%7D</code>.
Like other plugins, scrapers are also interact with Kodi via the functions of <code>xbmcplugin</code> Python API module and pass information to Kodi through <code>xbmcgui.ListItem</code> instances.
Basically, a Python scraper is a media plugin that passes information to the Kodi database instead of presenting lists of playable media items.

A TV shows scraper must support "action" calls that are described below.

=Actions=

==Find==

The <code>find</code> action is used for searching for a specific TV show by title and optionally a year that are passed as additional query parameters of the plugin call.
This action should use <code>xbmcplugin.addDirectoryItem</code> or <code>xbmcplugin.addDirectoryItems</code> to pass <code>xbmcgui.ListItem</code> instances to Kodi.
If only one instance is passed then it is considered as a perfect match. Otherwise the media file won't be matched and will need to be resolved manually by selecting
a necessary item from a list.
The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''label''': passed as <code>label</code> parameter to the class constructor. This is the label that is presented to a user during scraping.

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary TV show info from data provider's
website or API. It can be, for example, a link to a TV show page on a TV information website or a some unique ID to request TV show information from a REST API.

* '''thumb''' (optional): passed via <code>setArtwork()</code> of <code>xbmcgui.ListItem</code> class instance method. This should be a URL of a TV show poster, for example.

==NfoUrl (TV show)==

The <code>NfoUrl</code> action is called as an alternative to <code>find</code> if <code>tvshow.nfo</code> file is present in a TV show directory.
The entire .NFO file contents are passed as <code>nfo</code>parameter. This action should use <code>xbmcplugin.addDirectoryItem</code> to pass a single <code>xbmcgui.ListItem</code> instance to Kodi.

The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary TV show info from data provider's
website or API. It can be, for example, a link to a TV show page on a TV show information website or a some unique ID to request TV show information from a REST API.

* '''Unique IDs''': set via <code>ListItem.setUniqueIDs()</code> instance method. A scraper should set at least the default unique ID from the TV information site it works with.
Optionally it can set unique IDs from other online TV show databases.

If <code>tvshow.nfo</code> contains all the necessary info about the respective TV show in XML format then this actions may not pass any <code>xbmcgui.ListItem</code> instance to Kodi.
In this case Kodi will use all the information parsed from the XML file and will not call '''getdetails''' action.

==NfoUrl (episode)==

The <code>NfoUrl</code> action is also called as for each episode .NFO file that is present in a TV show directory.
The entire .NFO file contents are passed as <code>nfo</code>parameter. This action should use <code>xbmcplugin.addDirectoryItem</code> to pass a single <code>xbmcgui.ListItem</code> instance to Kodi.

The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary episode info from data provider's
website or API. It can be, for example, a link to the episode page on a TV show information website or some unique ID to request episode information from a REST API.

If an episode .NFO file contains all the necessary info about the respective episode in XML format then this action may not pass any <code>xbmcgui.ListItem</code> instance to Kodi.
In this case Kodi will use all the information parsed from the XML file and will not call '''getepisodedetails''' action.

==getdetails==

The <code>getdetails</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''url''' query parameter
from the previous stages and should set as much information to the <code>xbmcgui.ListItem</code> instance as possible using appropriate methods.
One of the necessary properties that are set via <code>ListItem.setInfo</code> method is <code>episodeguide</code>. This should be some unique string that can be used to retrieve
the list of TV show episoded with all the necessary info.

==getepisodelist==

The <code>getepisodelist</code> action should use <code>xbmcplugin.addDirectoryItem</code> or <code>xbmcplugin.addDirectoryItems</code> to pass <code>xbmcgui.ListItem</code> instances to Kodi
with information about available TV show episodes. This action receives '''url''' query parameter that is <code>episodeguide</code> property set by <code>getdetails</code> or <code>NfoUrl</code>action.

The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''url''': passed as <code>url</code> parameter to the class constructor. This is some unique string that can be used to retrieve information about a specific episode.

* '''season''': season number that is passed via <code>ListItem.setInfo</code> method.

* '''episode''': episode number that is passed <code>ListItem.setInfo</code> method.

* '''aired''': episode air date (if available) that is passed <code>ListItem.setInfo</code> method.

==getepisodedetails==

The <code>getepisodedetails</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''url''' query parameter
from the previous stage and should set as much information to the <code>xbmcgui.ListItem</code> instance as possible using appropriate methods.

==getartwork==

The <code>getartwork</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''id''' query parameter
that is the default unique ID set by '''NfoUrl''' or '''getdetails''' actions or parsed form an XML <code>tvshow.nfo</code> file if the scraper haven't set any. This action should set available artwork using <code>ListItem.addAvailableArtwork()</code> and <code>ListItem.setAvailableFanart()</code> instance methods.

'''Warning''': if your .NFO file contains the default unique ID that is different from the ID of the TV show database your scraper works with,
you must set the correct default unique ID in <code>NfoUrl</code> action. Otherwise <code>getdetails</code> will receive the default unique ID from your .NFO file
and your scraper may not be able to find any artwork using this ID.

=episodeguide=

The <code>episodeguide</code> string is passed to <code>getepisodedetails</code> action to retrieve the list of episodes. In legacy XML scrapers it was an actual URL of a page that was supposed to contain
the list of TV show episodes. Now in Python scrapers it can be any string that allows to correctly identify a TV show and retrieve the list of its episodes. However, the commonly accepted convention for <code>episodeguide</code> is that it should be a JSON-encoded string containing IDs of the specific TV show in various online databases. As a minimum it should include the ID of a show at the database
the scraper works with. An example of a JSON-encoded <code>episodeguide</code> string:

<code>{"tvmaze": "60", "tvrage": "4628", "tvdb": "72108", "imdb": "tt0364845"}</code>

Note that all IDs should be strings.

=Path Settings=

A scraper receives the current path settings via <code>pathSettings</code> query parameters as a JSON-encoded string with each action call.

=Example TV Shows Scraper=

A very high-level example of a Python TV show scraper can be found in the Kodi source code as [https://github.com/xbmc/xbmc/tree/master/addons/metadata.demo.tv metadata.demo.tv addon].

[[Category:Scraper]]

HOW-TO:Debug Python Scripts with Web-PDB

2022-09-02T17:53:13Z

Roman V M:

{{mininav|[[Development]]|[[Add-on development]]|[[Python development]]}}

[https://github.com/romanvm/kodi.web-pdb Web-PDB] is a remote web-interface to Python's built-in [https://docs.python.org/3/library/pdb.html PDB] debugger with additional convenience features. It is not tied to any IDE or other software, all you need is a common web-browser, e.g. Chrome or Firefox. Web-PDB is compatible with both Python 2 and 3, so you can use it to debug your Python 3 compatible addons.

[[File:web-pdb.png]]

Web-PDB for Kodi is available as an addon in the official Kodi addons repo.

== How To Use Web-PDB for Kodi ==

1. Install Web-PDB addon: '''Kodi Add-on repository > Program add-ons > Web-PDB'''.

2. Add <code>script.module.web-pdb</code> to [[addon.xml]] as a dependency:
<syntaxhighlight lang="xml" enclose="div">
<requires>
...
<import addon="script.module.web-pdb" version="1.5.6"/>
<requires>
</syntaxhighlight>

3. Restart Kodi so that it re-reads addon dependencies.

4. Insert the following line into your addon code at the point where you want to start debugging:
<syntaxhighlight lang="python" enclose="div">
import web_pdb; web_pdb.set_trace()
</syntaxhighlight>

The <code>set_trace()</code> call will suspend your addon and open a web-UI at the default port 5555 (port value can be changed). At the same time a notification will be displayed in Kodi, indicating that a debug session is active. The notification also shows web-UI host/port.

5. Enter in your the address bar of your browser: <code>http://<your Kodi machine hostname or IP>:5555</code>, for example <code>http://monty-python:5555</code>. Use <code>localhost</code> as a hostname if you are connecting from the same machine that runs Kodi. If everything is OK, you should see the Web-PDB UI. Now you can use all PDB commands and features. Additional '''Current file''', '''Globals''' and '''Locals''' information boxes help you better track your program runtime state.

== More Information ==

* [https://github.com/romanvm/kodi.web-pdb Web-PDB for Kodi on GitHub].
* [https://docs.python.org/3.10/library/pdb.html PDB debugger documentation].

[[Category:Development]]
[[Category:Python]]

Python Problems

2022-09-01T19:44:30Z

Roman V M:

{{Mininav|[[Development]]|[[Add-on development]]|[[About Add-ons]]}}<br />

This page is meant to document Python issues of general interest to Kodi addon developers.

== datetime.strptime ==
There is an old [[Python debugging|Python bug]]<ref name="bug_27400" /> which ''only'' impacts embedded Python applications, such as the Kodi Python environment. The issue is that <code>datetime.strptime</code><ref name="pydocs_datetime" /> is only initialized once per process, and not every time the embedded environment is reinitialized. This causes <code>datetime.strptime</code> to return <code>None</code> (and perhaps other strange behavior).

=== Workaround ===
The workaround is to monkey-patch <code>datetime.strptime</code> so that any user of the Python runtime will use it. It is more voodoo-like, but situations like this are why Python ''natively supports'' [[wikipedia:Monkey patch|monkey patching]], after all. The typically excellent Python documentation manages to be both thorough and concise simultaneously with regards to <code>datetime.strptime</code> and is well worth reviewing before deciding which angle of attack best suits your use case.<ref name="pydocs_strptime" />; it discusses some of the differences between <code>datetime.strptime</code> and <code>time.strptime</code>.<ref name="pydocs_differences" />

=== Patch ===
This patch simply replaces <code>datetime.strptime</code> with <code>time.strptime</code> as they are nearly identical in function. The original Kodi-specific implementation and its commit history are available on GitHub as part of <code>script.module.kutils</code>.<ref name="kutils_source" /> Essentially, the patch is:
<syntaxhighlight lang="python">
import datetime
import time

class proxydt(datetime.datetime):

@classmethod
def strptime(cls, date_string, format):
return datetime.datetime(*(time.strptime(date_string, format)[:6]))

datetime.datetime = proxydt
</syntaxhighlight>

== asyncio ==

It is a known [https://github.com/python/cpython/issues/91375 issue] that <code>asyncio</code> module or rather its C-based implementation does not support embedded Python environments that use sub-interpreters, such as Kodi Python environment. Essentially it means that only one addon can start the event loop by calling <code>asyncio.run()</code> and others will fail with <code>RuntimeError</code>.

=== Workaround ===

The workaround is to disable C-based <code>asyncio</code> module and use its pure-Python implementation. This can be done by the following code that should be put at the beginning of your addon entrypoint script:
<syntaxhighlight lang="python">
import sys
sys.modules['_asyncio'] = None
</syntaxhighlight>

You will loose possible performance benefits of <code>asyncio</code> but your async Python code won't fail.

== References ==
<div class="plainlinks"><references>
<ref name="bug_27400">Weinberg, Denny ({{#Dateformat:2016-06-27|mdy}}). [https://github.com/python/cpython/issues/71587 '''Issue #71587: Datetime NoneType after calling Py_Finalize and Py_Initialize''']. [https://github.com/python/cpython/issues ''Python Issue Tracker''] on [https://github.com/ GitHub].</ref>
<ref name="pydocs_datetime">[https://docs.python.org/3/ <span style="text-decoration: underline;">Python 3 Documentation</span>] ({{#dateformat:2022-08-28|mdy}}). [https://docs.python.org/3/library/index.html The Python Standard Library] » [https://docs.python.org/3/library/datatypes.html Data Types] » [https://docs.python.org/3/library/datetime.html#module-datetime datetime — Basic date and time types] ― <span style="font-weight: 600;">[https://docs.python.org/3/library/datetime.html#datetime.datetime.strptime ''classmethod'' <kbd>datetime.'''strptime'''(''date_string, format'')</kbd>]</span> ― from the source code found in [https://github.com/python/cpython/blob/main/Lib/datetime.py Lib/datetime.py].</ref>
<ref name="pydocs_strptime">[https://docs.python.org/3/ <span style="text-decoration: underline;">Python 3 Documentation</span>] ({{#dateformat:2022-08-28|mdy}}). datetime — Basic date and time types » <span style="font-weight: 600;">[https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behavior <kbd>strftime()</kbd> and <kbd>strptime()</kbd> Behavior]</span>.</ref>
<ref name="pydocs_differences">[https://docs.python.org/3/ <span style="text-decoration: underline;">Python 3 Documentation</span>] ({{#dateformat:2022-08-28|mdy}}). <kbd>strftime()</kbd> and <kbd>strptime()</kbd> Behavior » <span style="font-weight: 600;">[https://docs.python.org/3.10/library/datetime.html#technical-detail Technical Detail]</span>.</ref>
<ref name="kutils_source"><span style="font-weight: 600;">[https://github.com/fbacher/script.module.kutils/blob/master/lib/kutils/strptime_patch.py <kbd>lib/kutils/strptime_patch.py</kbd>]</span> ({{#dateformat:2022-01-11|mdy}}) in <code>script.module.kutils</code>, ''"A helper module for Kodi development"'' on GitHub.</ref>
</references></div>

[[Category:Python]]

Python tv scraper development

2022-08-13T22:17:28Z

Roman V M:

{{mininav|[[Scrapers]] {{l2|[[Development]]}} }}

=Introduction=

Historically, Kodi has been supporting [https://kodi.wiki/view/HOW-TO:Write_media_scrapers XML scraping addons] that allow parsing online information sources
about movies, TV shows, music and so on. However, this approach has its limitations. First, XML parsing definitions with regular expressions are difficult to write and maintain.
Second, many information sources implemented REST APIs for getting information and regular expressions are not suitable for parsing JSON data. That is why Python
scrapers have been introduced.

Python scrapers are written in Python language and have similar structure to [https://kodi.wiki/view/HOW-TO:Video_addon media addons] (media plugins).
They are also called by special <code>plugin://</code> URLs with query parameters. The main query parameter is <code>action</code> that defines a scraping stage.
An example of a URL query sting passed to a scraper plugin: <code>?action=getdetails&url=foo&pathSettings=%7B%22foo%22%3A+%22bar%22%7D</code>.
Like other plugins, scrapers are also interact with Kodi via the functions of <code>xbmcplugin</code> Python API module and pass information to Kodi through <code>xbmcgui.ListItem</code> instances.
Basically, a Python scraper is a media plugin that passes information to the Kodi database instead of presenting lists of playable media items.

A TV shows scraper must support "action" calls that are described below.

=Actions=

==Find==

The <code>find</code> action is used for searching for a specific TV show by title and optionally a year that are passed as additional query parameters of the plugin call.
This action should use <code>xbmcplugin.addDirectoryItem</code> or <code>xbmcplugin.addDirectoryItems</code> to pass <code>xbmcgui.ListItem</code> instances to Kodi.
If only one instance is passed then it is considered as a perfect match. Otherwise the media file won't be matched and will need to be resolved manually by selecting
a necessary item from a list.
The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''label''': passed as <code>label</code> parameter to the class constructor. This is the label that is presented to a user during scraping.

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary TV show info from data provider's
website or API. It can be, for example, a link to a TV show page on a TV information website or a some unique ID to request TV show information from a REST API.

* '''thumb''' (optional): passed via <code>setArtwork()</code> of <code>xbmcgui.ListItem</code> class instance method. This should be a URL of a TV show poster, for example.

==NfoUrl (TV show)==

The <code>NfoUrl</code> action is called as an alternative to <code>find</code> if <code>tvshow.nfo</code> file is present in a TV show directory.
The entire .NFO file contents are passed as <code>nfo</code>parameter. This action should use <code>xbmcplugin.addDirectoryItem</code> to pass a single <code>xbmcgui.ListItem</code> instance to Kodi.

The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary TV show info from data provider's
website or API. It can be, for example, a link to a TV show page on a TV show information website or a some unique ID to request TV show information from a REST API.

* '''Unique IDs''': set via <code>ListItem.setUniqueIDs()</code> instance method. A scraper should set at least the default unique ID from the TV information site it works with.
Optionally it can set unique IDs from other online TV show databases.

If <code>tvshow.nfo</code> contains all the necessary info about the respective TV show in XML format then this actions may not pass any <code>xbmcgui.ListItem</code> instance to Kodi.
In this case Kodi will use all the information parsed from the XML file and will not call '''getdetails''' action.

==NfoUrl (episode)==

The <code>NfoUrl</code> action is also called as for each episode .NFO file that is present in a TV show directory.
The entire .NFO file contents are passed as <code>nfo</code>parameter. This action should use <code>xbmcplugin.addDirectoryItem</code> to pass a single <code>xbmcgui.ListItem</code> instance to Kodi.

The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary episode info from data provider's
website or API. It can be, for example, a link to the episode page on a TV show information website or some unique ID to request episode information from a REST API.

If an episode .NFO file contains all the necessary info about the respective episode in XML format then this action may not pass any <code>xbmcgui.ListItem</code> instance to Kodi.
In this case Kodi will use all the information parsed from the XML file and will not call '''getepisodedetails''' action.

==getdetails==

The <code>getdetails</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''url''' query parameter
from the previous stages and should set as much information to the <code>xbmcgui.ListItem</code> instance as possible using appropriate methods.
One of the necessary properties that are set via <code>ListItem.setInfo</code> method is <code>episodeguide</code>. This should be some unique string that can be used to retrieve
the list of TV show episoded with all the necessary info.

==getepisodelist==

The <code>getepisodelist</code> action should use <code>xbmcplugin.addDirectoryItem</code> or <code>xbmcplugin.addDirectoryItems</code> to pass <code>xbmcgui.ListItem</code> instances to Kodi
with information about available TV show episodes. This action receives '''url''' query parameter that is <code>episodeguide</code> property set by <code>getdetails</code> action.

The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''url''': passed as <code>url</code> parameter to the class constructor. This is some unique string that can be used to retrieve information about a specific episode.

* '''season''': season number that is passed via <code>ListItem.setInfo</code> method.

* '''episode''': episode number that is passed <code>ListItem.setInfo</code> method.

* '''aired''': episode air date (if available) that is passed <code>ListItem.setInfo</code> method.

==getepisodedetails==

The <code>getepisodedetails</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''url''' query parameter
from the previous stage and should set as much information to the <code>xbmcgui.ListItem</code> instance as possible using appropriate methods.

==getartwork==

The <code>getartwork</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''id''' query parameter
that is the default unique ID set by '''NfoUrl''' or '''getdetails''' actions or parsed form an XML <code>tvshow.nfo</code> file if the scraper haven't set any. This action should set available artwork using <code>ListItem.addAvailableArtwork()</code> and <code>ListItem.setAvailableFanart()</code> instance methods.

'''Warning''': if your .NFO file contains the default unique ID that is different from the ID of the TV show database your scraper works with,
you must set the correct default unique ID in <code>NfoUrl</code> action. Otherwise <code>getdetails</code> will receive the default unique ID from your .NFO file
and your scraper may not be able to find any artwork using this ID.

=Path Settings=

A scraper receives the current path settings via <code>pathSettings</code> query parameters as a JSON-encoded string with each action call.

=Example TV Shows Scraper=

A very high-level example of a Python TV show scraper can be found in the Kodi source code as [https://github.com/xbmc/xbmc/tree/master/addons/metadata.demo.tv metadata.demo.tv addon].

[[Category:Scraper]]

Python movie scraper development

2022-08-05T11:47:33Z

Roman V M:

Python movie scraper development

2022-08-05T11:47:03Z

Roman V M:

Python tv scraper development

2022-08-04T20:54:26Z

Roman V M:

{{mininav|[[Scrapers]] {{l2|[[Development]]}} }}

=Introduction=

Historically, Kodi has been supporting [https://kodi.wiki/view/HOW-TO:Write_media_scrapers XML scraping addons] that allow parsing online information sources
about movies, TV shows, music and so on. However, this approach has its limitations. First, XML parsing definitions with regular expressions are difficult to write and maintain.
Second, many information sources implemented REST APIs for getting information and regular expressions are not suitable for parsing JSON data. That is why Python
scrapers have been introduced.

Python scrapers are written in Python language and have similar structure to [https://kodi.wiki/view/HOW-TO:Video_addon media addons] (media plugins).
They are also called by special <code>plugin://</code> URLs with query parameters. The main query parameter is <code>action</code> that defines a scraping stage.
An example of a URL query sting passed to a scraper plugin: <code>?action=getdetails&url=foo&pathSettings=%7B%22foo%22%3A+%22bar%22%7D</code>.
Like other plugins, scrapers are also interact with Kodi via the functions of <code>xbmcplugin</code> Python API module and pass information to Kodi through <code>xbmcgui.ListItem</code> instances.
Basically, a Python scraper is a media plugin that passes information to the Kodi database instead of presenting lists of playable media items.

A TV shows scraper must support "action" calls that are described below.

=Actions=

==Find==

The <code>find</code> action is used for searching for a specific TV show by title and optionally a year that are passed as additional query parameters of the plugin call.
This action should use <code>xbmcplugin.addDirectoryItem</code> or <code>xbmcplugin.addDirectoryItems</code> to pass <code>xbmcgui.ListItem</code> instances to Kodi.
If only one instance is passed then it is considered as a perfect match. Otherwise the media file won't be matched and will need to be resolved manually by selecting
a necessary item from a list.
The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''label''': passed as <code>label</code> parameter to the class constructor. This is the label that is presented to a user during scraping.

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary TV show info from data provider's
website or API. It can be, for example, a link to a TV show page on a TV information website or a some unique ID to request TV show information from a REST API.

* '''thumb''' (optional): passed via <code>setArtwork()</code> of <code>xbmcgui.ListItem</code> class instance method. This should be a URL of a TV show poster, for example.

==NfoUrl (TV show)==

The <code>NfoUrl</code> action is called as an alternative to <code>find</code> if <code>tvshow.nfo</code> file is present in a TV show directory.
The entire .NFO file contents are passed as <code>nfo</code>parameter. This action should use <code>xbmcplugin.addDirectoryItem</code> to pass a single <code>xbmcgui.ListItem</code> instance to Kodi.

The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary TV show info from data provider's
website or API. It can be, for example, a link to a TV show page on a TV show information website or a some unique ID to request TV show information from a REST API.

* '''Unique IDs''': set via <code>ListItem.setUniqueIDs()</code> instance method. A scraper should set at least the default unique ID from the TV information site it works with.
Optionally it can set unique IDs from other online TV show databases.

If <code>tvshow.nfo</code> contains all the necessary info about the respective TV show in XML format then this actions may not pass any <code>xbmcgui.ListItem</code> instance to Kodi.
In this case Kodi will use all the information parsed from the XML file and will not call '''getdetails''' action.

==NfoUrl (episode)==

The <code>NfoUrl</code> action is also called as for each episode .NFO file that is present in a TV show directory.
The entire .NFO file contents are passed as <code>nfo</code>parameter. This action should use <code>xbmcplugin.addDirectoryItem</code> to pass a single <code>xbmcgui.ListItem</code> instance to Kodi.

The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary episode info from data provider's
website or API. It can be, for example, a link to the episode page on a TV show information website or some unique ID to request episode information from a REST API.

If an episode .NFO file contains all the necessary info about the respective episode in XML format then this action may not pass any <code>xbmcgui.ListItem</code> instance to Kodi.
In this case Kodi will use all the information parsed from the XML file and will not call '''getepisodedetails''' action.

==getdetails==

The <code>getdetails</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''url''' query parameter
from the previous stages and should set as much information to the <code>xbmcgui.ListItem</code> instance as possible using appropriate methods.
One of the necessary properties that are set via <code>ListItem.setInfo</code> method is <code>episodeguide</code>. This should be some unique string that can be used to retrieve
the list of TV show episoded with all the necessary info.

==getepisodelist==

The <code>getepisodelist</code> action should use <code>xbmcplugin.addDirectoryItem</code> or <code>xbmcplugin.addDirectoryItems</code> to pass <code>xbmcgui.ListItem</code> instances to Kodi
with information about available TV show episodes. This action receives '''url''' query parameter that is <code>episodeguide</code> property set by <code>getdetails</code> action.
A scraper needs to set '''url''' for each <code>xbmcgui.ListItem</code> instance that is some unique string that can be used to retrieve information about a specific episode.

==getepisodedetails==

The <code>getepisodedetails</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''url''' query parameter
from the previous stage and should set as much information to the <code>xbmcgui.ListItem</code> instance as possible using appropriate methods.

==getartwork==

The <code>getartwork</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''id''' query parameter
that is the default unique ID set by '''NfoUrl''' or '''getdetails''' actions or parsed form an XML <code>tvshow.nfo</code> file if the scraper haven't set any. This action should set available artwork using <code>ListItem.addAvailableArtwork()</code> and <code>ListItem.setAvailableFanart()</code> instance methods.

'''Warning''': if your .NFO file contains the default unique ID that is different from the ID of the TV show database your scraper works with,
you must set the correct default unique ID in <code>NfoUrl</code> action. Otherwise <code>getdetails</code> will receive the default unique ID from your .NFO file
and your scraper may not be able to find any artwork using this ID.

=Path Settings=

A scraper receives the current path settings via <code>pathSettings</code> query parameters as a JSON-encoded string with each action call.

=Example TV Shows Scraper=

A very high-level example of a Python TV show scraper can be found in the Kodi source code as [https://github.com/xbmc/xbmc/tree/master/addons/metadata.demo.tv metadata.demo.tv addon].

[[Category:Scraper]]

Python tv scraper development

2022-08-04T20:53:33Z

Roman V M:

{{mininav|[[Scrapers]] {{l2|[[Development]]}} }}

=Introduction=

Historically, Kodi has been supporting [https://kodi.wiki/view/HOW-TO:Write_media_scrapers XML scraping addons] that allow parsing online information sources
about movies, TV shows, music and so on. However, this approach has its limitations. First, XML parsing definitions with regular expressions are difficult to write and maintain.
Second, many information sources implemented REST APIs for getting information and regular expressions are not suitable for parsing JSON data. That is why Python
scrapers have been introduced.

Python scrapers are written in Python language and have similar structure to [https://kodi.wiki/view/HOW-TO:Video_addon media addons] (media plugins).
They are also called by special <code>plugin://</code> URLs with query parameters. The main query parameter is <code>action</code> that defines a scraping stage.
An example of a URL query sting passed to a scraper plugin: <code>?action=getdetails&url=foo&pathSettings=%7B%22foo%22%3A+%22bar%22%7D</code>.
Like other plugins, scrapers are also interact with Kodi via the functions of <code>xbmcplugin</code> Python API module and pass information to Kodi through <code>xbmcgui.ListItem</code> instances.
Basically, a Python scraper is a media plugin that passes information to the Kodi database instead of presenting lists of playable media items.

A TV shows scraper must support "action" calls that are described below.

=Actions=

==Find==

The <code>find</code> action is used for searching for a specific TV show by title and optionally a year that are passed as additional query parameters of the plugin call.
This action should use <code>xbmcplugin.addDirectoryItem</code> or <code>xbmcplugin.addDirectoryItems</code> to pass <code>xbmcgui.ListItem</code> instances to Kodi.
If only one instance is passed then it is considered as a perfect match. Otherwise the media file won't be matched and will need to be resolved manually by selecting
a necessary item from a list.
The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''label''': passed as <code>label</code> parameter to the class constructor. This is the label that is presented to a user during scraping.

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary TV show info from data provider's
website or API. It can be, for example, a link to a TV show page on a TV information website or a some unique ID to request TV show information from a REST API.

* '''thumb''' (optional): passed via <code>setArtwork()</code> of <code>xbmcgui.ListItem</code> class instance method. This should be a URL of a TV show poster, for example.

==NfoUrl (TV show)==

The <code>NfoUrl</code> action is called as an alternative to <code>find</code> if <code>tvshow.nfo</code> file is present in a TV show directory.
The entire .NFO file contents are passed as <code>nfo</code>parameter. This action should use <code>xbmcplugin.addDirectoryItem</code> to pass a single <code>xbmcgui.ListItem</code> instance to Kodi.

The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary TV show info from data provider's
website or API. It can be, for example, a link to a TV show page on a TV show information website or a some unique ID to request TV show information from a REST API.

* '''Unique IDs''': set via <code>ListItem.setUniqueIDs()</code> instance method. A scraper should set at least the default unique ID from the TV information site it works with.
Optionally it can set unique IDs from other online TV show databases.
If <code>tvshow.nfo</code> contains all the necessary info about the respective TV show in XML format then this actions may not pass any <code>xbmcgui.ListItem</code> instance to Kodi.
In this case Kodi will use all the information parsed from the XML file and will not call '''getdetails''' action.

==NfoUrl (episode)==

The <code>NfoUrl</code> action is also called as for each episode .NFO file that is present in a TV show directory.
The entire .NFO file contents are passed as <code>nfo</code>parameter. This action should use <code>xbmcplugin.addDirectoryItem</code> to pass a single <code>xbmcgui.ListItem</code> instance to Kodi.

The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary episode info from data provider's
website or API. It can be, for example, a link to the episode page on a TV show information website or some unique ID to request episode information from a REST API.

If an episode .NFO file contains all the necessary info about the respective episode in XML format then this action may not pass any <code>xbmcgui.ListItem</code> instance to Kodi.
In this case Kodi will use all the information parsed from the XML file and will not call '''getepisodedetails''' action.

==getdetails==

The <code>getdetails</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''url''' query parameter
from the previous stages and should set as much information to the <code>xbmcgui.ListItem</code> instance as possible using appropriate methods.
One of the necessary properties that are set via <code>ListItem.setInfo</code> method is <code>episodeguide</code>. This should be some unique string that can be used to retrieve
the list of TV show episoded with all the necessary info.

==getepisodelist==

The <code>getepisodelist</code> action should use <code>xbmcplugin.addDirectoryItem</code> or <code>xbmcplugin.addDirectoryItems</code> to pass <code>xbmcgui.ListItem</code> instances to Kodi
with information about available TV show episodes. This action receives '''url''' query parameter that is <code>episodeguide</code> property set by <code>getdetails</code> action.
A scraper needs to set '''url''' for each <code>xbmcgui.ListItem</code> instance that is some unique string that can be used to retrieve information about a specific episode.

==getepisodedetails==

The <code>getepisodedetails</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''url''' query parameter
from the previous stage and should set as much information to the <code>xbmcgui.ListItem</code> instance as possible using appropriate methods.

==getartwork==

The <code>getartwork</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''id''' query parameter
that is the default unique ID set by '''NfoUrl''' or '''getdetails''' actions or parsed form an XML <code>tvshow.nfo</code> file if the scraper haven't set any. This action should set available artwork using <code>ListItem.addAvailableArtwork()</code> and <code>ListItem.setAvailableFanart()</code> instance methods.

'''Warning''': if your .NFO file contains the default unique ID that is different from the ID of the TV show database your scraper works with,
you must set the correct default unique ID in <code>NfoUrl</code> action. Otherwise <code>getdetails</code> will receive the default unique ID from your .NFO file
and your scraper may not be able to find any artwork using this ID.

=Path Settings=

A scraper receives the current path settings via <code>pathSettings</code> query parameters as a JSON-encoded string with each action call.

=Example TV Shows Scraper=

A very high-level example of a Python TV show scraper can be found in the Kodi source code as [https://github.com/xbmc/xbmc/tree/master/addons/metadata.demo.tv metadata.demo.tv addon].

[[Category:Scraper]]

Python movie scraper development

2022-06-19T18:50:40Z

Roman V M:

Python tv scraper development

2022-06-19T18:50:17Z

Roman V M:

Python tv scraper development

2022-06-19T07:14:02Z

Roman V M:

{{mininav|[[Scrapers]] {{l2|[[Development]]}} }}

{{Wiki_revamp}}

=Introduction=

Historically, Kodi has been supporting [https://kodi.wiki/view/HOW-TO:Write_media_scrapers XML scraping addons] that allow parsing online information sources
about movies, TV shows, music and so on. However, this approach has its limitations. First, XML parsing definitions with regular expressions are difficult to write and maintain.
Second, many information sources implemented REST APIs for getting information and regular expressions are not suitable for parsing JSON data. That is why Python
scrapers have been introduced.

Python scrapers are written in Python language and have similar structure to [https://kodi.wiki/view/HOW-TO:Video_addon media addons] (media plugins).
They are also called by special <code>plugin://</code> URLs with query parameters. The main query parameter is <code>action</code> that defines a scraping stage.
An example of a URL query sting passed to a scraper plugin: <code>?action=getdetails&url=foo&pathSettings=%7B%22foo%22%3A+%22bar%22%7D</code>.
Like other plugins, scrapers are also interact with Kodi via the functions of <code>xbmcplugin</code> Python API module and pass information to Kodi through <code>xbmcgui.ListItem</code> instances.
Basically, a Python scraper is a media plugin that passes information to the Kodi database instead of presenting lists of playable media items.

A TV shows scraper must support "action" calls that are described below.

=Actions=

==Find==

The <code>find</code> action is used for searching for a specific TV show by title and optionally a year that are passed as additional query parameters of the plugin call.
This action should use <code>xbmcplugin.addDirectoryItem</code> or <code>xbmcplugin.addDirectoryItems</code> to pass <code>xbmcgui.ListItem</code> instances to Kodi.
If only one instance is passed then it is considered as a perfect match. Otherwise the media file won't be matched and will need to be resolved manually by selecting
a necessary item from a list.
The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''label''': passed as <code>label</code> parameter to the class constructor. This is the label that is presented to a user during scraping.

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary TV show info from data provider's
website or API. It can be, for example, a link to a TV show page on a TV information website or a some unique ID to request TV show information from a REST API.

* '''thumb''' (optional): passed via <code>setArtwork()</code> of <code>xbmcgui.ListItem</code> class instance method. This should be a URL of a TV show poster, for example.

==NfoUrl==

The <code>NfoUrl</code> action is called as an alternative to <code>find</code> if <code>tvshow.nfo</code> file is present in a TV show directory.
The entire .NFO file contents are passed as <code>nfo</code>parameter. This action should use <code>xbmcplugin.addDirectoryItem</code> to pass a single <code>xbmcgui.ListItem</code> instance to Kodi.

The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary TV show info from data provider's
website or API. It can be, for example, a link to a TV show page on a TV show information website or a some unique ID to request TV show information from a REST API.

* '''Unique IDs''': set via <code>ListItem.setUniqueIDs()</code> instance method. A scraper should set at least the default unique ID from the TV information site it works with.
Optionally it can set unique IDs from other online TV show databases.

==getdetails==

The <code>getdetails</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''url''' query parameter
from the previous stages and should set as much information to the <code>xbmcgui.ListItem</code> instance as possible using appropriate methods.
One of the necessary properties that are set via <code>ListItem.setInfo</code> method is <code>episodeguide</code>. This should be some unique string that can be used to retrieve
the list of TV show episoded with all the necessary info.

==getepisodelist==

The <code>getepisodelist</code> action should use <code>xbmcplugin.addDirectoryItem</code> or <code>xbmcplugin.addDirectoryItems</code> to pass <code>xbmcgui.ListItem</code> instances to Kodi
with information about available TV show episodes. This action receives '''url''' query parameter that is <code>episodeguide</code> property set by <code>getdetails</code> action.
A scraper needs to set '''url''' for each <code>xbmcgui.ListItem</code> instance that is some unique string that can be used to retrieve information about a specific episode.

==getepisodedetails==

The <code>getepisodedetails</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''url''' query parameter
from the previous stage and should set as much information to the <code>xbmcgui.ListItem</code> instance as possible using appropriate methods.

==getartwork==

The <code>getartwork</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''id''' query parameter
that is the default unique ID set by '''NfoUrl''' or '''getdetails''' actions. This action should set available artwork using <code>ListItem.addAvailableArtwork()</code>
and <code>ListItem.setAvailableFanart()</code> instance methods.

'''Warning''': if your .NFO file contains the default unique ID that is different from the ID of the TV show database your scraper works with,
you must set the correct default unique ID in <code>NfoUrl</code> action. Otherwise <code>getdetails</code> will receive the default unique ID from your .NFO file
and your scraper may not be able to find any artwork using this ID.

=Path Settings=

A scraper receives the current path settings via <code>pathSettings</code> query parameters as a JSON-encoded string with each action call.

=Example TV Shows Scraper=

A very high-level example of a Python TV show scraper can be found in the Kodi source code as [https://github.com/xbmc/xbmc/tree/master/addons/metadata.demo.tv metadata.demo.tv addon].

Python movie scraper development

2022-06-08T07:44:14Z

Roman V M:

{{mininav|[[Scrapers]] {{l2|[[Development]]}} }}

{{Wiki_revamp}}

=Introduction=

Historically, Kodi has been supporting [https://kodi.wiki/view/HOW-TO:Write_media_scrapers XML scraping addons] that allow parsing online information sources
about movies, TV shows, music and so on. However, this approach has its limitations. First, XML parsing definitions with regular expressions are difficult to write and maintain.
Second, many information sources implemented REST APIs for getting information and regular expressions are not suitable for parsing JSON data. That is why Python
scrapers have been introduced.

Python scrapers are written in Python language and have similar structure to [https://kodi.wiki/view/HOW-TO:Video_addon media addons] (media plugins).
They are also called by special <code>plugin://</code> URLs with query parameters. The main query parameter is <code>action</code> that defines a scraping stage.
An example of a URL query sting passed to a scraper plugin: <code>?action=getdetails&url=foo&pathSettings=%7B%22foo%22%3A+%22bar%22%7D</code>.
Like other plugins, scrapers are also interact with Kodi via the functions of <code>xbmcplugin</code> Python API module and pass information to Kodi through <code>xbmcgui.ListItem</code> instances.
Basically, a Python scraper is a media plugin that passes information to the Kodi database instead of presenting lists of playable media items.

A movie scraper must support "action" calls that are described below.

=Actions=

==Find==

The <code>find</code> action is used for searching for a specific movie by title and optionally a year that are passed as additional query parameters of the plugin call.
This action should use <code>xbmcplugin.addDirectoryItem</code> or <code>xbmcplugin.addDirectoryItems</code> to pass <code>xbmcgui.ListItem</code> instances to Kodi.
If only one instance is passed then it is considered as a perfect match. Otherwise the media file won't be matched and will need to be resolved manually by selecting
a necessary item from a list.
The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''label''': passed as <code>label</code> parameter to the class constructor. This is the label that is presented to a user during scraping.

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary movie info from data provider's
website or API. It can be, for example, a link to a movie page on a movie information website or a some unique ID to request movie information from a REST API.

* '''thumb''' (optional): passed via <code>setArtwork()</code> of <code>xbmcgui.ListItem</code> class instance method. This should be a URL of a movie poster, for example.

==NfoUrl==

The <code>NfoUrl</code> action is called as an alternative to <code>find</code> if an .NFO file is present along with a movie file. The entire .NFO file contents are passed as <code>nfo</code>
parameter. This action should use <code>xbmcplugin.addDirectoryItem</code> to pass a single <code>xbmcgui.ListItem</code> instance to Kodi.

The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary movie info from data provider's
website or API. It can be, for example, a link to a movie page on a movie information website or a some unique ID to request movie information from a REST API.

* '''Unique IDs''': set via <code>ListItem.setUniqueIDs()</code> instance method. A scraper should set at least the default unique ID from the movie information site it works with.
Optionally it can set unique IDs from other online movie databases.

==getdetails==

The <code>getdetails</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''url''' query parameter
from the previous stages and should set as much information to the <code>xbmcgui.ListItem</code> instance as possible using appropriate methods.

==getartwork==

The <code>getartwork</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''id''' query parameter
that is the default unique ID set by '''NfoUrl''' or '''getdetails''' actions. This action should set available artwork using <code>ListItem.addAvailableArtwork()</code>
and <code>ListItem.setAvailableFanart()</code> instance methods.

'''Warning''': if your .NFO file contains the default unique ID that is different from the ID of the movie database your scraper works with,
you must set the correct default unique ID in <code>NfoUrl</code> action. Otherwise <code>getdetails</code> will receive the default unique ID from your .NFO file
and your scraper may not be able to find any artwork using this ID.

=Path Settings=

A scraper receives the current path settings via <code>pathSettings</code> query parameters as a JSON-encoded string with each action call.

=Example Movie Scraper=

A very high-level example of a Python movie scraper can be found in the Kodi source code as [https://github.com/xbmc/xbmc/tree/master/addons/metadata.demo.movies metadata.demo.movie addon].

Python movie scraper development

2022-06-07T10:12:35Z

Roman V M: /* getartwork */

{{mininav|[[Scrapers]] {{l2|[[Development]]}} }}

{{Wiki_revamp}}

=Introduction=

Historically, Kodi has been supporting [https://kodi.wiki/view/HOW-TO:Write_media_scrapers XML scraping addons] that allow parsing online information sources
about movies, TV shows, music and so on. However, this approach has its limitations. First, XML parsing definitions with regular expressions are difficult to write and maintain.
Second, many information sources implemented REST APIs for getting information and regular expressions are not suitable for parsing JSON and data. That is why Python
scrapers have been introduced.

Python scrapers are written in Python language and have similar structure to [https://kodi.wiki/view/HOW-TO:Video_addon media addons] (media plugins).
They are also called by special <code>plugin://</code> URLs with query parameters. The main query parameter is <code>action</code> that defines a scraping stage.
An example of a URL query sting passed to a scraper plugin: <code>?action=getdetails&url=foo&pathSettings=%7B%22foo%22%3A+%22bar%22%7D</code>.
Like other plugins, scrapers are also interact with Kodi via the functions of <code>xbmcplugin</code> Python API module and pass information to Kodi through <code>xbmcgui.ListItem</code> instances.
Basically, a Python scraper is a media plugin that passes information to the Kodi database instead of presenting lists of playable media items.

A movie scraper must support "action" calls that are described below.

=Actions=

==Find==

The <code>find</code> action is used for searching for a specific movie by title and optionally a year that are passed as additional query parameters of the plugin call.
This action should use <code>xbmcplugin.addDirectoryItem</code> or <code>xbmcplugin.addDirectoryItems</code> to pass <code>xbmcgui.ListItem</code> instances to Kodi.
If only one instance is passed then it is considered as a perfect match. Otherwise the media file won't be matched and will need to be resolved manually by selecting
a necessary item from a list.
The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''label''': passed as <code>label</code> parameter to the class constructor. This is the label that is presented to a user during scraping.

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary movie info from data provider's
website or API. It can be, for example, a link to a movie page on a movie information website or a some unique ID to request movie information from a REST API.

* '''thumb''' (optional): passed via <code>setArtwork()</code> of <code>xbmcgui.ListItem</code> class instance method. This should be a URL of a movie poster, for example.

==NfoUrl==

The <code>NfoUrl</code> action is called as an alternative to <code>find</code> if an .NFO file is present along with a movie file. The entire .NFO file contents are passed as <code>nfo</code>
parameter. This action should use <code>xbmcplugin.addDirectoryItem</code> to pass a single <code>xbmcgui.ListItem</code> instance to Kodi.

The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary movie info from data provider's
website or API. It can be, for example, a link to a movie page on a movie information website or a some unique ID to request movie information from a REST API.

* '''Unique IDs''': set via <code>ListItem.setUniqueIDs()</code> instance method. A scraper should set at least the default unique ID from the movie information site it works with.
Optionally it can set unique IDs from other online movie databases.

==getdetails==

The <code>getdetails</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''url''' query parameter
from the previous stages and should set as much information to the <code>xbmcgui.ListItem</code> instance as possible using appropriate methods.

==getartwork==

The <code>getartwork</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''id''' query parameter
that is the default unique ID set by '''NfoUrl''' or '''getdetails''' actions. This action should set available artwork using <code>ListItem.addAvailableArtwork()</code>
and <code>ListItem.setAvailableFanart()</code> instance methods.

'''Warning''': if your .NFO file contains the default unique ID that is different from the ID of the movie database your scraper works with,
you must set the correct default unique ID in <code>NfoUrl</code> action. Otherwise <code>getdetails</code> will receive the default unique ID from your .NFO file
and your scraper may not be able to find any artwork using this ID.

=Path Settings=

A scraper receives the current path settings via <code>pathSettings</code> query parameters as a JSON-encoded string with each action call.

=Example Movie Scraper=

A very high-level example of a Python movie scraper can be found in the Kodi source code as [https://github.com/xbmc/xbmc/tree/master/addons/metadata.demo.movies metadata.demo.movie addon].

Python movie scraper development

2022-06-07T10:11:11Z

Roman V M: /* getdetails */

{{mininav|[[Scrapers]] {{l2|[[Development]]}} }}

{{Wiki_revamp}}

=Introduction=

Historically, Kodi has been supporting [https://kodi.wiki/view/HOW-TO:Write_media_scrapers XML scraping addons] that allow parsing online information sources
about movies, TV shows, music and so on. However, this approach has its limitations. First, XML parsing definitions with regular expressions are difficult to write and maintain.
Second, many information sources implemented REST APIs for getting information and regular expressions are not suitable for parsing JSON and data. That is why Python
scrapers have been introduced.

Python scrapers are written in Python language and have similar structure to [https://kodi.wiki/view/HOW-TO:Video_addon media addons] (media plugins).
They are also called by special <code>plugin://</code> URLs with query parameters. The main query parameter is <code>action</code> that defines a scraping stage.
An example of a URL query sting passed to a scraper plugin: <code>?action=getdetails&url=foo&pathSettings=%7B%22foo%22%3A+%22bar%22%7D</code>.
Like other plugins, scrapers are also interact with Kodi via the functions of <code>xbmcplugin</code> Python API module and pass information to Kodi through <code>xbmcgui.ListItem</code> instances.
Basically, a Python scraper is a media plugin that passes information to the Kodi database instead of presenting lists of playable media items.

A movie scraper must support "action" calls that are described below.

=Actions=

==Find==

The <code>find</code> action is used for searching for a specific movie by title and optionally a year that are passed as additional query parameters of the plugin call.
This action should use <code>xbmcplugin.addDirectoryItem</code> or <code>xbmcplugin.addDirectoryItems</code> to pass <code>xbmcgui.ListItem</code> instances to Kodi.
If only one instance is passed then it is considered as a perfect match. Otherwise the media file won't be matched and will need to be resolved manually by selecting
a necessary item from a list.
The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''label''': passed as <code>label</code> parameter to the class constructor. This is the label that is presented to a user during scraping.

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary movie info from data provider's
website or API. It can be, for example, a link to a movie page on a movie information website or a some unique ID to request movie information from a REST API.

* '''thumb''' (optional): passed via <code>setArtwork()</code> of <code>xbmcgui.ListItem</code> class instance method. This should be a URL of a movie poster, for example.

==NfoUrl==

The <code>NfoUrl</code> action is called as an alternative to <code>find</code> if an .NFO file is present along with a movie file. The entire .NFO file contents are passed as <code>nfo</code>
parameter. This action should use <code>xbmcplugin.addDirectoryItem</code> to pass a single <code>xbmcgui.ListItem</code> instance to Kodi.

The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary movie info from data provider's
website or API. It can be, for example, a link to a movie page on a movie information website or a some unique ID to request movie information from a REST API.

* '''Unique IDs''': set via <code>ListItem.setUniqueIDs()</code> instance method. A scraper should set at least the default unique ID from the movie information site it works with.
Optionally it can set unique IDs from other online movie databases.

==getdetails==

The <code>getdetails</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''url''' query parameter
from the previous stages and should set as much information to the <code>xbmcgui.ListItem</code> instance as possible using appropriate methods.

==getartwork==

The <code>getdetails</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''id''' query parameter
that is the default unique ID set by '''NfoUrl''' or '''getdetails''' actions. This action should set available artwork using <code>ListItem.addAvailableArtwork()</code>
and <code>ListItem.setAvailableFanart()</code> instance methods.

'''Warning''': if your .NFO file contains the default unique ID that is different from the ID of the movie database your scraper works with,
you must set the correct default unique ID in <code>NfoUrl</code> action. Otherwise <code>getdetails</code> will receive the default unique ID from your .NFO file
and your scraper may not be able to find any artwork using this ID.

=Path Settings=

A scraper receives the current path settings via <code>pathSettings</code> query parameters as a JSON-encoded string with each action call.

=Example Movie Scraper=

A very high-level example of a Python movie scraper can be found in the Kodi source code as [https://github.com/xbmc/xbmc/tree/master/addons/metadata.demo.movies metadata.demo.movie addon].

Python movie scraper development

2022-06-07T08:33:04Z

Roman V M:

{{mininav|[[Scrapers]] {{l2|[[Development]]}} }}

{{Wiki_revamp}}

=Introduction=

Historically, Kodi has been supporting [https://kodi.wiki/view/HOW-TO:Write_media_scrapers XML scraping addons] that allow parsing online information sources
about movies, TV shows, music and so on. However, this approach has its limitations. First, XML parsing definitions with regular expressions are difficult to write and maintain.
Second, many information sources implemented REST APIs for getting information and regular expressions are not suitable for parsing JSON and data. That is why Python
scrapers have been introduced.

Python scrapers are written in Python language and have similar structure to [https://kodi.wiki/view/HOW-TO:Video_addon media addons] (media plugins).
They are also called by special <code>plugin://</code> URLs with query parameters. The main query parameter is <code>action</code> that defines a scraping stage.
An example of a URL query sting passed to a scraper plugin: <code>?action=getdetails&url=foo&pathSettings=%7B%22foo%22%3A+%22bar%22%7D</code>.
Like other plugins, scrapers are also interact with Kodi via the functions of <code>xbmcplugin</code> Python API module and pass information to Kodi through <code>xbmcgui.ListItem</code> instances.
Basically, a Python scraper is a media plugin that passes information to the Kodi database instead of presenting lists of playable media items.

A movie scraper must support "action" calls that are described below.

=Actions=

==Find==

The <code>find</code> action is used for searching for a specific movie by title and optionally a year that are passed as additional query parameters of the plugin call.
This action should use <code>xbmcplugin.addDirectoryItem</code> or <code>xbmcplugin.addDirectoryItems</code> to pass <code>xbmcgui.ListItem</code> instances to Kodi.
If only one instance is passed then it is considered as a perfect match. Otherwise the media file won't be matched and will need to be resolved manually by selecting
a necessary item from a list.
The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''label''': passed as <code>label</code> parameter to the class constructor. This is the label that is presented to a user during scraping.

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary movie info from data provider's
website or API. It can be, for example, a link to a movie page on a movie information website or a some unique ID to request movie information from a REST API.

* '''thumb''' (optional): passed via <code>setArtwork()</code> of <code>xbmcgui.ListItem</code> class instance method. This should be a URL of a movie poster, for example.

==NfoUrl==

The <code>NfoUrl</code> action is called as an alternative to <code>find</code> if an .NFO file is present along with a movie file. The entire .NFO file contents are passed as <code>nfo</code>
parameter. This action should use <code>xbmcplugin.addDirectoryItem</code> to pass a single <code>xbmcgui.ListItem</code> instance to Kodi.

The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary movie info from data provider's
website or API. It can be, for example, a link to a movie page on a movie information website or a some unique ID to request movie information from a REST API.

* '''Unique IDs''': set via <code>ListItem.setUniqueIDs()</code> instance method. A scraper should set at least the default unique ID from the movie information site it works with.
Optionally it can set unique IDs from other online movie databases.

==getdetails==

The <code>getdetails</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''url''' query parameter
from the previous stages and must set as much information to the <code>xbmcgui.ListItem</code> instance as possible.

==getartwork==

The <code>getdetails</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''id''' query parameter
that is the default unique ID set by '''NfoUrl''' or '''getdetails''' actions. This action should set available artwork using <code>ListItem.addAvailableArtwork()</code>
and <code>ListItem.setAvailableFanart()</code> instance methods.

'''Warning''': if your .NFO file contains the default unique ID that is different from the ID of the movie database your scraper works with,
you must set the correct default unique ID in <code>NfoUrl</code> action. Otherwise <code>getdetails</code> will receive the default unique ID from your .NFO file
and your scraper may not be able to find any artwork using this ID.

=Path Settings=

A scraper receives the current path settings via <code>pathSettings</code> query parameters as a JSON-encoded string with each action call.

=Example Movie Scraper=

A very high-level example of a Python movie scraper can be found in the Kodi source code as [https://github.com/xbmc/xbmc/tree/master/addons/metadata.demo.movies metadata.demo.movie addon].

Python movie scraper development

2022-06-07T08:32:26Z

Roman V M:

{{mininav|[[Scrapers]] {{l2|[[Development]]}} }}

{{Wiki_revamp}}

=Introduction=

Historically, Kodi has been supporting [https://kodi.wiki/view/HOW-TO:Write_media_scrapers XML scraping addons] that allow parsing online information sources
about movies, TV shows, music and so on. However, this approach has its limitations. First, XML parsing definitions with regular expressions are difficult to write and maintain.
Second, many information sources implemented REST APIs for getting information and regular expressions are not suitable for parsing JSON and data. That is why Python
scrapers have been introduced.

Python scrapers are written in Python language and have similar structure to [https://kodi.wiki/view/HOW-TO:Video_addon media addons] (media plugins).
They are also called by special <code>plugin://</code> URLs with query parameters. The main query parameter is <code>action</code> that defines a scraping stage.
An example of a URL query sting passed to a scraper plugin: <code>?action=getdetails&url=foo&pathSettings=%7B%22foo%22%3A+%22bar%22%7D</code>.
Like other plugins, scrapers are also interact with Kodi via the functions of <code>xbmcplugin</code> Python API module and pass information to Kodi through <code>xbmcgui.ListItem</code> instances.
Basically, a Python scraper is a media plugin that passes information to the Kodi database instead of presenting lists of playable media items.

A movie scraper must support "action" calls that are described below.

=Actions=

==Find==

The <code>find</code> action is used for searching for a specific movie by title and optionally a year that are passed as additional query parameters of the plugin call.
This action should use <code>xbmcplugin.addDirectoryItem</code> or <code>xbmcplugin.addDirectoryItems</code> to pass <code>xbmcgui.ListItem</code> instances to Kodi.
If only one instance is passed then it is considered as a perfect match. Otherwise the media file won't be matched and will need to be resolved manually by selecting
a necessary item from a list.
The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''label''': passed as <code>label</code> parameter to the class constructor. This is the label that is presented to a user during scraping.

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary movie info from data provider's
website or API. It can be, for example, a link to a movie page on a movie information website or a some unique ID to request movie information from a REST API.

* '''thumb''' (optional): passed via <code>setArtwork()</code> of <code>xbmcgui.ListItem</code> class instance method. This should be a URL of a movie poster, for example.

==NfoUrl==

The <code>NfoUrl</code> action is called as an alternative to <code>find</code> if an .NFO file is present along with a movie file. The entire .NFO file contents are passed as <code>nfo</code>
parameter. This action should use <code>xbmcplugin.addDirectoryItem</code> to pass a single <code>xbmcgui.ListItem</code> instance to Kodi.

The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary movie info from data provider's
website or API. It can be, for example, a link to a movie page on a movie information website or a some unique ID to request movie information from a REST API.

* '''Unique IDs''': set via <code>ListItem.setUniqueIDs()</code> instance method. A scraper should set at least the default unique from the movie information site it works with.
Optionally it can set unique IDs from other online movie databases.

==getdetails==

The <code>getdetails</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''url''' query parameter
from the previous stages and must set as much information to the <code>xbmcgui.ListItem</code> instance as possible.

==getartwork==

The <code>getdetails</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''id''' query parameter
that is the default unique ID set by '''NfoUrl''' or '''getdetails''' actions. This action should set available artwork using <code>ListItem.addAvailableArtwork()</code>
and <code>ListItem.setAvailableFanart()</code> instance methods.

'''Warning''': if your .NFO file contains the default unique ID that is different from the ID of the movie database your scraper works with,
you must set the correct default unique ID in <code>NfoUrl</code> action. Otherwise <code>getdetails</code> will receive the default unique ID from your .NFO file
and your scraper may not be able to find any artwork using this ID.

=Path Settings=

A scraper receives the current path settings via <code>pathSettings</code> query parameters as a JSON-encoded string with each action call.

=Example Movie Scraper=

A very high-level example of a Python movie scraper can be found in the Kodi source code as [https://github.com/xbmc/xbmc/tree/master/addons/metadata.demo.movies metadata.demo.movie addon].

Python movie scraper development

2022-06-07T08:26:06Z

Roman V M:

{{mininav|[[Scrapers]] {{l2|[[Development]]}} }}

{{Wiki_revamp}}

=Introduction=

Historically, Kodi has been supporting [https://kodi.wiki/view/HOW-TO:Write_media_scrapers XML scraping addons] that allow parsing online information sources
about movies, TV shows, music and so on. However, this approach has its limitations. First, XML parsing definitions with regular expressions are difficult to write and maintain.
Second, many information sources implemented REST APIs for getting information and regular expressions are not suitable for parsing JSON and data. That is why Python
scrapers have been introduced.

Python scrapers are written in Python language and have similar structure to [https://kodi.wiki/view/HOW-TO:Video_addon media addons] (media plugins).
They are also called by special <code>plugin://</code> URLs with query parameters. The main query parameter is <code>action</code> that defines a scraping stage.
An example of a URL query sting passed to a scraper plugin: <code>?action=getdetails&url=foo&pathSettings=%7B%22foo%22%3A+%22bar%22%7D</code>.
Like other plugins, scrapers are also interact with Kodi via the functions of <code>xbmcplugin</code> Python API module and pass information to Kodi through <code>xbmcgui.ListItem</code> instances.
Basically, a Python scraper is a media plugin that passes information to the Kodi database instead of presenting lists of playable media items.

A movie scraper must support "action" calls that are described below.

=Actions=

==Find==

The <code>find</code> action is used for searching for a specific movie by title and optionally a year that are passed as additional query parameters of the plugin call.
This action should use <code>xbmcplugin.addDirectoryItem</code> or <code>xbmcplugin.addDirectoryItems</code> to pass <code>xbmcgui.ListItem</code> instances to Kodi.
If only one instance is passed then it is considered as a perfect match. Otherwise the media file won't be matched and will need to be resolved manually by selecting
a necessary item from a list.
The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''label''': passed as <code>label</code> parameter to the class constructor. This is the label that is presented to a user during scraping.

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary movie info from data provider's
website or API. It can be, for example, a link to a movie page on a movie information website or a some unique ID to request movie information from a REST API.

* '''thumb''' (optional): passed via <code>setArtwork()</code> of <code>xbmcgui.ListItem</code> class instance method. This should be a URL of a movie poster, for example.

==NfoUrl==

The <code>NfoUrl</code> action is called as an alternative to <code>find</code> if an .NFO file is present along with a movie file. The entire .NFO file contents are passed as <code>nfo</code>
parameter. This action should use <code>xbmcplugin.addDirectoryItem</code> to pass a single <code>xbmcgui.ListItem</code> instance to Kodi.

The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary movie info from data provider's
website or API. It can be, for example, a link to a movie page on a movie information website or a some unique ID to request movie information from a REST API.

* '''Unique IDs''': set via <code>ListItem.setUniqueIDs()</code> instance method. A scraper should set at least the default unique from the movie information site it works with.
Optionally it can set unique IDs from other online movie databases.

==getdetails==

The <code>getdetails</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''url''' query parameter
from the previous stages and must set as much information to the <code>xbmcgui.ListItem</code> instance as possible.

==getartwork==

The <code>getdetails</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''id''' query parameter
that is the default unique ID set by '''NfoUrl''' or '''getdetails''' actions. This action should set available artwork using <code>ListItem.addAvailableArtwork()</code>
and <code>ListItem.setAvailableFanart()</code> instance methods.

=Path Settings=

A scraper receives the current path settings via <code>pathSettings</code> query parameters as a JSON-encoded string with each action call.

=Example Movie Scraper=

A very high-level example of a Python movie scraper can be found in the Kodi source code as [https://github.com/xbmc/xbmc/tree/master/addons/metadata.demo.movies metadata.demo.movie addon].

Python movie scraper development

2022-06-07T08:25:05Z

Roman V M:

{{mininav|[[Scrapers]] {{l2|[[Development]]}} }}

{{Wiki_revamp}}

=Introduction=

Historically, Kodi has been supporting [https://kodi.wiki/view/HOW-TO:Write_media_scrapers XML scraping addons] that allow parsing online information sources
about movies, TV shows, music and so on. However, this approach has its limitations. First, XML parsing definitions with regular expressions are difficult to write and maintain.
Second, many information sources implemented REST APIs for getting information and regular expressions are not suitable for parsing JSON and data. That is why Python
scrapers have been introduced.

Python scrapers are written in Python language and have similar structure to [https://kodi.wiki/view/HOW-TO:Video_addon media addons] (media plugins).
They are also called by special <code>plugin://</code> URLs with query parameters. The main query parameter is <code>action</code> that defines a scraping stage.
An example of a URL query sting passed to a scraper plugin: <code>?action=getdetails&url=foo&pathSettings=%7B%22foo%22%3A+%22bar%22%7D</code>.
Like other plugins, scrapers are also interact with Kodi via the functions of <code>xbmcplugin</code> Python API module and pass information to Kodi through <code>xbmcgui.ListItem</code> instances.

Basically, a Python scraper is a media plugin that passes information to the Kodi database instead of presenting lists of playable media items.
A movie scraper must support "action" calls that are described below.

=Actions=

==Find==

The <code>find</code> action is used for searching for a specific movie by title and optionally a year that are passed as additional query parameters of the plugin call.
This action should use <code>xbmcplugin.addDirectoryItem</code> or <code>xbmcplugin.addDirectoryItems</code> to pass <code>xbmcgui.ListItem</code> instances to Kodi.
If only one instance is passed then it is considered as a perfect match. Otherwise the media file won't be matched and will need to be resolved manually by selecting
a necessary item from a list.
The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''label''': passed as <code>label</code> parameter to the class constructor. This is the label that is presented to a user during scraping.

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary movie info from data provider's
website or API. It can be, for example, a link to a movie page on a movie information website or a some unique ID to request movie information from a REST API.

* '''thumb''' (optional): passed via <code>setArtwork()</code> of <code>xbmcgui.ListItem</code> class instance method. This should be a URL of a movie poster, for example.

==NfoUrl==

The <code>NfoUrl</code> action is called as an alternative to <code>find</code> if an .NFO file is present along with a movie file. The entire .NFO file contents are passed as <code>nfo</code>
parameter. This action should use <code>xbmcplugin.addDirectoryItem</code> to pass a single <code>xbmcgui.ListItem</code> instance to Kodi.

The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary movie info from data provider's
website or API. It can be, for example, a link to a movie page on a movie information website or a some unique ID to request movie information from a REST API.

* '''Unique IDs''': set via <code>ListItem.setUniqueIDs()</code> instance method. A scraper should set at least the default unique from the movie information site it works with.
Optionally it can set unique IDs from other online movie databases.

==getdetails==

The <code>getdetails</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''url''' query parameter
from the previous stages and must set as much information to the <code>xbmcgui.ListItem</code> instance as possible.

==getartwork==

The <code>getdetails</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''id''' query parameter
that is the default unique ID set by '''NfoUrl''' or '''getdetails''' actions. This action should set available artwork using <code>ListItem.addAvailableArtwork()</code>
and <code>ListItem.setAvailableFanart()</code> instance methods.

=Path Settings=

A scraper receives the current path settings via <code>pathSettings</code> query parameters as a JSON-encoded string with each action call.

=Example Movie Scraper=

A very high-level example of a Python movie scraper can be found in the Kodi source code as [https://github.com/xbmc/xbmc/tree/master/addons/metadata.demo.movies metadata.demo.movie addon].

Python movie scraper development

2022-06-07T08:16:38Z

Roman V M:

{{mininav|[[Scrapers]] {{l2|[[Development]]}} }}

{{Wiki_revamp}}

=Introduction=

Historically, Kodi has been supporting [https://kodi.wiki/view/HOW-TO:Write_media_scrapers XML scraping addons] that allow parsing online information sources
about movies, TV shows, music and so on. However, this approach has its limitations. First, XML parsing definitions with regular expressions are difficult to write and maintain.
Second, many information sources implemented REST APIs for getting information and regular expressions are not suitable for parsing JSON and data. That is why Python
scrapers have been introduced.

Python scrapers are written in Python language and have similar structure to [https://kodi.wiki/view/HOW-TO:Video_addon media addons] (media plugins).
They are also called by special <code>plugin://</code> URLs with query parameters. The main query parameter is <code>action</code> that defines a scraping stage.
Scrapers are also interact with Kodi via the functions of <code>xbmcplugin</code> Python API module and pass information to Kodi through <code>xbmcgui.ListItem</code> instances.

Basically, a Python scraper is a media plugin that passes information to Kodi instead of presenting lists of playable media items. A movie scraper must support "action" calls
that are described below.

=Actions=

==Find==

The <code>find</code> action is used for searching for a specific movie by title and optionally a year that are passed as additional query parameters of the plugin call.
This action should use <code>xbmcplugin.addDirectoryItem</code> or <code>xbmcplugin.addDirectoryItems</code> to pass <code>xbmcgui.ListItem</code> instances to Kodi.
If only one instance is passed then it is considered as a perfect match. Otherwise the media file won't be matched and will need to be resolved manually by selecting
a necessary item from a list.
The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''label''': passed as <code>label</code> parameter to the class constructor. This is the label that is presented to a user during scraping.

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary movie info from data provider's
website or API. It can be, for example, a link to a movie page on a movie information website or a some unique ID to request movie information from a REST API.

* '''thumb''' (optional): passed via <code>setArtwork()</code> of <code>xbmcgui.ListItem</code> class instance method. This should be a URL of a movie poster, for example.

==NfoUrl==

The <code>NfoUrl</code> action is called as an alternative to <code>find</code> if an .NFO file is present along with a movie file. The entire .NFO file contents are passed as <code>nfo</code>
parameter. This action should use <code>xbmcplugin.addDirectoryItem</code> to pass a single <code>xbmcgui.ListItem</code> instance to Kodi.

The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary movie info from data provider's
website or API. It can be, for example, a link to a movie page on a movie information website or a some unique ID to request movie information from a REST API.

* '''Unique IDs''': set via <code>ListItem.setUniqueIDs()</code> instance method. A scraper should set at least the default unique from the movie information site it works with.
Optionally it can set unique IDs from other online movie databases.

==getdetails==

The <code>getdetails</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''url''' query parameter
from the previous stages and must set as much information to the <code>xbmcgui.ListItem</code> instance as possible.

==getartwork==

The <code>getdetails</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''id''' query parameter
that is the default unique ID set by '''NfoUrl''' or '''getdetails''' actions. This action should set available artwork using <code>ListItem.addAvailableArtwork()</code>
and <code>ListItem.setAvailableFanart()</code> instance methods.

=Path Settings=

A scraper receives the current path settings via <code>pathSettings</code> query parameters as a JSON-encoded string with each action call.

=Example Movie Scraper=

A very high-level example of a Python movie scraper can be found in the Kodi source code as [https://github.com/xbmc/xbmc/tree/master/addons/metadata.demo.movies metadata.demo.movie addon].

Python movie scraper development

2022-06-07T08:16:02Z

Roman V M:

{{mininav|[[Scrapers]] {{l2|[[Development]]}} }}

{{Wiki_revamp}}

=Introduction=

Historically, Kodi has been supporting [https://kodi.wiki/view/HOW-TO:Write_media_scrapers XML scraping addons] that allow parsing online information sources
about movies, TV shows, music and so on. However, this approach has its limitations. First, XML parsing definitions with regular expressions are difficult to write and maintain.
Second, many information sources implemented REST APIs for getting information and regular expressions are not suitable for parsing JSON and data. That is why Python
scrapers have been introduced.

Python scrapers are written in Python language and have similar structure to [https://kodi.wiki/view/HOW-TO:Video_addon media addons] (media plugins).
They are also called by special <code>plugin://</code> URLs with query parameters. The main query parameter is <code>action</code> that defines a scraping stage.
Scrapers are also interact with Kodi via the functions of <code>xbmcplugin</code> Python API module and pass information to Kodi through <code>xbmcgui.ListItem</code> instances.

Basically, a Python scraper is a media plugin that passes information to Kodi instead of presenting lists of playable media items. A movie scraper must support "action" calls
that are described below.

=Actions=

==Find==

The <code>find</code> action is used for searching for a specific movie by title and optionally a year that are passed as additional query parameters of the plugin call.
This action should use <code>xbmcplugin.addDirectoryItem</code> or <code>xbmcplugin.addDirectoryItems</code> to pass <code>xbmcgui.ListItem</code> instances to Kodi.
If only one instance is passed then it is considered as a perfect match. Otherwise the media file won't be matched and will need to be resolved manually by selecting
a necessary item from a list.
The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''label''': passed as <code>label</code> parameter to the class constructor. This is the label that is presented to a user during scraping.

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary movie info from data provider's
website or API. It can be, for example, a link to a movie page on a movie information website or a some unique ID to request movie information from a REST API.

* '''thumb''' (optional): passed via <code>setArtwork()</code> of <code>xbmcgui.ListItem</code> class instance method. This should be a URL of a movie poster, for example.

==NfoUrl==

The <code>NfoUrl</code> action is called as an alternative to <code>find</code> if an .NFO file is present along with a movie file. The entire .NFO file contents are passed as <code>nfo</code>
parameter. This action should use <code>xbmcplugin.addDirectoryItem</code> to pass a single <code>xbmcgui.ListItem</code> instance to Kodi.

The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary movie info from data provider's
website or API. It can be, for example, a link to a movie page on a movie information website or a some unique ID to request movie information from a REST API.

* '''Unique IDs''': set via <code>ListItem.setUniqueIDs()</code> instance method. A scraper should set at least the default unique from the movie information site it works with.
Optionally it can set unique IDs from other online movie databases.

==getdetails==

The <code>getdetails</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''url''' query parameter
from the previous stages and must set as much information to the <code>xbmcgui.ListItem</code> instance as possible.

==getartwork==

The <code>getdetails</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''id''' query parameter
that is the default unique ID set by '''NfoUrl''' or '''getdetails''' actions. This action should set available artwork using <code>ListItem.addAvailableArtwork()</code>
and <code>ListItem.setAvailableFanart()</code> instance methods.

=Path Settings=

A scraper receives the current path settings via <code>pathSettings</code> query parameters as a JSON-encoded string with each action call.

==Example Movie Scraper==

A very high-level example of a Python movie scraper can be found in the Kodi source code as [https://github.com/xbmc/xbmc/tree/master/addons/metadata.demo.movies metadata.demo.movie addon].

Python movie scraper development

2022-06-07T08:15:24Z

Roman V M:

{{mininav|[[Scrapers]] {{l2|[[Development]]}} }}

{{Wiki_revamp}}

=Introduction=

Historically, Kodi has been supporting [https://kodi.wiki/view/HOW-TO:Write_media_scrapers XML scraping addons] that allow parsing online information sources
about movies, TV shows, music and so on. However, this approach has its limitations. First, XML parsing definitions with regular expressions are difficult to write and maintain.
Second, many information sources implemented REST APIs for getting information and regular expressions are not suitable for parsing JSON and data. That is why Python
scrapers have been introduced.

Python scrapers are written in Python language and have similar structure to [https://kodi.wiki/view/HOW-TO:Video_addon media addons] (media plugins).
They are also called by special <code>plugin://</code> URLs with query parameters. The main query parameter is <code>action</code> that defines a scraping stage.
Scrapers are also interact with Kodi via the functions of <code>xbmcplugin</code> Python API module and pass information to Kodi through <code>xbmcgui.ListItem</code> instances.

Basically, a Python scraper is a media plugin that passes information to Kodi instead of presenting lists of playable media items. A movie scraper must support "action" calls
that are described below.

=Actions=

==Find==

The <code>find</code> action is used for searching for a specific movie by title and optionally a year that are passed as additional query parameters of the plugin call.
This action should use <code>xbmcplugin.addDirectoryItem</code> or <code>xbmcplugin.addDirectoryItems</code> to pass <code>xbmcgui.ListItem</code> instances to Kodi.
If only one instance is passed then it is considered as a perfect match. Otherwise the media file won't be matched and will need to be resolved manually by selecting
a necessary item from a list.
The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''label''': passed as <code>label</code> parameter to the class constructor. This is the label that is presented to a user during scraping.

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary movie info from data provider's
website or API. It can be, for example, a link to a movie page on a movie information website or a some unique ID to request movie information from a REST API.

* '''thumb''' (optional): passed via <code>setArtwork()</code> of <code>xbmcgui.ListItem</code> class instance method. This should be a URL of a movie poster, for example.

==NfoUrl==

The <code>NfoUrl</code> action is called as an alternative to <code>find</code> if an .NFO file is present along with a movie file. The entire .NFO file contents are passed as <code>nfo</code>
parameter. This action should use <code>xbmcplugin.addDirectoryItem</code> to pass a single <code>xbmcgui.ListItem</code> instance to Kodi.

The <code>xbmcgui.ListItem</code> instances must be assigned the following properties:

* '''url''': passed as <code>url</code> parameter to the class constructor. This should be some unique string that can be used to request all the necessary movie info from data provider's
website or API. It can be, for example, a link to a movie page on a movie information website or a some unique ID to request movie information from a REST API.

* Unique IDs: set via <code>ListItem.setUniqueIDs()</code> instance method. A scraper should set at least the default unique from the movie information site it works with.
Optionally it can set unique IDs from other online movie databases.

==getdetails==

The <code>getdetails</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''url''' query parameter
from the previous stages and must set as much information to the <code>xbmcgui.ListItem</code> instance as possible.

==getartwork==

The <code>getdetails</code> action must pass a single <code>xbmcgui.ListItem</code> via <code>xbmcplugin.setResolvedUrl()</code> function. This action receives '''id''' query parameter
that is the default unique ID set by '''NfoUrl''' or '''getdetails''' actions. This action should set available artwork using <code>ListItem.addAvailableArtwork()</code>
and <code>ListItem.setAvailableFanart()</code> instance methods.

=Path Settings=

A scraper receives the current path settings via <code>pathSettings</code> query parameters as a JSON-encoded string with each action call.

==Example Movie Scraper==

A very high-level example of a Python movie scraper can be found in the Kodi source code as [https://github.com/xbmc/xbmc/tree/master/addons/metadata.demo.movies metadata.demo.movie addon].

Python movie scraper development

2022-06-06T13:23:40Z

Roman V M:

Python movie scraper development

2022-05-25T07:41:23Z

Roman V M:

Naming video files/Episodes

2022-02-18T21:53:29Z

Roman V M:

{{VideoLibraryCreate}}
{{mininav| [[Video library]] |[[HOW-TO:Create_Video_Library|Create Video Library]] }}

__TOC__

= Introduction =
Be aware of the following:

* The only item in the episode file name that the {{kodi}} scanner searches for is the Season and Episode numbering (''SxxEyy''). Everything else is irrelevant.
* It is this SxxEyy that is used to match and scrape the episode metadata and artwork.
* To reduce the risk of the scanner becoming confused by complicated naming, ensure your episode filenames are clean and simple as described below.
* Contrary to advice from 3rd party internet guides, you cannot set season and episode numbering through NFO Files.

'''Do not refer to IMDB for episode orders. There is no IMDB Scraper.'''<br>
'''If you are using the default scraper then the site to check is [https://www.themoviedb.org/ TMDB]. If you have changed to [https://www.thetvdb.com/ TVDB] or [https://www.tvmaze.com/ TV Maze], check the appropriate site.'''

= Single Episode Files =
Episode files are assumed to contain a single episode. If you have files that contain two or more episodes, then read the Multi-Episodes section below.

The following table details the ''Minimum'' and ''Recommended'' naming for episodes files.

{| class="prettytable"
|-
| scope="row" colspan="2" style="text-align:center; background-color:#a1f5e4;" | '''Minimum required episode name:''' || ! scope="row" colspan="1" style="width:10px; background-color:#d3d3d5; text-align:center;"| || scope="row" colspan="2" style="text-align:center; background-color:#a1f5e4;" | '''Recommended episode name:'''
|-
! scope="row" colspan="2" style="width:400px; background-color:#f5f5a1; text-align:center;"|''S01E01.mkv''
! scope="row" colspan="1" style="width:10px; background-color:#d3d3d5; text-align:center;"|
! scope="row" colspan="2" style="width:400px; background-color:#f5f5a1; text-align:center;"| ''Angel (1999) S01E01.mkv''
|-
|Where:
|'''''S01E01''''' = Season 1, Episode 1 - no spaces || ! scope="row" colspan="1" style="width:10px; background-color:#d3d3d5; text-align:center;"| ||Where: ||'''''Angel (1999)''''' = TV Show name
|-
|
| || ! scope="row" colspan="1" style="width:10px; background-color:#d3d3d5; text-align:center;"| || || '''''S01E01''''' = Season 1, Episode 1 - no spaces
|}

The Pattern '''S01E01''' is the most accurate available. Other Patterns are available in the table below with the most accurate at the top and accuracy decreasing down the list.

{|
|-
|
{|class="{{{1|prettytable}}}" style="margin: 0.5em 2.0em 0.5em 0em; font-size: 0.80em; {{border-radius|5px}}; float: top; clear: top;" width="500"
|-
! colspan="2" style="background-color:#;" | With Season
! colspan="2" style="background-color:#;" | No Season
! colspan="2" style="background-color:#;" | By Date
! colspan="2" style="background-color:#;" | By Title
|+
! style="background-color:#;" | Order
! style="background-color:#; width:200px" | Episode Naming
! style="background-color:#;" | Order
! style="background-color:#; width:200px" | Episode Naming
! style="background-color:#;" | Order
! style="background-color:#; width:200px" | Episode Naming
! style="background-color:#;" | Order
! style="background-color:#; width:200px" | Episode Naming
|-
|style="width:50px; text-align:center;"|1
|style="width:50px; text-align:left;"|''Name'' '''S01E02'''.ext
|style="width:50px; text-align:center;"|2
|style="width:50px; text-align:left;"|''Name'' '''ep02'''.ext
|style="width:50px; text-align:center;"|3
|style="width:50px; text-align:left;"|''Name'' '''yyyy.mm.dd'''.ext
|style="width:50px; text-align:center;"|7
|style="width:50px; text-align:left;"|'''Name'''.''special''.ext
|-
|style="width:50px; text-align:center;"|1
|style="width:50px; text-align:left;"|''Name'' '''S1E2'''.ext
|style="width:50px; text-align:center;"|2
|style="width:50px; text-align:left;"|''Name'' '''ep_02'''.ext
|style="width:50px; text-align:center;"|3
|style="width:50px; text-align:left;"|''Name'' '''yyyy-mm-dd'''.ext
|style="width:50px; text-align:left;"|
|style="width:50px; text-align:left;"|
|-
|style="width:50px; text-align:center;"|1
|style="width:50px; text-align:left;"|''Name'' '''S01.E02'''.ext
|style="width:50px; text-align:center;"|7
|style="width:50px; text-align:left;"|''Name'' '''part.II'''.ext
|style="width:50px; text-align:center;"|4
|style="width:50px; text-align:left;"|''Name'' '''dd.mm.yyyy'''.ext
|style="width:50px; text-align:left;"|
|style="width:50px; text-align:left;"|
|-
|style="width:50px; text-align:center;"|1
|style="width:50px; text-align:left;"|''Name'' '''S01_E02'''.ext
|style="width:50px; text-align:center;"|7
|style="width:50px; text-align:left;"|''Name'' '''pt.II'''.ext
|style="width:50px; text-align:center;"|
|style="width:50px; text-align:left;"|
|style="width:50px; text-align:left;"|
|style="width:50px; text-align:left;"|
|-
|style="width:50px; text-align:center;"|1
|style="width:50px; text-align:left;"|''Name'' '''S01xE02'''.ext
|style="width:50px; text-align:center;"|7
|style="width:50px; text-align:left;"|''Name'' '''pt_II'''.ext
|style="width:50px; text-align:center;"|
|style="width:50px; text-align:left;"|
|style="width:50px; text-align:left;"|
|style="width:50px; text-align:left;"|
|-
|style="width:50px; text-align:center;"|5
|style="width:50px; text-align:left;"|''Name'' '''1x02'''.ext
|style="width:50px; text-align:center;"|
|style="width:50px; text-align:left;"|
|style="width:50px; text-align:center;"|
|style="width:50px; text-align:left;"|
|style="width:50px; text-align:left;"|
|style="width:50px; text-align:left;"|
|-
|style="width:50px; text-align:center;"|6
|style="width:50px; text-align:left;"|''Name'' '''102'''.ext
|style="width:50px; text-align:left;"|
|style="width:50px; text-align:left;"|
|style="width:50px; text-align:left;"|
|style="width:50px; text-align:left;"|
|style="width:50px; text-align:left;"|
|style="width:50px; text-align:left;"|
|}
|
;With Season
:The most commonly used for nearly all TV Shows.
;No Season
:Normally used for Anime or single season TV Shows. Not commonly used.
;By date
:Used for long-running daily shows. Check the scraper site for numbering method used.
;By title (added in v.20 "Nexus")
:Used for special episodes in case if a data provider does not support "zero season" convention for specials, e.g. TVmaze. It allows to match special episodes by their titles.
|}

= Multi-Episode Files =
If you video files contain two or more episodes, then the following naming is required to scan multiple episodes from a single file.

{| class="prettytable"
|-
| scope="row" colspan="2" style="text-align:center; background-color:#a1f5e4;" | '''Minimum required episode name:''' || ! scope="row" colspan="1" style="width:10px; background-color:#d3d3d5; text-align:center;"| || scope="row" colspan="2" style="text-align:center; background-color:#a1f5e4;" | '''Recommended episode name:'''
|-
! scope="row" colspan="2" style="width:400px; background-color:#f5f5a1; text-align:center;"|''S01E01E02E04.mkv''
! scope="row" colspan="1" style="width:10px; background-color:#d3d3d5; text-align:center;"|
! scope="row" colspan="2" style="width:400px; background-color:#f5f5a1; text-align:center;"| ''Angel (1999) S01E01E02E04.mkv''
|-
|Where:
|'''''S01E01E02E04''''' = Season 1, Episode 1, 2 & 4 || ! scope="row" colspan="1" style="width:10px; background-color:#d3d3d5; text-align:center;"| ||Where: ||'''''Angel (1999)''''' = TV Show name
|-
|
| ''Note that Episode 3 is not included'' || ! scope="row" colspan="1" style="width:10px; background-color:#d3d3d5; text-align:center;"| || || '''''S01E01E02E04''''' = Season 1, Episode 1, 2 & 4
|}

{|
|-
|
{|class="{{{1|prettytable}}}" style="margin: 0.5em 2.0em 0.5em 0em; font-size: 0.80em; {{border-radius|5px}}; float: top; clear: top;" width="500"
|-
! colspan="2" style="background-color:#;" | Complex Pattern
! colspan="2" style="background-color:#;" | Short Pattern
|+
! style="background-color:#;" | Order
! style="background-color:#; width:350px" | Example Name
! style="background-color:#;" | Order
! style="background-color:#; width:200px" | Example Name
|-
|style="width:50px; text-align:center;"|1
|style="width:300px; text-align:left;"|''name'' '''s01e01'''-'''s01e02'''.ext
|style="width:50px; text-align:center;"|1
|style="width:150px; text-align:left;"|''name'' '''s01e01e02'''.ext
|-
|style="width:50px; text-align:center;"|1
|style="width:300px; text-align:left;"|''name'' '''s01e01'''-''episode1.title''-'''s01e02'''-''episode2.title''.ext
|style="width:50px; text-align:center;"|1
|style="width:150px; text-align:left;"|''name'' '''s01e01-02-03'''.ext
|-
|style="width:50px; text-align:center;"|1
|style="width:300px; text-align:left;"|''name'' '''s01e01'''-'''s01e02'''-'''s01e03'''.ext
|style="width:50px; text-align:center;"|5
|style="width:150px; text-align:left;"|''name'' '''1x01x02'''.ext
|-
|style="width:50px; text-align:center;"|5
|style="width:300px; text-align:left;"|''name'' '''1x01'''-'''1x02'''.ext
|style="width:50px; text-align:center;"|2
|style="width:150px; text-align:left;"|''name'' '''ep01-02'''.ext
|-
|style="width:50px; text-align:center;"|2
|style="width:300px; text-align:left;"|''name'' '''ep01'''-'''ep02'''.ext
|style="width:50px; text-align:center;"|
|style="width:150px; text-align:left;"|
|}
|
'''It is recommended that multi-episode files be split into Single Episode files.'''

In cases where the video file contains more than one episode, you must nominate each episode in the filename.

* The patterns are the same as Single Episode numbering, but extended to include multiple episodes.
* Only the episodes in the file name will be added, e.g. '''''Angel (1999) S01E01E04.mkv''''' will scrape episodes 1 and 4 but not include episodes 2 and 3.
* When using a single video file for multiple episodes it is possible to tell set episode bookmarks for each episode in the file. See: '''''[[bookmarks|Episode Bookmarks]]'''''

:'''''See also: [[Advancedsettings.xml#tvmultipartmatching|Multi-Episode Matching]]''''' to create additional pattern matches
|}

= Episode Groups =
Episode Groups or Orders are listings with alternate episode orders. Some TV Shows are broadcast in one order but the DVD/Bluray release may have a different screening order. A well known example is the TV Show ''Firefly'' where the Studio opted to broadcast high action episodes for ratings first, while the Producer wanted a different order to correctly unfold the storyline.

== TheMovieDB-TV Shows ==
TheMovieDB uses Episode Groups for additional episode orders.

To use one of the orders for scraping, follow the images below. Paste the address into a '''''[[NFO_files/Parsing|Parsing NFO file]]''''' then scrape. The Parsing NFO File will point the scraper to the listing. <ref>https://forum.kodi.tv/showthread.php?tid=338467&pid=2801494#pid2801494</ref>

<gallery mode="traditional" widths="500px" heights="283px">
File:EpisodeGroups01.jpg|Image 1- Accessing Episode Groups
File:EpisodeGroups02.jpg|Image 2- Available Episode Groups
File:EpisodeGroups03.jpg|Image 3- Select a Group, then copy the address for use in a '''''[[NFO_files/Parsing|Parsing NFO file]]'''''.
</gallery>

== The TVDB ==
{{editor note|TVDB will shortly be updating to v4 API with major changes to episode orders. This is expected to occur <strike>April 2021; Oct 2021</strike> 2<sup>nd</sup> Qtr. 2022.}}

= References =
<references />

<center>
{| style="border: 1px solid black;"
| scope="row" rowspan="2" style=" text-align:center;" | [[File:Mergefrom.gif|60px|link=Naming_video_files/Seasons]]
| style="width:200px; text-align:center;" | '''Previous step'''
| style="width:200px; text-align:center;" | '''Next step'''
| scope="row" rowspan="2" style=" text-align:center;" | [[File:Merge-arrow.gif|60px|link=Adding_video_sources]]
|-
| style="width:200px; text-align:center;" | '''[[Naming_video_files/Seasons|Seasons Setup]]'''
| style="width:200px; text-align:center;" | '''[[Adding_video_sources|Add Source & Scrape]]'''
|}</center>

{{top}}
{{updated|19}}

[[Category:Guides]]
[[Category:Karellen]]
[[Category:Video library]]
[[Category:First time user]]

HOW-TO:Video addon

2021-05-14T08:55:43Z

Roman V M:

{{mininav|[[Development]]|[[Add-on development]]|[[Python development]]}}
<br />

=Introduction=
This tutorial will explain how to write your first Kodi/XBMC video plugin Add-on<br />

=Tools=
Assuming you have already followed the Hello World Add-on you will have a text editor already setup. Since we are dealing with videos this time its probably a good idea to have a video player setup. We recommend another great open source project for this

- VLC http://www.videolan.org/vlc/

=Installing=
For this example we will use 2 nice basic video Add-on tutorials You can find the source-code here:

https://github.com/romanvm/plugin.video.example

You can either download as a zip and install both of these inside the kodi GUI from the install from zip feature. Or extract the zip into your userdata/add-ons folder.

=Testing=
You can first give the add-on a test run by going to:
System >> Add-Ons >> Enabled Add-Ons >> Video Add-Ons >> Example Kodi video Plugin. You should now be able to watch some test videos hosted from an internet web server.

[[File:Videotutorial1.jpg]]

=Explanation=
So whats happening in this add-on?

Basically the Add-on is checking a website for the hosted videos. In this example we are directly linking to the videos, but there are possibilities for dynamic content and scraping of just about any online source

Once the video link is sent to Kodi, our video player takes over and buffers, then plays the video just like any other media.

=Structure=
main.py <-- This is the actual python code for your Add-On

addon.xml <-- This is the Add-Ons metadata

icon.png <-- A PNG icon for the add-on. It can be 256x256 or 512x512 pixels big. Try to make it look nice!

Readme.md <-- The readme text file with a description of the Add-on and install instructions. This shows on the front of the GitHub page and helps users to understand your Add-on before downloading.

=The Code=
So now we come to the actual Add-On code, this is where most of your Add-on is written and is a simple text file containing python code.

First we initialize the Add-on and import and bits we need

<syntaxhighlight lang="python" enclose="div">
import sys
from urllib.parse import parse_qsl
import xbmcgui
import xbmcplugin
</syntaxhighlight>

Now we do something else...

<syntaxhighlight lang="python" enclose="div">
# Get the plugin url in plugin:// notation.
__url__ = sys.argv[0]
# Get the plugin handle as an integer number.
__handle__ = int(sys.argv[1])
</syntaxhighlight>

Here we use a fixed set of properties simply for demonstrating purposes. In a "real life" plugin you will need to get info and links to video files/streams from some web-site or online service.

<syntaxhighlight lang="python" enclose="div">
VIDEOS = {'Animals': [{'name': 'Crab',
'thumb': 'http://www.vidsplay.com/vids/crab.jpg',
'video': 'http://www.vidsplay.com/vids/crab.mp4',
'genre': 'Animals'},
{'name': 'Alligator',
'thumb': 'http://www.vidsplay.com/vids/alligator.jpg',
'video': 'http://www.vidsplay.com/vids/alligator.mp4',
'genre': 'Animals'},
{'name': 'Turtle',
'thumb': 'http://www.vidsplay.com/vids/turtle.jpg',
'video': 'http://www.vidsplay.com/vids/turtle.mp4',
'genre': 'Animals'}
],
'Cars': [{'name': 'Postal Truck',
'thumb': 'http://www.vidsplay.com/vids/us_postal.jpg',
'video': 'http://www.vidsplay.com/vids/us_postal.mp4',
'genre': 'Cars'},
{'name': 'Traffic',
'thumb': 'http://www.vidsplay.com/vids/traffic1.jpg',
'video': 'http://www.vidsplay.com/vids/traffic1.avi',
'genre': 'Cars'},
{'name': 'Traffic Arrows',
'thumb': 'http://www.vidsplay.com/vids/traffic_arrows.jpg',
'video': 'http://www.vidsplay.com/vids/traffic_arrows.mp4',
'genre': 'Cars'}
],
'Food': [{'name': 'Chicken',
'thumb': 'http://www.vidsplay.com/vids/chicken.jpg',
'video': 'http://www.vidsplay.com/vids/bbqchicken.mp4',
'genre': 'Food'},
{'name': 'Hamburger',
'thumb': 'http://www.vidsplay.com/vids/hamburger.jpg',
'video': 'http://www.vidsplay.com/vids/hamburger.mp4',
'genre': 'Food'},
{'name': 'Pizza',
'thumb': 'http://www.vidsplay.com/vids/pizza.jpg',
'video': 'http://www.vidsplay.com/vids/pizza.mp4',
'genre': 'Food'}
]}
</syntaxhighlight>
And now we define something else...

<syntaxhighlight lang="python" enclose="div">
def get_categories():
"""
Get the list of video categories.
Here you can insert some parsing code that retrieves
the list of video categories (e.g. 'Movies', 'TV-shows', 'Documentaries' etc.)
from some site or server.
:return: list
"""
return VIDEOS.keys()
</syntaxhighlight>
And now we define something else...

<syntaxhighlight lang="python" enclose="div">
def get_videos(category):
"""
Get the list of videofiles/streams.
Here you can insert some parsing code that retrieves
the list of videostreams in a given category from some site or server.
:param category: str
:return: list
"""
return VIDEOS[category]
</syntaxhighlight>
And now we define the list of categories

<syntaxhighlight lang="python" enclose="div">
def list_categories():
"""
Create the list of video categories in the Kodi interface.
:return: None
"""
# Get video categories
categories = get_categories()
# Create a list for our items.
listing = []
# Iterate through categories
for category in categories:
# Create a list item with a text label and a thumbnail image.
list_item = xbmcgui.ListItem(label=category, thumbnailImage=VIDEOS[category][0]['thumb'])
# Set a fanart image for the list item.
# Here we use the same image as the thumbnail for simplicity's sake.
list_item.setProperty('fanart_image', VIDEOS[category][0]['thumb'])
# Set additional info for the list item.
# Here we use a category name for both properties for for simplicity's sake.
# setInfo allows to set various information for an item.
# For available properties see the following link:
# http://mirrors.xbmc.org/docs/python-docs/15.x-isengard/xbmcgui.html#ListItem-setInfo
list_item.setInfo('video', {'title': category, 'genre': category})
# Create a URL for the plugin recursive callback.
# Example: plugin://plugin.video.example/?action=listing&category=Animals
url = '{0}?action=listing&category={1}'.format(__url__, category)
# is_folder = True means that this item opens a sub-list of lower level items.
is_folder = True
# Add our item to the listing as a 3-element tuple.
listing.append((url, list_item, is_folder))
# Add our listing to Kodi.
# Large lists and/or slower systems benefit from adding all items at once via addDirectoryItems
# instead of adding one by ove via addDirectoryItem.
xbmcplugin.addDirectoryItems(__handle__, listing, len(listing))
# Add a sort method for the virtual folder items (alphabetically, ignore articles)
xbmcplugin.addSortMethod(__handle__, xbmcplugin.SORT_METHOD_LABEL_IGNORE_THE)
# Finish creating a virtual folder.
xbmcplugin.endOfDirectory(__handle__)
</syntaxhighlight>

And here we list the videos

<syntaxhighlight lang="python" enclose="div">
def list_videos(category):
"""
Create the list of playable videos in the Kodi interface.
:param category: str
:return: None
"""
# Get the list of videos in the category.
videos = get_videos(category)
# Create a list for our items.
listing = []
# Iterate through videos.
for video in videos:
# Create a list item with a text label and a thumbnail image.
list_item = xbmcgui.ListItem(label=video['name'], thumbnailImage=video['thumb'])
# Set a fanart image for the list item.
# Here we use the same image as the thumbnail for simplicity's sake.
list_item.setProperty('fanart_image', video['thumb'])
# Set additional info for the list item.
list_item.setInfo('video', {'title': video['name'], 'genre': video['genre']})
# Set 'IsPlayable' property to 'true'.
# This is mandatory for playable items!
list_item.setProperty('IsPlayable', 'true')
# Create a URL for the plugin recursive callback.
# Example: plugin://plugin.video.example/?action=play&video=http://www.vidsplay.com/vids/crab.mp4
url = '{0}?action=play&video={1}'.format(__url__, video['video'])
# Add the list item to a virtual Kodi folder.
# is_folder = False means that this item won't open any sub-list.
is_folder = False
# Add our item to the listing as a 3-element tuple.
listing.append((url, list_item, is_folder))
# Add our listing to Kodi.
# Large lists and/or slower systems benefit from adding all items at once via addDirectoryItems
# instead of adding one by ove via addDirectoryItem.
xbmcplugin.addDirectoryItems(__handle__, listing, len(listing))
# Add a sort method for the virtual folder items (alphabetically, ignore articles)
xbmcplugin.addSortMethod(__handle__, xbmcplugin.SORT_METHOD_LABEL_IGNORE_THE)
# Finish creating a virtual folder.
xbmcplugin.endOfDirectory(__handle__)
</syntaxhighlight>

Now we tell Kodi how to find the video path to play the videos

<syntaxhighlight lang="python" enclose="div">
def play_video(path):
"""
Play a video by the provided path.
:param path: str
:return: None
"""
# Create a playable item with a path to play.
play_item = xbmcgui.ListItem(path=path)
# Pass the item to the Kodi player.
xbmcplugin.setResolvedUrl(__handle__, True, listitem=play_item)
</syntaxhighlight>

This is the router function

<syntaxhighlight lang="python" enclose="div">
def router(paramstring):
"""
Router function that calls other functions
depending on the provided paramstring
:param paramstring:
:return:
"""
# Parse a URL-encoded paramstring to the dictionary of
# {<parameter>: <value>} elements
params = dict(parse_qsl(paramstring[1:]))
# Check the parameters passed to the plugin
if params:
if params['action'] == 'listing':
# Display the list of videos in a provided category.
list_videos(params['category'])
elif params['action'] == 'play':
# Play a video from a provided URL.
play_video(params['video'])
else:
# If the plugin is called from Kodi UI without any parameters,
# display the list of video categories
list_categories()
</syntaxhighlight>

Call the router function

<syntaxhighlight lang="python" enclose="div">
if __name__ == '__main__':
# Call the router function and pass the plugin call parameters to it.
router(sys.argv[2])
</syntaxhighlight>

=Changing the code=

Hopefully by now you have managed to run the Add-On, understand the structure and can see what the code can do. Now lets trying changing it a little!

Since we already have the Add-on installed we can go directly to the Kodi userdata folder and edit the python directly. You can also have Kodi open at this time ready to run the Add-on when required. Just switch between your text editor and Kodi any time.

Now open up the main.py file from your userdata folder and lets change what the text says.

'Cars': [{'name': 'Postal Truck',
'thumb': 'http://www.vidsplay.com/vids/us_postal.jpg',
'video': 'http://www.vidsplay.com/vids/us_postal.mp4',
'genre': 'Cars'},

Try changing some of the links to another jpeg file or hosted online video. Also try to change the genre and see how that effects the view in Kodi.

Click save on your text editor and now try running the Add-On inside Kodi. You should see the new video links and navigation right away.

Congratulations! you've just played an online video from your very own Video Add-on

=Streaming Video=
In addition to this guide, here are some comments from an experienced developer who has worked with streaming video sites:

The docs on codedocs.xyz will help you a lot, way more than any wiki.

https://codedocs.xyz/xbmc/xbmc/

Speaking of docs, the ListItem reference will be most useful, time and time again.

https://codedocs.xyz/xbmc/xbmc/group__python__xbmcgui__listitem.html#gac31a08def90f50295146753353cb9541

Since you're relying on a streaming platform you won't need to do anything particular yourself, you can get away with fetching the data and posting an item with the URL of the video. InputStream.Adaptive configuration might be needed, especially if the videos need DRM.

You can get a lot done with the requests module, depending on the data you're processing. It's fully supported in Kodi, cookie and sessions included.
In short, it's really simple, you:

- get the information you need off the internet (title/URL could suffice for a start);

- you generate items that Kodi will treat as video sources (see xbmcplugin.addDirectoryItem() and xbmcgui.ListItem());

You could also read the sources of plugins such as AsciiDisco's Netflix or Sandmann79's Amazon/AmazonVOD addons.

You can read more in this thread
https://forum.kodi.tv/showthread.php?tid=348328

=Final Thoughts=

Writing a Video Add-on is actually pretty simple once you get the basic structure correct. You can scrape most online sources with Regex, or host your own videos on the web. You can also play youtube clips or vevo videos using some additional easy to use modules.

If you have any questions about this tutorial, feel free to discuss it on our forums here: http://forum.kodi.tv/showthread.php?tid=248774

=Extra info=

Just a reminder...
IMPORTANT!!!!!!! Do not mix tabs and spaces in python

[[Category:Add-on development]]

Add-on development

2021-05-14T08:53:12Z

Roman V M:

{{mininav|[[Development]]}}
[[File:Wiki logo.png|150px|link=|left]]
<section begin="intro" />This area contains information, tutorials, and links for creating add-ons (and plug-ins) for Kodi.<section end="intro" />

{{-}}
----
{{huge|'''{{color|black|General}}'''}}
{| width="100%"
|- valign="top"
| width="16%" align="center" | {{Main page icon|image=Nuvola apps kthememgr.png|link=About Add-ons}}
| width="17%" align="center" | {{Main page icon|image=Administration.png|link=Add-on settings}}
| width="16%" align="center" | {{Main page icon|image=Applications-development.png|link=Add-on structure|title=Add-on Structure}}
| width="17%" align="center" | {{Main page icon|image=Tool-box-icon.png|link=Development Tools}}
| width="16%" align="center" | {{Main page icon|image=60.jpg|link=Special protocol}}
| width="18%" align="center" | {{Main page icon|image=Translate icon.jpg|link=Translation System}}
|}

----
{{huge|'''{{color|black|Add-on Types}}'''}}
{| width="100%"
|- valign="top"
| width="14%" align="center" | {{Main page icon|image=Nuvola apps kthememgr.png|link=Plugin sources}}
| width="14%" align="center" | {{Main page icon|image=Misc-Misc-Box-icon.png|link=Add-on_repositories|title=Repository add-ons}}
| width="14%" align="center" | {{Main page icon|image=Web-icon.png|link=HOW-TO:Write media scrapers|title=Scraper add-ons}}
| width="14%" align="center" | {{Main page icon|image=Applications-development.png|link=Script sources|title=Scripts}}
| width="14%" align="center" | {{Main page icon|image=Administration.png|link=Script subtitles}}
| width="14%" align="center" | {{Main page icon|image=Gnome-system-run.png|link=Service add-ons}}
| width="14%" align="center" | {{Main page icon|image=Weather_addon.png|link=Weather addons}}
|}

----
{{huge|'''{{color|black|Tutorials}}'''}}

{| width="100%"
|- valign="top"
| width="12.5%" align="center" | {{Main page icon|image=music-icon.png|link=HOW-TO:Audio_addon|title=Audio Tutorial}}
| width="12.5%" align="center" | {{Main page icon|image=context-menu-icon.png|link=Context_Item_Add-ons|title=Context Item Tutorial}}
| width="12.5%" align="center" | {{Main page icon|image=Addons-icon.png|link=Audio-video_add-on_tutorial|title=General Add-on Tutorial}}
| width="12.5%" align="center" | {{Main page icon|image=Translation - Noun project 987.png|link=HOW-TO:HelloWorld_addon|title=Hello World Tutorial}}
| width="12.5%" align="center" | {{Main page icon|image=Screensaver-icon.png|link=HOW-TO:Screensaver_addon|title=Screensaver Tutorial}}
| width="12.5%" align="center" | {{Main page icon|image=script-icon.png|link=HOW-TO:Script_addon|title=Script Tutorial}}
| width="12.5%" align="center" | {{Main page icon|image=Visualisation-icon.jpg|link=HOW-TO:Visualisation_addon|title=Visualisation Tutorial}}
| width="12.5%" align="center" | {{Main page icon|image=video-icon.png|link=HOW-TO:Video_addon|title=Video Addon Tutorial}}
|}
----
{{huge|'''{{color|black|Advanced}}'''}}
{| width="100%"
|- valign="top"
| width="12.5%" align="center" | {{Main page icon|image=Unicode icon.jpg|link=Add-on unicode paths}}
| width="12.5%" align="center" | {{Main page icon|image=501159.png‎‎|link=JSON-RPC API}}
| width="12.5%" align="center" | {{Main page icon|image=Book icon 1.png|link=Python_libraries|title=Kodi Python Libraries}}
| width="12.5%" align="center" | {{Main page icon|image=Boolean_operation_icon.jpg‎|link=List of boolean conditions}}
| width="12.5%" align="center" | {{Main page icon|image=Apps-Brackets-B-icon.png‎|link=List of built-in functions}}
| width="12.5%" align="center" | {{Main page icon|image=shopping-list-generator-icon.png‎|link=InfoLabels|title=List of info labels}}
| width="12.5%" align="center" | {{Main page icon|image=Web-pdb-icon.png‎|link=Python_debugging|title=Python Debugging}}
| width="12.5%" align="center" | {{Main page icon|image=Windowsicon.png‎‎|link=Window IDs|title=Window IDs}}
|}

----
{{huge|'''{{color|black|Publishing}}'''}}
{| width="100%"
|- valign="top"
| width="25%" align="center" | {{Main page icon|image=Thumbs up font awesome.png‎|link=Add-on rules}}
| width="25%" align="center" | {{Main page icon|image=Upload-128.png|link=Submitting Add-ons}}
| width="25%" align="center" | {{Main page icon|image=Thumbnail-symbol-transparent.png|link=Official_add-on_repository|title=Official Add-on Repository}}
| width="25%" align="center" | {{Main page icon|image=Box_icon.png‎|link=Unofficial add-on repositories|title=Third-party Add-on Repositories}}
|}
----

[[Category:Add-on development|*]]

HOW-TO:Debug Python Scripts with Web-PDB

2018-10-13T20:38:05Z

Roman V M:

{{mininav|[[Development]]|[[Add-on development]]|[[Python development]]}}

[https://github.com/romanvm/kodi.web-pdb Web-PDB] is a remote web-interface to Python's built-in [https://docs.python.org/2/library/pdb.html PDB] debugger with additional convenience features. It is not tied to any IDE or other software, all you need is a common web-browser, e.g. Chrome or Firefox. Web-PDB is compatible with both Python 2 and 3, so you can use it to debug your Python 3 compatible addons.

[[File:web-pdb.png]]

Web-PDB for Kodi is available as an addon in the official Kodi addons repo.

== How To Use Web-PDB for Kodi ==

1. Install Web-PDB addon: '''Kodi Add-on repository > Program add-ons > Web-PDB'''.

2. Add <code>script.module.web-pdb</code> to [[addon.xml]] as a dependency:
<syntaxhighlight lang="xml" enclose="div">
<requires>
...
<import addon="script.module.web-pdb" />
<requires>
</syntaxhighlight>

3. Restart Kodi so that it re-reads addon dependencies.

4. Insert the following line into your addon code at the point where you want to start debugging:
<syntaxhighlight lang="python" enclose="div">
import web_pdb; web_pdb.set_trace()
</syntaxhighlight>

The <code>set_trace()</code> call will suspend your addon and open a web-UI at the default port 5555 (port value can be changed). At the same time a notification will be displayed in Kodi, indicating that a debug session is active. The notification also shows web-UI host/port.

5. Enter in your the address bar of your browser: <code>http://<your Kodi machine hostname or IP>:5555</code>, for example <code>http://monty-python:5555</code>. Use <code>localhost</code> as a hostname if you are connecting from the same machine that runs Kodi. If everything is OK, you should see the Web-PDB UI. Now you can use all PDB commands and features. Additional '''Current file''', '''Globals''' and '''Locals''' information boxes help you better track your program runtime state.

== More Information ==

* [https://github.com/romanvm/kodi.web-pdb Web-PDB for Kodi on GitHub].
* [https://docs.python.org/2/library/pdb.html PDB debugger documentation].

HOW-TO:Debug Python Scripts with Web-PDB

2018-10-13T20:32:17Z

Roman V M: Add mininav

{{mininav|[[Development]]|[[Add-on development]]|[[Python development]]}}

[https://github.com/romanvm/kodi.web-pdb Web-PDB] is a remote web-interface to Python's built-in [https://docs.python.org/2/library/pdb.html PDB] debugger with additional convenience features. It is not tied to any IDE or other software, all you need is a common web-browser, e.g. Chrome or Firefox.
[[File:web-pdb.png]]

Web-PDB for Kodi is available as an addon in the official Kodi addons repo.

== How To Use Web-PDB for Kodi ==

1. Install Web-PDB addon: '''Kodi Add-on repository > Program add-ons > Web-PDB'''.

2. Add <code>script.module.web-pdb</code> to [[addon.xml]] as a dependency:
<syntaxhighlight lang="xml" enclose="div">
<requires>
...
<import addon="script.module.web-pdb" />
<requires>
</syntaxhighlight>

3. Restart Kodi so that it re-reads addon dependencies.

4. Insert the following line into your addon code at the point where you want to start debugging:
<syntaxhighlight lang="python" enclose="div">
import web_pdb; web_pdb.set_trace()
</syntaxhighlight>

The <code>set_trace()</code> call will suspend your addon and open a web-UI at the default port 5555 (port value can be changed). At the same time a notification will be displayed in Kodi, indicating that a debug session is active. The notification also shows web-UI host/port.

5. Enter in your the address bar of your browser: <code>http://<your Kodi machine hostname or IP>:5555</code>, for example <code>http://monty-python:5555</code>. Use <code>localhost</code> as a hostname if you are connecting from the same machine that runs Kodi. If everything is OK, you should see the Web-PDB UI. Now you can use all PDB commands and features. Additional '''Current file''', '''Globals''' and '''Locals''' information boxes help you better track your program runtime state.

== More Information ==

* [https://github.com/romanvm/kodi.web-pdb Web-PDB for Kodi on GitHub].
* [https://docs.python.org/2/library/pdb.html PDB debugger documentation].

Add-on development

2018-10-13T20:29:31Z

Roman V M: Replaced the link to obsolete WinPDB article (WinPDB is unsupported and seriously outdated)

{{mininav|[[Development]]}}
[[File:Wiki logo.png|150px|link=|left]]
<section begin="intro" />This area contains information, tutorials, and links for creating add-ons (and plug-ins) for Kodi.<section end="intro" />

{{-}}
----
{{huge|'''{{color|black|General}}'''}}
{| width="100%"
|- valign="top"
| width="33%" align="center" | {{Main page icon|image=Nuvola apps kthememgr.png|link=About Add-ons}}
| width="33%" align="center" | {{Main page icon|image=Applications-development.png|link=Add-on structure|title=Add-on Structure}}
| width="33%" align="center" | {{Main page icon|image=Tool-box-icon.png|link=Development Tools}}
|}
{| width="100%"
|- valign="top"
| width="33%" align="center" | {{Main page icon|image=Administration.png|link=Add-on settings}}
| width="33%" align="center" | {{Main page icon|image=60.jpg|link=Special protocol}}
| width="33%" align="center" | {{Main page icon|image=Translate icon.jpg|link=Translation System}}
|}
----
{{huge|'''{{color|black|Add-on Types}}'''}}
{| width="100%"
|- valign="top"
| width="33%" align="center" | {{Main page icon|image=Nuvola apps kthememgr.png|link=Plugin sources}}
| width="33%" align="center" | {{Main page icon|image=Applications-development.png|link=Script sources|title=Scripts}}
| width="33%" align="center" | {{Main page icon|image=Administration.png|link=Script Subtitles}}
|}
{| width="100%"
|- valign="top"
| width="25%" align="center" | {{Main page icon|image=Gnome-system-run.png|link=Service addons}}
| width="25%" align="center" | {{Main page icon|image=Misc-Misc-Box-icon.png|link=HOW-TO:Create a repository for add-ons|title=Repository add-ons}}
| width="25%" align="center" | {{Main page icon|image=Web-icon.png|link=HOW-TO:Write media scrapers|title=Scraper add-ons}}
| width="25%" align="center" | {{Main page icon|image=Weather_addon.png|link=Weather_addons}}
|}
----
{{huge|'''{{color|black|Tutorials}}'''}}
{| width="100%"
|- valign="top"
| width="33%" align="center" | {{Main page icon|image=Translation - Noun project 987.png|link=HOW-TO:HelloWorld_addon|title=Hello World Tutorial}}
| width="33%" align="center" | {{Main page icon|image=Addons-icon.png|link=Audio/video add-on tutorial|title=General Add-on Tutorial}}
| width="33%" align="center" | {{Main page icon|image=script-icon.png|link=HOW-TO:Script_addon|title=Script Tutorial}}
|}
{| width="100%"
|- valign="top"
| width="33%" align="center" | {{Main page icon|image=Video.png|link=3rd Party Tutorials}}
| width="33%" align="center" | {{Main page icon|image=music-icon.png|link=HOW-TO:Audio_addon|title=Audio Tutorial}}
| width="33%" align="center" | {{Main page icon|image=video-icon.png|link=HOW-TO:Video_addon|title=Video Tutorial}}
|}
{| width="100%"
|- valign="top"
| width="33%" align="center" | {{Main page icon|image=Visualisation-icon.jpg|link=HOW-TO:Visualisation_addon|title=Visualisation Tutorial}}
| width="33%" align="center" | {{Main page icon|image=Screensaver-icon.png|link=HOW-TO:Screensaver_addon|title=Screensaver Tutorial}}
| width="33%" align="center" | {{Main page icon|image=context-menu-icon.png|link=Context_Item_Add-ons|title=Context Item Tutorial}}
|}
----
{{huge|'''{{color|black|Advanced}}'''}}
{| width="100%"
|- valign="top"
| width="50%" align="center" | {{Main page icon|image=Book icon 1.png|link=Python Libraries|title=Kodi Python Libraries}}
| width="50%" align="center" | {{Main page icon|image=501159.png‎‎|link=JSON-RPC API}}
|}
{| width="100%"
|- valign="top"
| width="33%" align="center" | {{Main page icon|image=Eclipse_ide_icon_by_necromod-d5lt9zc.png‎|link=HOW-TO:Debug_Python_Scripts_with_Eclipse|title=Debugging with Eclipse}}
| width="33%" align="center" | {{Main page icon|image=Web-pdb-icon.png‎|link=HOW-TO:Debug_Python_Scripts_with_Web-PDB|title=Debugging with Web-PDB}}
| width="33%" align="center" | {{Main page icon|image=Unicode icon.jpg|link=Add-on unicode paths}}
|}
{| width="100%"
|- valign="top"
| width="25%" align="center" | {{Main page icon|image=Apps-Brackets-B-icon.png‎|link=List of built-in functions}}
| width="25%" align="center" | {{Main page icon|image=shopping-list-generator-icon.png‎|link=InfoLabels|title=List of info labels}}
| width="25%" align="center" | {{Main page icon|image=Windowsicon.png‎‎|link=Window IDs|title=List of window IDs}}
| width="25%" align="center" | {{Main page icon|image=Boolean_operation_icon.jpg‎|link=List of boolean conditions}}
|}
----
{{huge|'''{{color|black|Publishing}}'''}}
{| width="100%"
|- valign="top"
| width="33%" align="center" | {{Main page icon|image=Upload-128.png|link=Submitting Add-ons}}
| width="33%" align="center" | {{Main page icon|image=Thumbs up font awesome.png‎|link=Add-on Rules}}
|}
----
{{huge|'''{{color|black|Repositories}}'''}}
{| width="100%"
|- valign="top"
| width="33%" align="center" | {{Main page icon|image=Thumbnail-symbol-transparent.png|link=Official_add-on_repository|title=Official Add-on Repository}}
| width="33%" align="center" | {{Main page icon|image=Zappy.png|link=Add-on Website}}
| width="33%" align="center" | {{Main page icon|image=Box_icon.png‎|link=Unofficial add-on repositories|title=Third-party Add-on Repositories}}
|}

[[Category:Add-on development|*]]

File:Web-pdb-icon.png

2018-10-13T20:26:01Z

Roman V M:

Web-PDB debugger icon

General information about migration to Python 3

2018-02-12T12:56:23Z

Roman V M: /* Absolute Import */

== Overview ==

Python 3 was released 9 years ago and EOL for Python 2.7 is scheduled for 2020. Currently more and more Python project are switching to Python 3 or 2/3 compatible code. The issue of migrating Kodi’s built-in Python interpreter to Python 3.x was brought up on the Kodi forum several times. Now, thanks to a successful GSOC 2017 project, we have a working Python 3 interpreter for Kodi. That is why on Kodi DevCon 2017 Team Kodi decided that we will switch to Python 3 in the future Kodi release (v.19 preliminarily).

Unfortunately, Python 3.x versions are not backward-compatible with Python 2.x, so we decided to urge addon developers to start porting their addons to Python 3 by making the addons compatible with both Python 2 and 3 and submitting them to the official repo. This way, when Kodi with Python 3 is released, we won’t have a situation when we have no working addons.

== The Process ==

* Starting from Kodi 18 (Leia), only addons that are compatible with both Python 2 and 3 will be accepted to the official addon repository.
* We will provide Kodi builds with Python 3 so that addon developers can tests if their addons are really compatible with Python 3.
* A thread will be created on the Kodi forum so that addon devs can get help with their migration process.

== Python 3 And 2 Differences ==

Below is a brief overview of the main differences between Python 3 and 2.

=== Unicode strings by default ===

In Python 2 built-in <code>str</code> type holds the sequence of bytes so it can be used to store both binary data and textual data in ASCII or any of 8-bit fixed-length encodings (there were plenty). Python 2 also has <code>unicode</code> type that can store text in various writing systems. A minimal unit of a <code>unicode</code> object is a Unicode codepoint (a Unicode character). Both <code>str</code> and <code>unicode</code> objects can be mixed and matched together, for example, concatenated using <code>+</code> operator.

In Python 3 <code>str</code> type holds Unicode characters, and for binary data a new <code>bytes</code> type was added. This type was also added to Python 2.7 to simplify porting to Python 3 but in Python 2.7 bytes is simply an alias for <code>str</code>. The <code>str</code> and <code>bytes</code> types are incompatible, and an attempt to use both together will result in <code>TypeError</code> exception. A <code>bytes</code> object can be converted to <code>str</code> using .decode() method, and a <code>str</code> is converted to <code>bytes</code> using .encode() method.

=== Absolute Import ===
Python 3 uses so called “absolute import” and you can no longer import neighbor modules from inside a package using only a module name. You need to either to use either a fully qualified name or a “dotted” notation.

Let’s assume that you have a <code>foo</code> package that contains <code>bar</code> and <code>spam</code> modules:
<pre>
foo/
__init__.py
bar.py
spam.py
</pre>
And you need to import <code>bar</code> module from <code>spam</code>. In Python 2 you can do:
<source lang="python">
import bar
</source>

But in Python 3 you need to use either

<source lang="python">
import foo.bar
</source>

or

<source lang="python">
from . import bar
</source>

=== Floating Point Division ===

In Python 2 applying division operator to int numbers produces int result. For example:

<source lang="python">
>>> 4 / 3
1
</source>

In Python 3 division operator always gives you a float result.

<source lang="python">
>>> 4/3
1.3333333333333333
</source>

To get an int result you need to use integer division operator:

<source lang="python">
>>> 4 // 3
1
</source>

=== Print Function ===

In Python 3 print statement has been replaced with print() function. This is not very relevant to Kodi because using print in Python addons is discouraged and xbmc.log() function should be used to write messages to the Kodi log.

=== Default Iterators ===

In Python 3 most function and methods that produce sequences return iterators instead of lists. For example, range() function and .items(), .keys(), .values() of dict type produce iterators instead of lists, and their Python 2 analogues — xrange() function and .iter* methods of dict type — have been removed.

=== No long Integers ===

In Python 3 long integer type has been removed and int type can now hold a number of arbitrary length.

Those are only some of the differences between Python 2 and 3. For a more complete list see [https://docs.python.org/3.6/whatsnew/3.0.html official Python documentation].

== Some Useful Information ==

As it was said above, Python 3 includes a number of incompatible changes, so to avoid situation when after upgrading Kodi for Python 3 support most Python addons will be broken developers should start to port their addons to be compatible with both Python 2 and 3. This is not a trivial task but fortunately there are a number of tools and recommendations to simplify this process. Here I’ll give you some advice and information about tools that will simplify creating portable code that runs on both Python 2 and 3.

=== Know Your Strings! ===

This is the most problematic part of porting Python 2 code to Python 3 so I put it first. One of the most notable differences between Python 2 and 3 is clear separation between “textual” and “binary” strings, that is, between textual data and their binary representation. This difference is often difficult to understand, especially for novice developers (not only in Python), and the fact that in Python 2 you can mix together str (binary data) and unicode (textual data) adds to that difficulty.

There are many explanations in the Internet, but here are the most basic things about strings that you need to know:

“Textual” or Unicode strings (unicode in Python 2 and str in Python 3) hold text as a sequence of characters. A minimal unit in such strings is a Unicode character — an abstract entity that represent an alphabet symbol, a punctuation sign or other symbol included in the Unicode character table. Of course, such symbols have concrete internal binary representation (a computer is a binary machine after all), but this representation is not relevant to us. All you need to know is that “textual” strings contain text units without being tied to concrete binary representation.

Unlike Unicode strings, binary strings (str in Python 2 and bytes in both Python 2 and 3) hold binary data, and a minimal unit of such strings is a byte or 8 bits. Historically, in Python 2 such strings are used for textual data encoded in ASCII or other 1-byte fixed-length encoding (e.g. Windows 1251 for Cyrillic alphabets). However, this approach has its limitation, that is why unicode type was introduced in Python 2, and in Python 3 it was renamed to str and made a default container for textual data.

Unfortunately, Python 2 allows to mix and match Unicode and binary strings in the same context and this creates big problems when porting Python 2 code to Python 3. So in order to successfully port your code to Python 3 you need to carefully consider how textual data are processed in your addon. The following recommendations will help you avoid problems when writing Python 2/3 compatible code.

=== Avoid using binary strings for text ===

Store all your text as Unicode strings. This means that all binary strings received from external sources (textual files, websites, various APIs) need to be decoded using appropriate encoding (UTF-8 in most cases). If you need to use strings literals (although using hardcoded strings for user-facing text in Kodi is strongly discouraged), they need to be defined as unicode strings as well. You can use either
<source lang="python">
from __future__ import unicode_literals
</source>
at the beginning of your modules to enable unicode strings by default as in Python 3 or u-strings, e.g. <code>u'foo'</code> (Python 3 understands them too) — it doesn't really matter as long as you are doing it consistently.

=== Know the libraries you are using ===
Working with Python Standard Library and third-party libraries (including Kodi Python API) you should know that types their functions and methods expect and what types they return. For example, in popular requests library Response.text property returns a Unicode string and Response.content returns a binary string.

If a function/method/property returns a binary string, you need to decode it to a Unicode string using .decode() method. Don’t use unicode class constructor because unicode type does not exist in Python 3.

If you are reading textual files from disk, it is better to use io.open() instead of built-in open(). io.open() can decode file contents to Unicode using specified encoding and works identically both in Python 2 and 3. Example:
<source lang="python">
import io

with io.open('foo.txt', 'r', encoding='utf-8') as fo:
text = fo.read()
</source>

=== Potential pitfalls ===

When writing Python 2/3-compatible code or converting existing Python 2 codebase to compatible code you should pay attention to the following Python language constructions where TypeError exceptions may happen because of incompatible string types if you don’t get your strings in order:

* String concatenations with + operator (<code>'foo' + 'bar'</code>).
* String formatting — both “old style” (<code>'foo %s' % 'bar'</code>) and “new style” (<code>'foo {}'.format('bar')</code>).
* String joining (<code>', '.join(['foo', 'bar'])</code>).

In all those cases always make sure that you work with Unicode strings only.
Another source of subtle bugs is functions and methods that accept and return binary strings (str type) in Python 2 but Unicode strings in Python 3 (again, str type but with different properties). You can use the following utility functions to “normalize” strings that are provided to such functions/methods and received from them:

<source lang="python">
import sys

__all__ = ['PY2', 'py2_encode', 'py2_decode']

PY2 = sys.version_info[0] == 2

def py2_encode(s, encoding='utf-8'):
"""
Encode Python 2 ``unicode`` to ``str``

In Python 3 the string is not changed.
"""
if PY2 and isinstance(s, unicode):
s = s.encode(encoding)
return s

def py2_decode(s, encoding='utf-8'):
"""
Decode Python 2 ``str`` to ``unicode``

In Python 3 the string is not changed.
"""
if PY2 and isinstance(s, str):
s = s.decode(encoding)
return s
</source>

Those utility functions are included in Kodi Six library that is described in the following section.

== Helper Libraries and Tools ==

=== 2to3 ===

[https://docs.python.org/2/library/2to3.html 2to3] script is created by Python developers to help converting existing Python 2 code to Python 3. On Windows it is included in Python distribution (Tools subfolder) but on other OSes you man need to install it separately. For example, on Ubuntu it is included in python-examples package. Note that this script is created for conversion from Python 2 to 3, not for writing portable code, so you need to treat its output with care.

=== Modernize ===

https://python-modernize.readthedocs.io/en/latest/ Modernize] script works on top of 2to3 and is supposed to help you to convert your existing Python 2 code to Python 2/3-compatible. However, in my experiments it did not work reliably even in simple cases so I cannot recommend it for usage.

=== Six ===

[https://pythonhosted.org/six/ Six] was the first library developed to simplify Python 2 to 3 migration. It provides a set of wrappers that hide differences between Python 2 and 3 behind its API. It is less intrusive than the following alternative because it does not monkey-patch built-in names, but in order to use Six library effectively you need to learn its API. Six can be used both for writing new Python 2/3-compatible addons and converting existing ones, but it requires good knowledge of Python 2/3 differences to pick necessary Six features that address specific differences.

Six library is included in the Kodi addon repository as '''script.module.six''' addon.

=== Future ===

[http://python-future.org/ Future] library, like Six, was created to simplify porting existing Python 2 code to 3. But it uses a different approach than Six. Future monkey-patches built-in Python objects to make Python 2 behave like Python 3. The advantage of such approach is that code written using Future is close to idiomatic Python 3 and requires little re-work if you decide to drop Python 2 support in the future. However, such approach may cause problems in some rare edge-cases.

Future library also includes <code>futurize</code> command-line utility for converting existing Python 2 code to 2/3-compatible and in my experiments this utility showed good results.
Future library (without additional utilities) is included in the Kodi addon repository as script.module.future addon.

=== Kodi Six ===

[https://github.com/romanvm/kodi.six Kodi Six] library (available as '''script.module.kodi-six''' addon) is created to normalize string handling in Kodi Python API based on different Python versions by applying wrappers around Kodi API functions and classes. In Kodi API based on Python 2 Kodi Six wrappers have the following effects:

* All functions and methods that expect string arguments can accept both UTF-8 encoded <code>str</code> objects and <code>unicode</code> objects.
* All functions and methods that return string data return <code>unicode</code> objects.

In Kodi API based on Python 3 the wrappers have no effect because Python 3-based API works only with <code>str</code> (Unicode) objects. This eliminates the need for ad hoc encoding and decoding string when working with Kodi API. To use the wrappers you need to import Kodi API <code>xbmc*</code> modules from Kodi Six instead of importing them directly:

<source lang="python">
from kodi_six import xbmc, xbmcaddon, xbmcgui, xbmcplugin, xbmcvfs
</source>

=== PyCharm ===

PyCharm, except for being a really good Python IDE, also provides code compatibility inspection that helps you write Python 2/3-portable code. Open '''Settings''' (Alt+F7) > '''Editor''' > '''Inspections''' > '''Python''' > '''Code compatibility inspections''' and check Python version that you want to support.

PyCharm Community Edition is free and provides all features you need for creating Python addons for Kodi.

== Writing Portable Code ==

Here I’ll give you some tips about how to write portable code:

* Learn the differences between Python 2 and 3. You need to know at least the most important differences between the two major Python versions.

* Use version control system — git or mercurial — to track changes in your code. If you are porting existing code, do it in a separate branch.

* No matter if you are going to write a brand new addon or to port existing addon to Python 3, carefully choose your tools. It is totally possible to write portable code without any helper tools and libraries, but you need to know what you are doing. However, in most cases I’d recommend you to use Future library and its utilities. Carefully read Future documentation.

* Put the following line at the beginning of all your modules:
<source lang="python">
from __future__ import absolute_import, division, unicode_literals
</source>
This will enable respective Python 3 features in your code. You don’t need to import print_function (another Python 3 feature) because in Kodi addons you should to use xbmc.log() to write messages to the Kodi log file.

=== Writing New Addons ===

Check the [http://python-future.org/quickstart.html#if-you-are-writing-code-from-scratch Quick Start Guide] section of Future library documentation. You can also use Six library, but, as it was said previously, you need to learn its API to pick the necessary features to address specific Python version differences, while Future allows to write your code in (almost) idiomatic Python 3.

A brief procedure for writing new Python 2/3-compatible addons with Future library:

# Create a new virtual environment with Python 3 interpreter and activate it.
# Install Future library: <code>pip install future</code>.
# Point your IDE (Integrated Development Environment) to that environment. For example, in PyCharm: '''File''' > '''Setting''' > '''Project''' > '''Project Interpreter'''.
# Put the following line at the beginning of your Python code:
<source lang="python">
from __future__ import absolute_import, division, unicode_literals
from future import standard_library
from future.builtins import *
standard_library.install_aliases()
</source>
Write your addon using Python 3 syntax and standard library names. However, you may still need to use wrappers from Future library for some specific cases (e.g. iterators over <code>dict</code> elements). Read Future docs for more info.

=== Porting Existing Addons ===

A brief procedure for porting existing code to Python 3-compatible with Future library:

# Install Future library into your working Python 2 virtual environment: <code>pip install future</code>.
# Run <code>futurize</code> utility to convert all your scripts and modules to portable code.

Test your new or converted addon in Kodi with Python 2 interpreter and fix all found issues. After that test the addon in Kodi with Python 3 interpreter and again fix all found issues.
You can use [https://github.com/romanvm/kodi.web-pdb Web-PDB] debugger for troubleshooting issues in your code. It is compatible with both Python 2 and 3.

== Links ==

* Python 2 and 3 differences: https://docs.python.org/3.6/whatsnew/3.0.html
* Future library documentation: http://python-future.org/
* Six library documentation: https://pythonhosted.org/six/
* Kodi test builds with Python 3 for Windows: http://mirrors.xbmc.org/test-builds/windows/win32/
* Kodi test builds with Python 3 for Linux Ubuntu: https://launchpad.net/~wsnipex/+archive/ubuntu/kodi-python3/+packages
* “Python 3 migration” section on the Kodi official forum: https://forum.kodi.tv/forumdisplay.php?fid=281

General information about migration to Python 3

2018-02-04T13:57:34Z

Roman V M: /* Absolute Import */

== Overview ==

Python 3 was released 9 years ago and EOL for Python 2.7 is scheduled for 2020. Currently more and more Python project are switching to Python 3 or 2/3 compatible code. The issue of migrating Kodi’s built-in Python interpreter to Python 3.x was brought up on the Kodi forum several times. Now, thanks to a successful GSOC 2017 project, we have a working Python 3 interpreter for Kodi. That is why on Kodi DevCon 2017 Team Kodi decided that we will switch to Python 3 in the future Kodi release (v.19 preliminarily).

Unfortunately, Python 3.x versions are not backward-compatible with Python 2.x, so we decided to urge addon developers to start porting their addons to Python 3 by making the addons compatible with both Python 2 and 3 and submitting them to the official repo. This way, when Kodi with Python 3 is released, we won’t have a situation when we have no working addons.

== The Process ==

* Starting from Kodi 18 (Leia), only addons that are compatible with both Python 2 and 3 will be accepted to the official addon repository.
* We will provide Kodi builds with Python 3 so that addon developers can tests if their addons are really compatible with Python 3.
* A thread will be created on the Kodi forum so that addon devs can get help with their migration process.

== Python 3 And 2 Differences ==

Below is a brief overview of the main differences between Python 3 and 2.

=== Unicode strings by default ===

In Python 2 built-in <code>str</code> type holds the sequence of bytes so it can be used to store both binary data and textual data in ASCII or any of 8-bit fixed-length encodings (there were plenty). Python 2 also has <code>unicode</code> type that can store text in various writing systems. A minimal unit of a <code>unicode</code> object is a Unicode codepoint (a Unicode character). Both <code>str</code> and <code>unicode</code> objects can be mixed and matched together, for example, concatenated using <code>+</code> operator.

In Python 3 <code>str</code> type holds Unicode characters, and for binary data a new <code>bytes</code> type was added. This type was also added to Python 2.7 to simplify porting to Python 3 but in Python 2.7 bytes is simply an alias for <code>str</code>. The <code>str</code> and <code>bytes</code> types are incompatible, and an attempt to use both together will result in <code>TypeError</code> exception. A <code>bytes</code> object can be converted to <code>str</code> using .decode() method, and a <code>str</code> is converted to <code>bytes</code> using .encode() method.

=== Absolute Import ===
Python 3 uses so called “absolute import” and you can no longer import modules from inside a package using only a module name. You need to either to use either a fully qualified name or a “dotted” notation.

Let’s assume that you have a <code>foo</code> package that contains <code>bar</code> and <code>spam</code> modules:
<pre>
foo/
__init__.py
bar.py
spam.py
</pre>
And you need to import <code>bar</code> module from <code>spam</code>. In Python 2 you can do:
<source lang="python">
import bar
</source>

But in Python 3 you need to use either

<source lang="python">
import foo.bar
</source>

or

<source lang="python">
from . import bar
</source>

=== Floating Point Division ===

In Python 2 applying division operator to int numbers produces int result. For example:

<source lang="python">
>>> 4 / 3
1
</source>

In Python 3 division operator always gives you a float result.

<source lang="python">
>>> 4/3
1.3333333333333333
</source>

To get an int result you need to use integer division operator:

<source lang="python">
>>> 4 // 3
1
</source>

=== Print Function ===

In Python 3 print statement has been replaced with print() function. This is not very relevant to Kodi because using print in Python addons is discouraged and xbmc.log() function should be used to write messages to the Kodi log.

=== Default Iterators ===

In Python 3 most function and methods that produce sequences return iterators instead of lists. For example, range() function and .items(), .keys(), .values() of dict type produce iterators instead of lists, and their Python 2 analogues — xrange() function and .iter* methods of dict type — have been removed.

=== No long Integers ===

In Python 3 long integer type has been removed and int type can now hold a number of arbitrary length.

Those are only some of the differences between Python 2 and 3. For a more complete list see [https://docs.python.org/3.6/whatsnew/3.0.html official Python documentation].

== Some Useful Information ==

As it was said above, Python 3 includes a number of incompatible changes, so to avoid situation when after upgrading Kodi for Python 3 support most Python addons will be broken developers should start to port their addons to be compatible with both Python 2 and 3. This is not a trivial task but fortunately there are a number of tools and recommendations to simplify this process. Here I’ll give you some advice and information about tools that will simplify creating portable code that runs on both Python 2 and 3.

=== Know Your Strings! ===

This is the most problematic part of porting Python 2 code to Python 3 so I put it first. One of the most notable differences between Python 2 and 3 is clear separation between “textual” and “binary” strings, that is, between textual data and their binary representation. This difference is often difficult to understand, especially for novice developers (not only in Python), and the fact that in Python 2 you can mix together str (binary data) and unicode (textual data) adds to that difficulty.

There are many explanations in the Internet, but here are the most basic things about strings that you need to know:

“Textual” or Unicode strings (unicode in Python 2 and str in Python 3) hold text as a sequence of characters. A minimal unit in such strings is a Unicode character — an abstract entity that represent an alphabet symbol, a punctuation sign or other symbol included in the Unicode character table. Of course, such symbols have concrete internal binary representation (a computer is a binary machine after all), but this representation is not relevant to us. All you need to know is that “textual” strings contain text units without being tied to concrete binary representation.

Unlike Unicode strings, binary strings (str in Python 2 and bytes in both Python 2 and 3) hold binary data, and a minimal unit of such strings is a byte or 8 bits. Historically, in Python 2 such strings are used for textual data encoded in ASCII or other 1-byte fixed-length encoding (e.g. Windows 1251 for Cyrillic alphabets). However, this approach has its limitation, that is why unicode type was introduced in Python 2, and in Python 3 it was renamed to str and made a default container for textual data.

Unfortunately, Python 2 allows to mix and match Unicode and binary strings in the same context and this creates big problems when porting Python 2 code to Python 3. So in order to successfully port your code to Python 3 you need to carefully consider how textual data are processed in your addon. The following recommendations will help you avoid problems when writing Python 2/3 compatible code.

=== Avoid using binary strings for text ===

Store all your text as Unicode strings. This means that all binary strings received from external sources (textual files, websites, various APIs) need to be decoded using appropriate encoding (UTF-8 in most cases). If you need to use strings literals (although using hardcoded strings for user-facing text in Kodi is strongly discouraged), they need to be defined as unicode strings as well. You can use either
<source lang="python">
from __future__ import unicode_literals
</source>
at the beginning of your modules to enable unicode strings by default as in Python 3 or u-strings, e.g. <code>u'foo'</code> (Python 3 understands them too) — it doesn't really matter as long as you are doing it consistently.

=== Know the libraries you are using ===
Working with Python Standard Library and third-party libraries (including Kodi Python API) you should know that types their functions and methods expect and what types they return. For example, in popular requests library Response.text property returns a Unicode string and Response.content returns a binary string.

If a function/method/property returns a binary string, you need to decode it to a Unicode string using .decode() method. Don’t use unicode class constructor because unicode type does not exist in Python 3.

If you are reading textual files from disk, it is better to use io.open() instead of built-in open(). io.open() can decode file contents to Unicode using specified encoding and works identically both in Python 2 and 3. Example:
<source lang="python">
import io

with io.open('foo.txt', 'r', encoding='utf-8') as fo:
text = fo.read()
</source>

=== Potential pitfalls ===

When writing Python 2/3-compatible code or converting existing Python 2 codebase to compatible code you should pay attention to the following Python language constructions where TypeError exceptions may happen because of incompatible string types if you don’t get your strings in order:

* String concatenations with + operator (<code>'foo' + 'bar'</code>).
* String formatting — both “old style” (<code>'foo %s' % 'bar'</code>) and “new style” (<code>'foo {}'.format('bar')</code>).
* String joining (<code>', '.join(['foo', 'bar'])</code>).

In all those cases always make sure that you work with Unicode strings only.
Another source of subtle bugs is functions and methods that accept and return binary strings (str type) in Python 2 but Unicode strings in Python 3 (again, str type but with different properties). You can use the following utility functions to “normalize” strings that are provided to such functions/methods and received from them:

<source lang="python">
import sys

__all__ = ['PY2', 'py2_encode', 'py2_decode']

PY2 = sys.version_info[0] == 2

def py2_encode(s, encoding='utf-8'):
"""
Encode Python 2 ``unicode`` to ``str``

In Python 3 the string is not changed.
"""
if PY2 and isinstance(s, unicode):
s = s.encode(encoding)
return s

def py2_decode(s, encoding='utf-8'):
"""
Decode Python 2 ``str`` to ``unicode``

In Python 3 the string is not changed.
"""
if PY2 and isinstance(s, str):
s = s.decode(encoding)
return s
</source>

Those utility functions are included in Kodi Six library that is described in the following section.

== Helper Libraries and Tools ==

=== 2to3 ===

[https://docs.python.org/2/library/2to3.html 2to3] script is created by Python developers to help converting existing Python 2 code to Python 3. On Windows it is included in Python distribution (Tools subfolder) but on other OSes you man need to install it separately. For example, on Ubuntu it is included in python-examples package. Note that this script is created for conversion from Python 2 to 3, not for writing portable code, so you need to treat its output with care.

=== Modernize ===

https://python-modernize.readthedocs.io/en/latest/ Modernize] script works on top of 2to3 and is supposed to help you to convert your existing Python 2 code to Python 2/3-compatible. However, in my experiments it did not work reliably even in simple cases so I cannot recommend it for usage.

=== Six ===

[https://pythonhosted.org/six/ Six] was the first library developed to simplify Python 2 to 3 migration. It provides a set of wrappers that hide differences between Python 2 and 3 behind its API. It is less intrusive than the following alternative because it does not monkey-patch built-in names, but in order to use Six library effectively you need to learn its API. Six can be used both for writing new Python 2/3-compatible addons and converting existing ones, but it requires good knowledge of Python 2/3 differences to pick necessary Six features that address specific differences.

Six library is included in the Kodi addon repository as '''script.module.six''' addon.

=== Future ===

[http://python-future.org/ Future] library, like Six, was created to simplify porting existing Python 2 code to 3. But it uses a different approach than Six. Future monkey-patches built-in Python objects to make Python 2 behave like Python 3. The advantage of such approach is that code written using Future is close to idiomatic Python 3 and requires little re-work if you decide to drop Python 2 support in the future. However, such approach may cause problems in some rare edge-cases.

Future library also includes <code>futurize</code> command-line utility for converting existing Python 2 code to 2/3-compatible and in my experiments this utility showed good results.
Future library (without additional utilities) is included in the Kodi addon repository as script.module.future addon.

=== Kodi Six ===

[https://github.com/romanvm/kodi.six Kodi Six] library (available as '''script.module.kodi-six''' addon) is created to normalize string handling in Kodi Python API based on different Python versions by applying wrappers around Kodi API functions and classes. In Kodi API based on Python 2 Kodi Six wrappers have the following effects:

* All functions and methods that expect string arguments can accept both UTF-8 encoded <code>str</code> objects and <code>unicode</code> objects.
* All functions and methods that return string data return <code>unicode</code> objects.

In Kodi API based on Python 3 the wrappers have no effect because Python 3-based API works only with <code>str</code> (Unicode) objects. This eliminates the need for ad hoc encoding and decoding string when working with Kodi API. To use the wrappers you need to import Kodi API <code>xbmc*</code> modules from Kodi Six instead of importing them directly:

<source lang="python">
from kodi_six import xbmc, xbmcaddon, xbmcgui, xbmcplugin, xbmcvfs
</source>

=== PyCharm ===

PyCharm, except for being a really good Python IDE, also provides code compatibility inspection that helps you write Python 2/3-portable code. Open '''Settings''' (Alt+F7) > '''Editor''' > '''Inspections''' > '''Python''' > '''Code compatibility inspections''' and check Python version that you want to support.

PyCharm Community Edition is free and provides all features you need for creating Python addons for Kodi.

== Writing Portable Code ==

Here I’ll give you some tips about how to write portable code:

* Learn the differences between Python 2 and 3. You need to know at least the most important differences between the two major Python versions.

* Use version control system — git or mercurial — to track changes in your code. If you are porting existing code, do it in a separate branch.

* No matter if you are going to write a brand new addon or to port existing addon to Python 3, carefully choose your tools. It is totally possible to write portable code without any helper tools and libraries, but you need to know what you are doing. However, in most cases I’d recommend you to use Future library and its utilities. Carefully read Future documentation.

* Put the following line at the beginning of all your modules:
<source lang="python">
from __future__ import absolute_import, division, unicode_literals
</source>
This will enable respective Python 3 features in your code. You don’t need to import print_function (another Python 3 feature) because in Kodi addons you should to use xbmc.log() to write messages to the Kodi log file.

=== Writing New Addons ===

Check the [http://python-future.org/quickstart.html#if-you-are-writing-code-from-scratch Quick Start Guide] section of Future library documentation. You can also use Six library, but, as it was said previously, you need to learn its API to pick the necessary features to address specific Python version differences, while Future allows to write your code in (almost) idiomatic Python 3.

A brief procedure for writing new Python 2/3-compatible addons with Future library:

# Create a new virtual environment with Python 3 interpreter and activate it.
# Install Future library: <code>pip install future</code>.
# Point your IDE (Integrated Development Environment) to that environment. For example, in PyCharm: '''File''' > '''Setting''' > '''Project''' > '''Project Interpreter'''.
# Put the following line at the beginning of your Python code:
<source lang="python">
from __future__ import absolute_import, division, unicode_literals
from future import standard_library
from future.builtins import *
standard_library.install_aliases()
</source>
Write your addon using Python 3 syntax and standard library names. However, you may still need to use wrappers from Future library for some specific cases (e.g. iterators over <code>dict</code> elements). Read Future docs for more info.

=== Porting Existing Addons ===

A brief procedure for porting existing code to Python 3-compatible with Future library:

# Install Future library into your working Python 2 virtual environment: <code>pip install future</code>.
# Run <code>futurize</code> utility to convert all your scripts and modules to portable code.

Test your new or converted addon in Kodi with Python 2 interpreter and fix all found issues. After that test the addon in Kodi with Python 3 interpreter and again fix all found issues.
You can use [https://github.com/romanvm/kodi.web-pdb Web-PDB] debugger for troubleshooting issues in your code. It is compatible with both Python 2 and 3.

== Links ==

* Python 2 and 3 differences: https://docs.python.org/3.6/whatsnew/3.0.html
* Future library documentation: http://python-future.org/
* Six library documentation: https://pythonhosted.org/six/
* Kodi test builds with Python 3 for Windows: http://mirrors.xbmc.org/test-builds/windows/win32/
* Kodi test builds with Python 3 for Linux Ubuntu: https://launchpad.net/~wsnipex/+archive/ubuntu/kodi-python3/+packages
* “Python 3 migration” section on the Kodi official forum: https://forum.kodi.tv/forumdisplay.php?fid=281

General information about migration to Python 3

2018-01-19T11:55:41Z

Roman V M: /* Potential pitfalls */

== Overview ==

Python 3 was released 9 years ago and EOL for Python 2.7 is scheduled for 2020. Currently more and more Python project are switching to Python 3 or 2/3 compatible code. The issue of migrating Kodi’s built-in Python interpreter to Python 3.x was brought up on the Kodi forum several times. Now, thanks to a successful GSOC 2017 project, we have a working Python 3 interpreter for Kodi. That is why on Kodi DevCon 2017 Team Kodi decided that we will switch to Python 3 in the future Kodi release (v.19 preliminarily).

Unfortunately, Python 3.x versions are not backward-compatible with Python 2.x, so we decided to urge addon developers to start porting their addons to Python 3 by making the addons compatible with both Python 2 and 3 and submitting them to the official repo. This way, when Kodi with Python 3 is released, we won’t have a situation when we have no working addons.

== The Process ==

* Starting from Kodi 18 (Leia), only addons that are compatible with both Python 2 and 3 will be accepted to the official addon repository.
* We will provide Kodi builds with Python 3 so that addon developers can tests if their addons are really compatible with Python 3.
* A thread will be created on the Kodi forum so that addon devs can get help with their migration process.

== Python 3 And 2 Differences ==

Below is a brief overview of the main differences between Python 3 and 2.

=== Unicode strings by default ===

In Python 2 built-in <code>str</code> type holds the sequence of bytes so it can be used to store both binary data and textual data in ASCII or any of 8-bit fixed-length encodings (there were plenty). Python 2 also has <code>unicode</code> type that can store text in various writing systems. A minimal unit of a <code>unicode</code> object is a Unicode codepoint (a Unicode character). Both <code>str</code> and <code>unicode</code> objects can be mixed and matched together, for example, concatenated using <code>+</code> operator.

In Python 3 <code>str</code> type holds Unicode characters, and for binary data a new <code>bytes</code> type was added. This type was also added to Python 2.7 to simplify porting to Python 3 but in Python 2.7 bytes is simply an alias for <code>str</code>. The <code>str</code> and <code>bytes</code> types are incompatible, and an attempt to use both together will result in <code>TypeError</code> exception. A <code>bytes</code> object can be converted to <code>str</code> using .decode() method, and a <code>str</code> is converted to <code>bytes</code> using .encode() method.

=== Absolute Import ===
Python 3 uses so called “absolute import” and you can no longer import modules from inside a package using only a module name. You need to either to use either a fully qualified name or a “dotted” notation.

Let’s assume that you have a foo package that contains bar and spam modules, and you need to import bar module from spam. In Python 2 you can do:
<source lang="python">
import bar
</source>

But in Python 3 you need to use either

<source lang="python">
import foo.bar
</source>

or

<source lang="python">
from . import bar
</source>

=== Floating Point Division ===

In Python 2 applying division operator to int numbers produces int result. For example:

<source lang="python">
>>> 4 / 3
1
</source>

In Python 3 division operator always gives you a float result.

<source lang="python">
>>> 4/3
1.3333333333333333
</source>

To get an int result you need to use integer division operator:

<source lang="python">
>>> 4 // 3
1
</source>

=== Print Function ===

In Python 3 print statement has been replaced with print() function. This is not very relevant to Kodi because using print in Python addons is discouraged and xbmc.log() function should be used to write messages to the Kodi log.

=== Default Iterators ===

In Python 3 most function and methods that produce sequences return iterators instead of lists. For example, range() function and .items(), .keys(), .values() of dict type produce iterators instead of lists, and their Python 2 analogues — xrange() function and .iter* methods of dict type — have been removed.

=== No long Integers ===

In Python 3 long integer type has been removed and int type can now hold a number of arbitrary length.

Those are only some of the differences between Python 2 and 3. For a more complete list see [https://docs.python.org/3.6/whatsnew/3.0.html official Python documentation].

== Some Useful Information ==

As it was said above, Python 3 includes a number of incompatible changes, so to avoid situation when after upgrading Kodi for Python 3 support most Python addons will be broken developers should start to port their addons to be compatible with both Python 2 and 3. This is not a trivial task but fortunately there are a number of tools and recommendations to simplify this process. Here I’ll give you some advice and information about tools that will simplify creating portable code that runs on both Python 2 and 3.

=== Know Your Strings! ===

This is the most problematic part of porting Python 2 code to Python 3 so I put it first. One of the most notable differences between Python 2 and 3 is clear separation between “textual” and “binary” strings, that is, between textual data and their binary representation. This difference is often difficult to understand, especially for novice developers (not only in Python), and the fact that in Python 2 you can mix together str (binary data) and unicode (textual data) adds to that difficulty.

There are many explanations in the Internet, but here are the most basic things about strings that you need to know:

“Textual” or Unicode strings (unicode in Python 2 and str in Python 3) hold text as a sequence of characters. A minimal unit in such strings is a Unicode character — an abstract entity that represent an alphabet symbol, a punctuation sign or other symbol included in the Unicode character table. Of course, such symbols have concrete internal binary representation (a computer is a binary machine after all), but this representation is not relevant to us. All you need to know is that “textual” strings contain text units without being tied to concrete binary representation.

Unlike Unicode strings, binary strings (str in Python 2 and bytes in both Python 2 and 3) hold binary data, and a minimal unit of such strings is a byte or 8 bits. Historically, in Python 2 such strings are used for textual data encoded in ASCII or other 1-byte fixed-length encoding (e.g. Windows 1251 for Cyrillic alphabets). However, this approach has its limitation, that is why unicode type was introduced in Python 2, and in Python 3 it was renamed to str and made a default container for textual data.

Unfortunately, Python 2 allows to mix and match Unicode and binary strings in the same context and this creates big problems when porting Python 2 code to Python 3. So in order to successfully port your code to Python 3 you need to carefully consider how textual data are processed in your addon. The following recommendations will help you avoid problems when writing Python 2/3 compatible code.

=== Avoid using binary strings for text ===

Store all your text as Unicode strings. This means that all binary strings received from external sources (textual files, websites, various APIs) need to be decoded using appropriate encoding (UTF-8 in most cases). If you need to use strings literals (although using hardcoded strings for user-facing text in Kodi is strongly discouraged), they need to be defined as unicode strings as well. You can use either
<source lang="python">
from __future__ import unicode_literals
</source>
at the beginning of your modules to enable unicode strings by default as in Python 3 or u-strings, e.g. <code>u'foo'</code> (Python 3 understands them too) — it doesn't really matter as long as you are doing it consistently.

=== Know the libraries you are using ===
Working with Python Standard Library and third-party libraries (including Kodi Python API) you should know that types their functions and methods expect and what types they return. For example, in popular requests library Response.text property returns a Unicode string and Response.content returns a binary string.

If a function/method/property returns a binary string, you need to decode it to a Unicode string using .decode() method. Don’t use unicode class constructor because unicode type does not exist in Python 3.

If you are reading textual files from disk, it is better to use io.open() instead of built-in open(). io.open() can decode file contents to Unicode using specified encoding and works identically both in Python 2 and 3. Example:
<source lang="python">
import io

with io.open('foo.txt', 'r', encoding='utf-8') as fo:
text = fo.read()
</source>

=== Potential pitfalls ===

When writing Python 2/3-compatible code or converting existing Python 2 codebase to compatible code you should pay attention to the following Python language constructions where TypeError exceptions may happen because of incompatible string types if you don’t get your strings in order:

* String concatenations with + operator (<code>'foo' + 'bar'</code>).
* String formatting — both “old style” (<code>'foo %s' % 'bar'</code>) and “new style” (<code>'foo {}'.format('bar')</code>).
* String joining (<code>', '.join(['foo', 'bar'])</code>).

In all those cases always make sure that you work with Unicode strings only.
Another source of subtle bugs is functions and methods that accept and return binary strings (str type) in Python 2 but Unicode strings in Python 3 (again, str type but with different properties). You can use the following utility functions to “normalize” strings that are provided to such functions/methods and received from them:

<source lang="python">
import sys

__all__ = ['PY2', 'py2_encode', 'py2_decode']

PY2 = sys.version_info[0] == 2

def py2_encode(s, encoding='utf-8'):
"""
Encode Python 2 ``unicode`` to ``str``

In Python 3 the string is not changed.
"""
if PY2 and isinstance(s, unicode):
s = s.encode(encoding)
return s

def py2_decode(s, encoding='utf-8'):
"""
Decode Python 2 ``str`` to ``unicode``

In Python 3 the string is not changed.
"""
if PY2 and isinstance(s, str):
s = s.decode(encoding)
return s
</source>

Those utility functions are included in Kodi Six library that is described in the following section.

== Helper Libraries and Tools ==

=== 2to3 ===

[https://docs.python.org/2/library/2to3.html 2to3] script is created by Python developers to help converting existing Python 2 code to Python 3. On Windows it is included in Python distribution (Tools subfolder) but on other OSes you man need to install it separately. For example, on Ubuntu it is included in python-examples package. Note that this script is created for conversion from Python 2 to 3, not for writing portable code, so you need to treat its output with care.

=== Modernize ===

https://python-modernize.readthedocs.io/en/latest/ Modernize] script works on top of 2to3 and is supposed to help you to convert your existing Python 2 code to Python 2/3-compatible. However, in my experiments it did not work reliably even in simple cases so I cannot recommend it for usage.

=== Six ===

[https://pythonhosted.org/six/ Six] was the first library developed to simplify Python 2 to 3 migration. It provides a set of wrappers that hide differences between Python 2 and 3 behind its API. It is less intrusive than the following alternative because it does not monkey-patch built-in names, but in order to use Six library effectively you need to learn its API. Six can be used both for writing new Python 2/3-compatible addons and converting existing ones, but it requires good knowledge of Python 2/3 differences to pick necessary Six features that address specific differences.

Six library is included in the Kodi addon repository as '''script.module.six''' addon.

=== Future ===

[http://python-future.org/ Future] library, like Six, was created to simplify porting existing Python 2 code to 3. But it uses a different approach than Six. Future monkey-patches built-in Python objects to make Python 2 behave like Python 3. The advantage of such approach is that code written using Future is close to idiomatic Python 3 and requires little re-work if you decide to drop Python 2 support in the future. However, such approach may cause problems in some rare edge-cases.

Future library also includes <code>futurize</code> command-line utility for converting existing Python 2 code to 2/3-compatible and in my experiments this utility showed good results.
Future library (without additional utilities) is included in the Kodi addon repository as script.module.future addon.

=== Kodi Six ===

[https://github.com/romanvm/kodi.six Kodi Six] library (available as '''script.module.kodi-six''' addon) is created to normalize string handling in Kodi Python API based on different Python versions by applying wrappers around Kodi API functions and classes. In Kodi API based on Python 2 Kodi Six wrappers have the following effects:

* All functions and methods that expect string arguments can accept both UTF-8 encoded <code>str</code> objects and <code>unicode</code> objects.
* All functions and methods that return string data return <code>unicode</code> objects.

In Kodi API based on Python 3 the wrappers have no effect because Python 3-based API works only with <code>str</code> (Unicode) objects. This eliminates the need for ad hoc encoding and decoding string when working with Kodi API. To use the wrappers you need to import Kodi API <code>xbmc*</code> modules from Kodi Six instead of importing them directly:

<source lang="python">
from kodi_six import xbmc, xbmcaddon, xbmcgui, xbmcplugin, xbmcvfs
</source>

=== PyCharm ===

PyCharm, except for being a really good Python IDE, also provides code compatibility inspection that helps you write Python 2/3-portable code. Open '''Settings''' (Alt+F7) > '''Editor''' > '''Inspections''' > '''Python''' > '''Code compatibility inspections''' and check Python version that you want to support.

PyCharm Community Edition is free and provides all features you need for creating Python addons for Kodi.

== Writing Portable Code ==

Here I’ll give you some tips about how to write portable code:

* Learn the differences between Python 2 and 3. You need to know at least the most important differences between the two major Python versions.

* Use version control system — git or mercurial — to track changes in your code. If you are porting existing code, do it in a separate branch.

* No matter if you are going to write a brand new addon or to port existing addon to Python 3, carefully choose your tools. It is totally possible to write portable code without any helper tools and libraries, but you need to know what you are doing. However, in most cases I’d recommend you to use Future library and its utilities. Carefully read Future documentation.

* Put the following line at the beginning of all your modules:
<source lang="python">
from __future__ import absolute_import, division, unicode_literals
</source>
This will enable respective Python 3 features in your code. You don’t need to import print_function (another Python 3 feature) because in Kodi addons you should to use xbmc.log() to write messages to the Kodi log file.

=== Writing New Addons ===

Check the [http://python-future.org/quickstart.html#if-you-are-writing-code-from-scratch Quick Start Guide] section of Future library documentation. You can also use Six library, but, as it was said previously, you need to learn its API to pick the necessary features to address specific Python version differences, while Future allows to write your code in (almost) idiomatic Python 3.

A brief procedure for writing new Python 2/3-compatible addons with Future library:

# Create a new virtual environment with Python 3 interpreter and activate it.
# Install Future library: <code>pip install future</code>.
# Point your IDE (Integrated Development Environment) to that environment. For example, in PyCharm: '''File''' > '''Setting''' > '''Project''' > '''Project Interpreter'''.
# Put the following line at the beginning of your Python code:
<source lang="python">
from __future__ import absolute_import, division, unicode_literals
from future import standard_library
from future.builtins import *
standard_library.install_aliases()
</source>
Write your addon using Python 3 syntax and standard library names. However, you may still need to use wrappers from Future library for some specific cases (e.g. iterators over <code>dict</code> elements). Read Future docs for more info.

=== Porting Existing Addons ===

A brief procedure for porting existing code to Python 3-compatible with Future library:

# Install Future library into your working Python 2 virtual environment: <code>pip install future</code>.
# Run <code>futurize</code> utility to convert all your scripts and modules to portable code.

Test your new or converted addon in Kodi with Python 2 interpreter and fix all found issues. After that test the addon in Kodi with Python 3 interpreter and again fix all found issues.
You can use [https://github.com/romanvm/kodi.web-pdb Web-PDB] debugger for troubleshooting issues in your code. It is compatible with both Python 2 and 3.

== Links ==

* Python 2 and 3 differences: https://docs.python.org/3.6/whatsnew/3.0.html
* Future library documentation: http://python-future.org/
* Six library documentation: https://pythonhosted.org/six/
* Kodi test builds with Python 3 for Windows: http://mirrors.xbmc.org/test-builds/windows/win32/
* Kodi test builds with Python 3 for Linux Ubuntu: https://launchpad.net/~wsnipex/+archive/ubuntu/kodi-python3/+packages
* “Python 3 migration” section on the Kodi official forum: https://forum.kodi.tv/forumdisplay.php?fid=281

General information about migration to Python 3

2018-01-19T11:54:32Z

Roman V M: /* Helper Libraries and Tools */

== Overview ==

Python 3 was released 9 years ago and EOL for Python 2.7 is scheduled for 2020. Currently more and more Python project are switching to Python 3 or 2/3 compatible code. The issue of migrating Kodi’s built-in Python interpreter to Python 3.x was brought up on the Kodi forum several times. Now, thanks to a successful GSOC 2017 project, we have a working Python 3 interpreter for Kodi. That is why on Kodi DevCon 2017 Team Kodi decided that we will switch to Python 3 in the future Kodi release (v.19 preliminarily).

Unfortunately, Python 3.x versions are not backward-compatible with Python 2.x, so we decided to urge addon developers to start porting their addons to Python 3 by making the addons compatible with both Python 2 and 3 and submitting them to the official repo. This way, when Kodi with Python 3 is released, we won’t have a situation when we have no working addons.

== The Process ==

* Starting from Kodi 18 (Leia), only addons that are compatible with both Python 2 and 3 will be accepted to the official addon repository.
* We will provide Kodi builds with Python 3 so that addon developers can tests if their addons are really compatible with Python 3.
* A thread will be created on the Kodi forum so that addon devs can get help with their migration process.

== Python 3 And 2 Differences ==

Below is a brief overview of the main differences between Python 3 and 2.

=== Unicode strings by default ===

In Python 2 built-in <code>str</code> type holds the sequence of bytes so it can be used to store both binary data and textual data in ASCII or any of 8-bit fixed-length encodings (there were plenty). Python 2 also has <code>unicode</code> type that can store text in various writing systems. A minimal unit of a <code>unicode</code> object is a Unicode codepoint (a Unicode character). Both <code>str</code> and <code>unicode</code> objects can be mixed and matched together, for example, concatenated using <code>+</code> operator.

In Python 3 <code>str</code> type holds Unicode characters, and for binary data a new <code>bytes</code> type was added. This type was also added to Python 2.7 to simplify porting to Python 3 but in Python 2.7 bytes is simply an alias for <code>str</code>. The <code>str</code> and <code>bytes</code> types are incompatible, and an attempt to use both together will result in <code>TypeError</code> exception. A <code>bytes</code> object can be converted to <code>str</code> using .decode() method, and a <code>str</code> is converted to <code>bytes</code> using .encode() method.

=== Absolute Import ===
Python 3 uses so called “absolute import” and you can no longer import modules from inside a package using only a module name. You need to either to use either a fully qualified name or a “dotted” notation.

Let’s assume that you have a foo package that contains bar and spam modules, and you need to import bar module from spam. In Python 2 you can do:
<source lang="python">
import bar
</source>

But in Python 3 you need to use either

<source lang="python">
import foo.bar
</source>

or

<source lang="python">
from . import bar
</source>

=== Floating Point Division ===

In Python 2 applying division operator to int numbers produces int result. For example:

<source lang="python">
>>> 4 / 3
1
</source>

In Python 3 division operator always gives you a float result.

<source lang="python">
>>> 4/3
1.3333333333333333
</source>

To get an int result you need to use integer division operator:

<source lang="python">
>>> 4 // 3
1
</source>

=== Print Function ===

In Python 3 print statement has been replaced with print() function. This is not very relevant to Kodi because using print in Python addons is discouraged and xbmc.log() function should be used to write messages to the Kodi log.

=== Default Iterators ===

In Python 3 most function and methods that produce sequences return iterators instead of lists. For example, range() function and .items(), .keys(), .values() of dict type produce iterators instead of lists, and their Python 2 analogues — xrange() function and .iter* methods of dict type — have been removed.

=== No long Integers ===

In Python 3 long integer type has been removed and int type can now hold a number of arbitrary length.

Those are only some of the differences between Python 2 and 3. For a more complete list see [https://docs.python.org/3.6/whatsnew/3.0.html official Python documentation].

== Some Useful Information ==

As it was said above, Python 3 includes a number of incompatible changes, so to avoid situation when after upgrading Kodi for Python 3 support most Python addons will be broken developers should start to port their addons to be compatible with both Python 2 and 3. This is not a trivial task but fortunately there are a number of tools and recommendations to simplify this process. Here I’ll give you some advice and information about tools that will simplify creating portable code that runs on both Python 2 and 3.

=== Know Your Strings! ===

This is the most problematic part of porting Python 2 code to Python 3 so I put it first. One of the most notable differences between Python 2 and 3 is clear separation between “textual” and “binary” strings, that is, between textual data and their binary representation. This difference is often difficult to understand, especially for novice developers (not only in Python), and the fact that in Python 2 you can mix together str (binary data) and unicode (textual data) adds to that difficulty.

There are many explanations in the Internet, but here are the most basic things about strings that you need to know:

“Textual” or Unicode strings (unicode in Python 2 and str in Python 3) hold text as a sequence of characters. A minimal unit in such strings is a Unicode character — an abstract entity that represent an alphabet symbol, a punctuation sign or other symbol included in the Unicode character table. Of course, such symbols have concrete internal binary representation (a computer is a binary machine after all), but this representation is not relevant to us. All you need to know is that “textual” strings contain text units without being tied to concrete binary representation.

Unlike Unicode strings, binary strings (str in Python 2 and bytes in both Python 2 and 3) hold binary data, and a minimal unit of such strings is a byte or 8 bits. Historically, in Python 2 such strings are used for textual data encoded in ASCII or other 1-byte fixed-length encoding (e.g. Windows 1251 for Cyrillic alphabets). However, this approach has its limitation, that is why unicode type was introduced in Python 2, and in Python 3 it was renamed to str and made a default container for textual data.

Unfortunately, Python 2 allows to mix and match Unicode and binary strings in the same context and this creates big problems when porting Python 2 code to Python 3. So in order to successfully port your code to Python 3 you need to carefully consider how textual data are processed in your addon. The following recommendations will help you avoid problems when writing Python 2/3 compatible code.

=== Avoid using binary strings for text ===

Store all your text as Unicode strings. This means that all binary strings received from external sources (textual files, websites, various APIs) need to be decoded using appropriate encoding (UTF-8 in most cases). If you need to use strings literals (although using hardcoded strings for user-facing text in Kodi is strongly discouraged), they need to be defined as unicode strings as well. You can use either
<source lang="python">
from __future__ import unicode_literals
</source>
at the beginning of your modules to enable unicode strings by default as in Python 3 or u-strings, e.g. <code>u'foo'</code> (Python 3 understands them too) — it doesn't really matter as long as you are doing it consistently.

=== Know the libraries you are using ===
Working with Python Standard Library and third-party libraries (including Kodi Python API) you should know that types their functions and methods expect and what types they return. For example, in popular requests library Response.text property returns a Unicode string and Response.content returns a binary string.

If a function/method/property returns a binary string, you need to decode it to a Unicode string using .decode() method. Don’t use unicode class constructor because unicode type does not exist in Python 3.

If you are reading textual files from disk, it is better to use io.open() instead of built-in open(). io.open() can decode file contents to Unicode using specified encoding and works identically both in Python 2 and 3. Example:
<source lang="python">
import io

with io.open('foo.txt', 'r', encoding='utf-8') as fo:
text = fo.read()
</source>

=== Potential pitfalls ===

When writing Python 2/3-compatible code or converting existing Python 2 codebase to compatible code you should pay attention to the following Python language constructions where TypeError exceptions may happen because of incompatible string types if you don’t get your strings in order:

* String concatenations with + operator (<code>'foo' + 'bar'</code>).
* String formatting — both “old style” (<code>'foo %s' % 'bar'</code>) and “new style” (<code>'foo {}'.format('bar')</code>).
* String joining (<code>', '.join(['foo', 'bar'])</code>).

In all those cases always make sure that you work with Unicode strings only.
Another source of subtle bugs is functions and methods that accept and return binary strings (str type) in Python 2 but Unicode strings in Python 3 (again, str type but with different properties). You can use the following utility functions to “normalize” strings that are provided to such functions/methods and received from them:

<source lang="python">
import sys

__all__ = ['PY2', 'py2_encode', 'py2_decode']

PY2 = sys.version_info[0] == 2

def py2_encode(s, encoding='utf-8'):
"""
Encode Python 2 ``unicode`` to ``str``

In Python 3 the string is not changed.
"""
if PY2 and isinstance(s, unicode):
s = s.encode(encoding)
return s

def py2_decode(s, encoding='utf-8'):
"""
Decode Python 2 ``str`` to ``unicode``

In Python 3 the string is not changed.
"""
if PY2 and isinstance(s, str):
s = s.decode(encoding)
return s
</source>

== Helper Libraries and Tools ==

=== 2to3 ===

[https://docs.python.org/2/library/2to3.html 2to3] script is created by Python developers to help converting existing Python 2 code to Python 3. On Windows it is included in Python distribution (Tools subfolder) but on other OSes you man need to install it separately. For example, on Ubuntu it is included in python-examples package. Note that this script is created for conversion from Python 2 to 3, not for writing portable code, so you need to treat its output with care.

=== Modernize ===

https://python-modernize.readthedocs.io/en/latest/ Modernize] script works on top of 2to3 and is supposed to help you to convert your existing Python 2 code to Python 2/3-compatible. However, in my experiments it did not work reliably even in simple cases so I cannot recommend it for usage.

=== Six ===

[https://pythonhosted.org/six/ Six] was the first library developed to simplify Python 2 to 3 migration. It provides a set of wrappers that hide differences between Python 2 and 3 behind its API. It is less intrusive than the following alternative because it does not monkey-patch built-in names, but in order to use Six library effectively you need to learn its API. Six can be used both for writing new Python 2/3-compatible addons and converting existing ones, but it requires good knowledge of Python 2/3 differences to pick necessary Six features that address specific differences.

Six library is included in the Kodi addon repository as '''script.module.six''' addon.

=== Future ===

[http://python-future.org/ Future] library, like Six, was created to simplify porting existing Python 2 code to 3. But it uses a different approach than Six. Future monkey-patches built-in Python objects to make Python 2 behave like Python 3. The advantage of such approach is that code written using Future is close to idiomatic Python 3 and requires little re-work if you decide to drop Python 2 support in the future. However, such approach may cause problems in some rare edge-cases.

Future library also includes <code>futurize</code> command-line utility for converting existing Python 2 code to 2/3-compatible and in my experiments this utility showed good results.
Future library (without additional utilities) is included in the Kodi addon repository as script.module.future addon.

=== Kodi Six ===

[https://github.com/romanvm/kodi.six Kodi Six] library (available as '''script.module.kodi-six''' addon) is created to normalize string handling in Kodi Python API based on different Python versions by applying wrappers around Kodi API functions and classes. In Kodi API based on Python 2 Kodi Six wrappers have the following effects:

* All functions and methods that expect string arguments can accept both UTF-8 encoded <code>str</code> objects and <code>unicode</code> objects.
* All functions and methods that return string data return <code>unicode</code> objects.

In Kodi API based on Python 3 the wrappers have no effect because Python 3-based API works only with <code>str</code> (Unicode) objects. This eliminates the need for ad hoc encoding and decoding string when working with Kodi API. To use the wrappers you need to import Kodi API <code>xbmc*</code> modules from Kodi Six instead of importing them directly:

<source lang="python">
from kodi_six import xbmc, xbmcaddon, xbmcgui, xbmcplugin, xbmcvfs
</source>

=== PyCharm ===

PyCharm, except for being a really good Python IDE, also provides code compatibility inspection that helps you write Python 2/3-portable code. Open '''Settings''' (Alt+F7) > '''Editor''' > '''Inspections''' > '''Python''' > '''Code compatibility inspections''' and check Python version that you want to support.

PyCharm Community Edition is free and provides all features you need for creating Python addons for Kodi.

== Writing Portable Code ==

Here I’ll give you some tips about how to write portable code:

* Learn the differences between Python 2 and 3. You need to know at least the most important differences between the two major Python versions.

* Use version control system — git or mercurial — to track changes in your code. If you are porting existing code, do it in a separate branch.

* No matter if you are going to write a brand new addon or to port existing addon to Python 3, carefully choose your tools. It is totally possible to write portable code without any helper tools and libraries, but you need to know what you are doing. However, in most cases I’d recommend you to use Future library and its utilities. Carefully read Future documentation.

* Put the following line at the beginning of all your modules:
<source lang="python">
from __future__ import absolute_import, division, unicode_literals
</source>
This will enable respective Python 3 features in your code. You don’t need to import print_function (another Python 3 feature) because in Kodi addons you should to use xbmc.log() to write messages to the Kodi log file.

=== Writing New Addons ===

Check the [http://python-future.org/quickstart.html#if-you-are-writing-code-from-scratch Quick Start Guide] section of Future library documentation. You can also use Six library, but, as it was said previously, you need to learn its API to pick the necessary features to address specific Python version differences, while Future allows to write your code in (almost) idiomatic Python 3.

A brief procedure for writing new Python 2/3-compatible addons with Future library:

# Create a new virtual environment with Python 3 interpreter and activate it.
# Install Future library: <code>pip install future</code>.
# Point your IDE (Integrated Development Environment) to that environment. For example, in PyCharm: '''File''' > '''Setting''' > '''Project''' > '''Project Interpreter'''.
# Put the following line at the beginning of your Python code:
<source lang="python">
from __future__ import absolute_import, division, unicode_literals
from future import standard_library
from future.builtins import *
standard_library.install_aliases()
</source>
Write your addon using Python 3 syntax and standard library names. However, you may still need to use wrappers from Future library for some specific cases (e.g. iterators over <code>dict</code> elements). Read Future docs for more info.

=== Porting Existing Addons ===

A brief procedure for porting existing code to Python 3-compatible with Future library:

# Install Future library into your working Python 2 virtual environment: <code>pip install future</code>.
# Run <code>futurize</code> utility to convert all your scripts and modules to portable code.

Test your new or converted addon in Kodi with Python 2 interpreter and fix all found issues. After that test the addon in Kodi with Python 3 interpreter and again fix all found issues.
You can use [https://github.com/romanvm/kodi.web-pdb Web-PDB] debugger for troubleshooting issues in your code. It is compatible with both Python 2 and 3.

== Links ==

* Python 2 and 3 differences: https://docs.python.org/3.6/whatsnew/3.0.html
* Future library documentation: http://python-future.org/
* Six library documentation: https://pythonhosted.org/six/
* Kodi test builds with Python 3 for Windows: http://mirrors.xbmc.org/test-builds/windows/win32/
* Kodi test builds with Python 3 for Linux Ubuntu: https://launchpad.net/~wsnipex/+archive/ubuntu/kodi-python3/+packages
* “Python 3 migration” section on the Kodi official forum: https://forum.kodi.tv/forumdisplay.php?fid=281

General information about migration to Python 3

2018-01-19T09:59:59Z

Roman V M: /* Six */

== Overview ==

Python 3 was released 9 years ago and EOL for Python 2.7 is scheduled for 2020. Currently more and more Python project are switching to Python 3 or 2/3 compatible code. The issue of migrating Kodi’s built-in Python interpreter to Python 3.x was brought up on the Kodi forum several times. Now, thanks to a successful GSOC 2017 project, we have a working Python 3 interpreter for Kodi. That is why on Kodi DevCon 2017 Team Kodi decided that we will switch to Python 3 in the future Kodi release (v.19 preliminarily).

Unfortunately, Python 3.x versions are not backward-compatible with Python 2.x, so we decided to urge addon developers to start porting their addons to Python 3 by making the addons compatible with both Python 2 and 3 and submitting them to the official repo. This way, when Kodi with Python 3 is released, we won’t have a situation when we have no working addons.

== The Process ==

* Starting from Kodi 18 (Leia), only addons that are compatible with both Python 2 and 3 will be accepted to the official addon repository.
* We will provide Kodi builds with Python 3 so that addon developers can tests if their addons are really compatible with Python 3.
* A thread will be created on the Kodi forum so that addon devs can get help with their migration process.

== Python 3 And 2 Differences ==

Below is a brief overview of the main differences between Python 3 and 2.

=== Unicode strings by default ===

In Python 2 built-in <code>str</code> type holds the sequence of bytes so it can be used to store both binary data and textual data in ASCII or any of 8-bit fixed-length encodings (there were plenty). Python 2 also has <code>unicode</code> type that can store text in various writing systems. A minimal unit of a <code>unicode</code> object is a Unicode codepoint (a Unicode character). Both <code>str</code> and <code>unicode</code> objects can be mixed and matched together, for example, concatenated using <code>+</code> operator.

In Python 3 <code>str</code> type holds Unicode characters, and for binary data a new <code>bytes</code> type was added. This type was also added to Python 2.7 to simplify porting to Python 3 but in Python 2.7 bytes is simply an alias for <code>str</code>. The <code>str</code> and <code>bytes</code> types are incompatible, and an attempt to use both together will result in <code>TypeError</code> exception. A <code>bytes</code> object can be converted to <code>str</code> using .decode() method, and a <code>str</code> is converted to <code>bytes</code> using .encode() method.

=== Absolute Import ===
Python 3 uses so called “absolute import” and you can no longer import modules from inside a package using only a module name. You need to either to use either a fully qualified name or a “dotted” notation.

Let’s assume that you have a foo package that contains bar and spam modules, and you need to import bar module from spam. In Python 2 you can do:
<source lang="python">
import bar
</source>

But in Python 3 you need to use either

<source lang="python">
import foo.bar
</source>

or

<source lang="python">
from . import bar
</source>

=== Floating Point Division ===

In Python 2 applying division operator to int numbers produces int result. For example:

<source lang="python">
>>> 4 / 3
1
</source>

In Python 3 division operator always gives you a float result.

<source lang="python">
>>> 4/3
1.3333333333333333
</source>

To get an int result you need to use integer division operator:

<source lang="python">
>>> 4 // 3
1
</source>

=== Print Function ===

In Python 3 print statement has been replaced with print() function. This is not very relevant to Kodi because using print in Python addons is discouraged and xbmc.log() function should be used to write messages to the Kodi log.

=== Default Iterators ===

In Python 3 most function and methods that produce sequences return iterators instead of lists. For example, range() function and .items(), .keys(), .values() of dict type produce iterators instead of lists, and their Python 2 analogues — xrange() function and .iter* methods of dict type — have been removed.

=== No long Integers ===

In Python 3 long integer type has been removed and int type can now hold a number of arbitrary length.

Those are only some of the differences between Python 2 and 3. For a more complete list see [https://docs.python.org/3.6/whatsnew/3.0.html official Python documentation].

== Some Useful Information ==

As it was said above, Python 3 includes a number of incompatible changes, so to avoid situation when after upgrading Kodi for Python 3 support most Python addons will be broken developers should start to port their addons to be compatible with both Python 2 and 3. This is not a trivial task but fortunately there are a number of tools and recommendations to simplify this process. Here I’ll give you some advice and information about tools that will simplify creating portable code that runs on both Python 2 and 3.

=== Know Your Strings! ===

This is the most problematic part of porting Python 2 code to Python 3 so I put it first. One of the most notable differences between Python 2 and 3 is clear separation between “textual” and “binary” strings, that is, between textual data and their binary representation. This difference is often difficult to understand, especially for novice developers (not only in Python), and the fact that in Python 2 you can mix together str (binary data) and unicode (textual data) adds to that difficulty.

There are many explanations in the Internet, but here are the most basic things about strings that you need to know:

“Textual” or Unicode strings (unicode in Python 2 and str in Python 3) hold text as a sequence of characters. A minimal unit in such strings is a Unicode character — an abstract entity that represent an alphabet symbol, a punctuation sign or other symbol included in the Unicode character table. Of course, such symbols have concrete internal binary representation (a computer is a binary machine after all), but this representation is not relevant to us. All you need to know is that “textual” strings contain text units without being tied to concrete binary representation.

Unlike Unicode strings, binary strings (str in Python 2 and bytes in both Python 2 and 3) hold binary data, and a minimal unit of such strings is a byte or 8 bits. Historically, in Python 2 such strings are used for textual data encoded in ASCII or other 1-byte fixed-length encoding (e.g. Windows 1251 for Cyrillic alphabets). However, this approach has its limitation, that is why unicode type was introduced in Python 2, and in Python 3 it was renamed to str and made a default container for textual data.

Unfortunately, Python 2 allows to mix and match Unicode and binary strings in the same context and this creates big problems when porting Python 2 code to Python 3. So in order to successfully port your code to Python 3 you need to carefully consider how textual data are processed in your addon. The following recommendations will help you avoid problems when writing Python 2/3 compatible code.

=== Avoid using binary strings for text ===

Store all your text as Unicode strings. This means that all binary strings received from external sources (textual files, websites, various APIs) need to be decoded using appropriate encoding (UTF-8 in most cases). If you need to use strings literals (although using hardcoded strings for user-facing text in Kodi is strongly discouraged), they need to be defined as unicode strings as well. You can use either
<source lang="python">
from __future__ import unicode_literals
</source>
at the beginning of your modules to enable unicode strings by default as in Python 3 or u-strings, e.g. <code>u'foo'</code> (Python 3 understands them too) — it doesn't really matter as long as you are doing it consistently.

=== Know the libraries you are using ===
Working with Python Standard Library and third-party libraries (including Kodi Python API) you should know that types their functions and methods expect and what types they return. For example, in popular requests library Response.text property returns a Unicode string and Response.content returns a binary string.

If a function/method/property returns a binary string, you need to decode it to a Unicode string using .decode() method. Don’t use unicode class constructor because unicode type does not exist in Python 3.

If you are reading textual files from disk, it is better to use io.open() instead of built-in open(). io.open() can decode file contents to Unicode using specified encoding and works identically both in Python 2 and 3. Example:
<source lang="python">
import io

with io.open('foo.txt', 'r', encoding='utf-8') as fo:
text = fo.read()
</source>

=== Potential pitfalls ===

When writing Python 2/3-compatible code or converting existing Python 2 codebase to compatible code you should pay attention to the following Python language constructions where TypeError exceptions may happen because of incompatible string types if you don’t get your strings in order:

* String concatenations with + operator (<code>'foo' + 'bar'</code>).
* String formatting — both “old style” (<code>'foo %s' % 'bar'</code>) and “new style” (<code>'foo {}'.format('bar')</code>).
* String joining (<code>', '.join(['foo', 'bar'])</code>).

In all those cases always make sure that you work with Unicode strings only.
Another source of subtle bugs is functions and methods that accept and return binary strings (str type) in Python 2 but Unicode strings in Python 3 (again, str type but with different properties). You can use the following utility functions to “normalize” strings that are provided to such functions/methods and received from them:

<source lang="python">
import sys

__all__ = ['PY2', 'py2_encode', 'py2_decode']

PY2 = sys.version_info[0] == 2

def py2_encode(s, encoding='utf-8'):
"""
Encode Python 2 ``unicode`` to ``str``

In Python 3 the string is not changed.
"""
if PY2 and isinstance(s, unicode):
s = s.encode(encoding)
return s

def py2_decode(s, encoding='utf-8'):
"""
Decode Python 2 ``str`` to ``unicode``

In Python 3 the string is not changed.
"""
if PY2 and isinstance(s, str):
s = s.decode(encoding)
return s
</source>

== Helper Libraries and Tools ==

=== 2to3 ===

[https://docs.python.org/2/library/2to3.html 2to3] script is created by Python developers to help converting existing Python 2 code to Python 3. On Windows it is included in Python distribution (Tools subfolder) but on other OSes you man need to install it separately. For example, on Ubuntu it is included in python-examples package. Note that this script is created for conversion from Python 2 to 3, not for writing portable code, so you need to treat its output with care.

=== Modernize ===

https://python-modernize.readthedocs.io/en/latest/ Modernize] script works on top of 2to3 and is supposed to help you to convert your existing Python 2 code to Python 2/3-compatible. However, in my experiments it did not work reliably even in simple cases so I cannot recommend it for usage.

=== Six ===

[https://pythonhosted.org/six/ Six] was the first library developed to simplify Python 2 to 3 migration. It provides a set of wrappers that hide differences between Python 2 and 3 behind its API. It is less intrusive than the following alternative because it does not monkey-patch built-in names, but in order to use Six library effectively you need to learn its API. Six can be used both for writing new Python 2/3-compatible addons and converting existing ones, but it requires good knowledge of Python 2/3 differences to pick necessary Six features that address specific differences.

Six library is included in the Kodi addon repository as '''script.module.six''' addon.

=== Future ===

[http://python-future.org/ Future] library, like Six, was created to simplify porting existing Python 2 code to 3. But it uses a different approach than Six. Future monkey-patches built-in Python objects to make Python 2 behave like Python 3. The advantage of such approach is that code written using Future is close to idiomatic Python 3 and requires little re-work if you decide to drop Python 2 support in the future. However, such approach may cause problems in some rare edge-cases.

Future library also includes <code>futurize</code> command-line utility for converting existing Python 2 code to 2/3-compatible and in my experiments this utility showed good results.
Future library (without additional utilities) is included in the Kodi addon repository as script.module.future addon.

=== PyCharm ===

PyCharm, except for being a really good Python IDE, also provides code compatibility inspection that helps you write Python 2/3-portable code. Open '''Settings''' (Alt+F7) > '''Editor''' > '''Inspections''' > '''Python''' > '''Code compatibility inspections''' and check Python version that you want to support.

PyCharm Community Edition is free and provides all features you need for creating Python addons for Kodi.

== Writing Portable Code ==

Here I’ll give you some tips about how to write portable code:

* Learn the differences between Python 2 and 3. You need to know at least the most important differences between the two major Python versions.

* Use version control system — git or mercurial — to track changes in your code. If you are porting existing code, do it in a separate branch.

* No matter if you are going to write a brand new addon or to port existing addon to Python 3, carefully choose your tools. It is totally possible to write portable code without any helper tools and libraries, but you need to know what you are doing. However, in most cases I’d recommend you to use Future library and its utilities. Carefully read Future documentation.

* Put the following line at the beginning of all your modules:
<source lang="python">
from __future__ import absolute_import, division, unicode_literals
</source>
This will enable respective Python 3 features in your code. You don’t need to import print_function (another Python 3 feature) because in Kodi addons you should to use xbmc.log() to write messages to the Kodi log file.

=== Writing New Addons ===

Check the [http://python-future.org/quickstart.html#if-you-are-writing-code-from-scratch Quick Start Guide] section of Future library documentation. You can also use Six library, but, as it was said previously, you need to learn its API to pick the necessary features to address specific Python version differences, while Future allows to write your code in (almost) idiomatic Python 3.

A brief procedure for writing new Python 2/3-compatible addons with Future library:

# Create a new virtual environment with Python 3 interpreter and activate it.
# Install Future library: <code>pip install future</code>.
# Point your IDE (Integrated Development Environment) to that environment. For example, in PyCharm: '''File''' > '''Setting''' > '''Project''' > '''Project Interpreter'''.
# Put the following line at the beginning of your Python code:
<source lang="python">
from __future__ import absolute_import, division, unicode_literals
from future import standard_library
from future.builtins import *
standard_library.install_aliases()
</source>
Write your addon using Python 3 syntax and standard library names. However, you may still need to use wrappers from Future library for some specific cases (e.g. iterators over <code>dict</code> elements). Read Future docs for more info.

=== Porting Existing Addons ===

A brief procedure for porting existing code to Python 3-compatible with Future library:

# Install Future library into your working Python 2 virtual environment: <code>pip install future</code>.
# Run <code>futurize</code> utility to convert all your scripts and modules to portable code.

Test your new or converted addon in Kodi with Python 2 interpreter and fix all found issues. After that test the addon in Kodi with Python 3 interpreter and again fix all found issues.
You can use [https://github.com/romanvm/kodi.web-pdb Web-PDB] debugger for troubleshooting issues in your code. It is compatible with both Python 2 and 3.

== Links ==

* Python 2 and 3 differences: https://docs.python.org/3.6/whatsnew/3.0.html
* Future library documentation: http://python-future.org/
* Six library documentation: https://pythonhosted.org/six/
* Kodi test builds with Python 3 for Windows: http://mirrors.xbmc.org/test-builds/windows/win32/
* Kodi test builds with Python 3 for Linux Ubuntu: https://launchpad.net/~wsnipex/+archive/ubuntu/kodi-python3/+packages
* “Python 3 migration” section on the Kodi official forum: https://forum.kodi.tv/forumdisplay.php?fid=281

General information about migration to Python 3

2018-01-19T09:58:46Z

Roman V M: /* Writing New Addons */

== Overview ==

Python 3 was released 9 years ago and EOL for Python 2.7 is scheduled for 2020. Currently more and more Python project are switching to Python 3 or 2/3 compatible code. The issue of migrating Kodi’s built-in Python interpreter to Python 3.x was brought up on the Kodi forum several times. Now, thanks to a successful GSOC 2017 project, we have a working Python 3 interpreter for Kodi. That is why on Kodi DevCon 2017 Team Kodi decided that we will switch to Python 3 in the future Kodi release (v.19 preliminarily).

Unfortunately, Python 3.x versions are not backward-compatible with Python 2.x, so we decided to urge addon developers to start porting their addons to Python 3 by making the addons compatible with both Python 2 and 3 and submitting them to the official repo. This way, when Kodi with Python 3 is released, we won’t have a situation when we have no working addons.

== The Process ==

* Starting from Kodi 18 (Leia), only addons that are compatible with both Python 2 and 3 will be accepted to the official addon repository.
* We will provide Kodi builds with Python 3 so that addon developers can tests if their addons are really compatible with Python 3.
* A thread will be created on the Kodi forum so that addon devs can get help with their migration process.

== Python 3 And 2 Differences ==

Below is a brief overview of the main differences between Python 3 and 2.

=== Unicode strings by default ===

In Python 2 built-in <code>str</code> type holds the sequence of bytes so it can be used to store both binary data and textual data in ASCII or any of 8-bit fixed-length encodings (there were plenty). Python 2 also has <code>unicode</code> type that can store text in various writing systems. A minimal unit of a <code>unicode</code> object is a Unicode codepoint (a Unicode character). Both <code>str</code> and <code>unicode</code> objects can be mixed and matched together, for example, concatenated using <code>+</code> operator.

In Python 3 <code>str</code> type holds Unicode characters, and for binary data a new <code>bytes</code> type was added. This type was also added to Python 2.7 to simplify porting to Python 3 but in Python 2.7 bytes is simply an alias for <code>str</code>. The <code>str</code> and <code>bytes</code> types are incompatible, and an attempt to use both together will result in <code>TypeError</code> exception. A <code>bytes</code> object can be converted to <code>str</code> using .decode() method, and a <code>str</code> is converted to <code>bytes</code> using .encode() method.

=== Absolute Import ===
Python 3 uses so called “absolute import” and you can no longer import modules from inside a package using only a module name. You need to either to use either a fully qualified name or a “dotted” notation.

Let’s assume that you have a foo package that contains bar and spam modules, and you need to import bar module from spam. In Python 2 you can do:
<source lang="python">
import bar
</source>

But in Python 3 you need to use either

<source lang="python">
import foo.bar
</source>

or

<source lang="python">
from . import bar
</source>

=== Floating Point Division ===

In Python 2 applying division operator to int numbers produces int result. For example:

<source lang="python">
>>> 4 / 3
1
</source>

In Python 3 division operator always gives you a float result.

<source lang="python">
>>> 4/3
1.3333333333333333
</source>

To get an int result you need to use integer division operator:

<source lang="python">
>>> 4 // 3
1
</source>

=== Print Function ===

In Python 3 print statement has been replaced with print() function. This is not very relevant to Kodi because using print in Python addons is discouraged and xbmc.log() function should be used to write messages to the Kodi log.

=== Default Iterators ===

In Python 3 most function and methods that produce sequences return iterators instead of lists. For example, range() function and .items(), .keys(), .values() of dict type produce iterators instead of lists, and their Python 2 analogues — xrange() function and .iter* methods of dict type — have been removed.

=== No long Integers ===

In Python 3 long integer type has been removed and int type can now hold a number of arbitrary length.

Those are only some of the differences between Python 2 and 3. For a more complete list see [https://docs.python.org/3.6/whatsnew/3.0.html official Python documentation].

== Some Useful Information ==

As it was said above, Python 3 includes a number of incompatible changes, so to avoid situation when after upgrading Kodi for Python 3 support most Python addons will be broken developers should start to port their addons to be compatible with both Python 2 and 3. This is not a trivial task but fortunately there are a number of tools and recommendations to simplify this process. Here I’ll give you some advice and information about tools that will simplify creating portable code that runs on both Python 2 and 3.

=== Know Your Strings! ===

This is the most problematic part of porting Python 2 code to Python 3 so I put it first. One of the most notable differences between Python 2 and 3 is clear separation between “textual” and “binary” strings, that is, between textual data and their binary representation. This difference is often difficult to understand, especially for novice developers (not only in Python), and the fact that in Python 2 you can mix together str (binary data) and unicode (textual data) adds to that difficulty.

There are many explanations in the Internet, but here are the most basic things about strings that you need to know:

“Textual” or Unicode strings (unicode in Python 2 and str in Python 3) hold text as a sequence of characters. A minimal unit in such strings is a Unicode character — an abstract entity that represent an alphabet symbol, a punctuation sign or other symbol included in the Unicode character table. Of course, such symbols have concrete internal binary representation (a computer is a binary machine after all), but this representation is not relevant to us. All you need to know is that “textual” strings contain text units without being tied to concrete binary representation.

Unlike Unicode strings, binary strings (str in Python 2 and bytes in both Python 2 and 3) hold binary data, and a minimal unit of such strings is a byte or 8 bits. Historically, in Python 2 such strings are used for textual data encoded in ASCII or other 1-byte fixed-length encoding (e.g. Windows 1251 for Cyrillic alphabets). However, this approach has its limitation, that is why unicode type was introduced in Python 2, and in Python 3 it was renamed to str and made a default container for textual data.

Unfortunately, Python 2 allows to mix and match Unicode and binary strings in the same context and this creates big problems when porting Python 2 code to Python 3. So in order to successfully port your code to Python 3 you need to carefully consider how textual data are processed in your addon. The following recommendations will help you avoid problems when writing Python 2/3 compatible code.

=== Avoid using binary strings for text ===

Store all your text as Unicode strings. This means that all binary strings received from external sources (textual files, websites, various APIs) need to be decoded using appropriate encoding (UTF-8 in most cases). If you need to use strings literals (although using hardcoded strings for user-facing text in Kodi is strongly discouraged), they need to be defined as unicode strings as well. You can use either
<source lang="python">
from __future__ import unicode_literals
</source>
at the beginning of your modules to enable unicode strings by default as in Python 3 or u-strings, e.g. <code>u'foo'</code> (Python 3 understands them too) — it doesn't really matter as long as you are doing it consistently.

=== Know the libraries you are using ===
Working with Python Standard Library and third-party libraries (including Kodi Python API) you should know that types their functions and methods expect and what types they return. For example, in popular requests library Response.text property returns a Unicode string and Response.content returns a binary string.

If a function/method/property returns a binary string, you need to decode it to a Unicode string using .decode() method. Don’t use unicode class constructor because unicode type does not exist in Python 3.

If you are reading textual files from disk, it is better to use io.open() instead of built-in open(). io.open() can decode file contents to Unicode using specified encoding and works identically both in Python 2 and 3. Example:
<source lang="python">
import io

with io.open('foo.txt', 'r', encoding='utf-8') as fo:
text = fo.read()
</source>

=== Potential pitfalls ===

When writing Python 2/3-compatible code or converting existing Python 2 codebase to compatible code you should pay attention to the following Python language constructions where TypeError exceptions may happen because of incompatible string types if you don’t get your strings in order:

* String concatenations with + operator (<code>'foo' + 'bar'</code>).
* String formatting — both “old style” (<code>'foo %s' % 'bar'</code>) and “new style” (<code>'foo {}'.format('bar')</code>).
* String joining (<code>', '.join(['foo', 'bar'])</code>).

In all those cases always make sure that you work with Unicode strings only.
Another source of subtle bugs is functions and methods that accept and return binary strings (str type) in Python 2 but Unicode strings in Python 3 (again, str type but with different properties). You can use the following utility functions to “normalize” strings that are provided to such functions/methods and received from them:

<source lang="python">
import sys

__all__ = ['PY2', 'py2_encode', 'py2_decode']

PY2 = sys.version_info[0] == 2

def py2_encode(s, encoding='utf-8'):
"""
Encode Python 2 ``unicode`` to ``str``

In Python 3 the string is not changed.
"""
if PY2 and isinstance(s, unicode):
s = s.encode(encoding)
return s

def py2_decode(s, encoding='utf-8'):
"""
Decode Python 2 ``str`` to ``unicode``

In Python 3 the string is not changed.
"""
if PY2 and isinstance(s, str):
s = s.decode(encoding)
return s
</source>

== Helper Libraries and Tools ==

=== 2to3 ===

[https://docs.python.org/2/library/2to3.html 2to3] script is created by Python developers to help converting existing Python 2 code to Python 3. On Windows it is included in Python distribution (Tools subfolder) but on other OSes you man need to install it separately. For example, on Ubuntu it is included in python-examples package. Note that this script is created for conversion from Python 2 to 3, not for writing portable code, so you need to treat its output with care.

=== Modernize ===

https://python-modernize.readthedocs.io/en/latest/ Modernize] script works on top of 2to3 and is supposed to help you to convert your existing Python 2 code to Python 2/3-compatible. However, in my experiments it did not work reliably even in simple cases so I cannot recommend it for usage.

=== Six ===

[https://pythonhosted.org/six/ Six] was the first library developed to simplify Python 2 to 3 migration. It provides a set of wrappers that hide differences between Python 2 and 3 behind its API. It is less intrusive than the following alternative because it does not monkey-patch built-in names, but in order to use Six library effectively you need to learn its API. Six can be used both for writing new Python 2/3-compatible addons and converting existing ones, but it requires good knowledge of Python 2/3 differences to pick necessary Six features that address specific differences.

Six library is included in the Kodi addon repository as '''script.module.six addon'''.

=== Future ===

[http://python-future.org/ Future] library, like Six, was created to simplify porting existing Python 2 code to 3. But it uses a different approach than Six. Future monkey-patches built-in Python objects to make Python 2 behave like Python 3. The advantage of such approach is that code written using Future is close to idiomatic Python 3 and requires little re-work if you decide to drop Python 2 support in the future. However, such approach may cause problems in some rare edge-cases.

Future library also includes <code>futurize</code> command-line utility for converting existing Python 2 code to 2/3-compatible and in my experiments this utility showed good results.
Future library (without additional utilities) is included in the Kodi addon repository as script.module.future addon.

=== PyCharm ===

PyCharm, except for being a really good Python IDE, also provides code compatibility inspection that helps you write Python 2/3-portable code. Open '''Settings''' (Alt+F7) > '''Editor''' > '''Inspections''' > '''Python''' > '''Code compatibility inspections''' and check Python version that you want to support.

PyCharm Community Edition is free and provides all features you need for creating Python addons for Kodi.

== Writing Portable Code ==

Here I’ll give you some tips about how to write portable code:

* Learn the differences between Python 2 and 3. You need to know at least the most important differences between the two major Python versions.

* Use version control system — git or mercurial — to track changes in your code. If you are porting existing code, do it in a separate branch.

* No matter if you are going to write a brand new addon or to port existing addon to Python 3, carefully choose your tools. It is totally possible to write portable code without any helper tools and libraries, but you need to know what you are doing. However, in most cases I’d recommend you to use Future library and its utilities. Carefully read Future documentation.

* Put the following line at the beginning of all your modules:
<source lang="python">
from __future__ import absolute_import, division, unicode_literals
</source>
This will enable respective Python 3 features in your code. You don’t need to import print_function (another Python 3 feature) because in Kodi addons you should to use xbmc.log() to write messages to the Kodi log file.

=== Writing New Addons ===

Check the [http://python-future.org/quickstart.html#if-you-are-writing-code-from-scratch Quick Start Guide] section of Future library documentation. You can also use Six library, but, as it was said previously, you need to learn its API to pick the necessary features to address specific Python version differences, while Future allows to write your code in (almost) idiomatic Python 3.

A brief procedure for writing new Python 2/3-compatible addons with Future library:

# Create a new virtual environment with Python 3 interpreter and activate it.
# Install Future library: <code>pip install future</code>.
# Point your IDE (Integrated Development Environment) to that environment. For example, in PyCharm: '''File''' > '''Setting''' > '''Project''' > '''Project Interpreter'''.
# Put the following line at the beginning of your Python code:
<source lang="python">
from __future__ import absolute_import, division, unicode_literals
from future import standard_library
from future.builtins import *
standard_library.install_aliases()
</source>
Write your addon using Python 3 syntax and standard library names. However, you may still need to use wrappers from Future library for some specific cases (e.g. iterators over <code>dict</code> elements). Read Future docs for more info.

=== Porting Existing Addons ===

A brief procedure for porting existing code to Python 3-compatible with Future library:

# Install Future library into your working Python 2 virtual environment: <code>pip install future</code>.
# Run <code>futurize</code> utility to convert all your scripts and modules to portable code.

Test your new or converted addon in Kodi with Python 2 interpreter and fix all found issues. After that test the addon in Kodi with Python 3 interpreter and again fix all found issues.
You can use [https://github.com/romanvm/kodi.web-pdb Web-PDB] debugger for troubleshooting issues in your code. It is compatible with both Python 2 and 3.

== Links ==

* Python 2 and 3 differences: https://docs.python.org/3.6/whatsnew/3.0.html
* Future library documentation: http://python-future.org/
* Six library documentation: https://pythonhosted.org/six/
* Kodi test builds with Python 3 for Windows: http://mirrors.xbmc.org/test-builds/windows/win32/
* Kodi test builds with Python 3 for Linux Ubuntu: https://launchpad.net/~wsnipex/+archive/ubuntu/kodi-python3/+packages
* “Python 3 migration” section on the Kodi official forum: https://forum.kodi.tv/forumdisplay.php?fid=281

General information about migration to Python 3

2018-01-19T09:57:16Z

Roman V M: /* Writing New Addons */

== Overview ==

Python 3 was released 9 years ago and EOL for Python 2.7 is scheduled for 2020. Currently more and more Python project are switching to Python 3 or 2/3 compatible code. The issue of migrating Kodi’s built-in Python interpreter to Python 3.x was brought up on the Kodi forum several times. Now, thanks to a successful GSOC 2017 project, we have a working Python 3 interpreter for Kodi. That is why on Kodi DevCon 2017 Team Kodi decided that we will switch to Python 3 in the future Kodi release (v.19 preliminarily).

Unfortunately, Python 3.x versions are not backward-compatible with Python 2.x, so we decided to urge addon developers to start porting their addons to Python 3 by making the addons compatible with both Python 2 and 3 and submitting them to the official repo. This way, when Kodi with Python 3 is released, we won’t have a situation when we have no working addons.

== The Process ==

* Starting from Kodi 18 (Leia), only addons that are compatible with both Python 2 and 3 will be accepted to the official addon repository.
* We will provide Kodi builds with Python 3 so that addon developers can tests if their addons are really compatible with Python 3.
* A thread will be created on the Kodi forum so that addon devs can get help with their migration process.

== Python 3 And 2 Differences ==

Below is a brief overview of the main differences between Python 3 and 2.

=== Unicode strings by default ===

In Python 2 built-in <code>str</code> type holds the sequence of bytes so it can be used to store both binary data and textual data in ASCII or any of 8-bit fixed-length encodings (there were plenty). Python 2 also has <code>unicode</code> type that can store text in various writing systems. A minimal unit of a <code>unicode</code> object is a Unicode codepoint (a Unicode character). Both <code>str</code> and <code>unicode</code> objects can be mixed and matched together, for example, concatenated using <code>+</code> operator.

In Python 3 <code>str</code> type holds Unicode characters, and for binary data a new <code>bytes</code> type was added. This type was also added to Python 2.7 to simplify porting to Python 3 but in Python 2.7 bytes is simply an alias for <code>str</code>. The <code>str</code> and <code>bytes</code> types are incompatible, and an attempt to use both together will result in <code>TypeError</code> exception. A <code>bytes</code> object can be converted to <code>str</code> using .decode() method, and a <code>str</code> is converted to <code>bytes</code> using .encode() method.

=== Absolute Import ===
Python 3 uses so called “absolute import” and you can no longer import modules from inside a package using only a module name. You need to either to use either a fully qualified name or a “dotted” notation.

Let’s assume that you have a foo package that contains bar and spam modules, and you need to import bar module from spam. In Python 2 you can do:
<source lang="python">
import bar
</source>

But in Python 3 you need to use either

<source lang="python">
import foo.bar
</source>

or

<source lang="python">
from . import bar
</source>

=== Floating Point Division ===

In Python 2 applying division operator to int numbers produces int result. For example:

<source lang="python">
>>> 4 / 3
1
</source>

In Python 3 division operator always gives you a float result.

<source lang="python">
>>> 4/3
1.3333333333333333
</source>

To get an int result you need to use integer division operator:

<source lang="python">
>>> 4 // 3
1
</source>

=== Print Function ===

In Python 3 print statement has been replaced with print() function. This is not very relevant to Kodi because using print in Python addons is discouraged and xbmc.log() function should be used to write messages to the Kodi log.

=== Default Iterators ===

In Python 3 most function and methods that produce sequences return iterators instead of lists. For example, range() function and .items(), .keys(), .values() of dict type produce iterators instead of lists, and their Python 2 analogues — xrange() function and .iter* methods of dict type — have been removed.

=== No long Integers ===

In Python 3 long integer type has been removed and int type can now hold a number of arbitrary length.

Those are only some of the differences between Python 2 and 3. For a more complete list see [https://docs.python.org/3.6/whatsnew/3.0.html official Python documentation].

== Some Useful Information ==

As it was said above, Python 3 includes a number of incompatible changes, so to avoid situation when after upgrading Kodi for Python 3 support most Python addons will be broken developers should start to port their addons to be compatible with both Python 2 and 3. This is not a trivial task but fortunately there are a number of tools and recommendations to simplify this process. Here I’ll give you some advice and information about tools that will simplify creating portable code that runs on both Python 2 and 3.

=== Know Your Strings! ===

This is the most problematic part of porting Python 2 code to Python 3 so I put it first. One of the most notable differences between Python 2 and 3 is clear separation between “textual” and “binary” strings, that is, between textual data and their binary representation. This difference is often difficult to understand, especially for novice developers (not only in Python), and the fact that in Python 2 you can mix together str (binary data) and unicode (textual data) adds to that difficulty.

There are many explanations in the Internet, but here are the most basic things about strings that you need to know:

“Textual” or Unicode strings (unicode in Python 2 and str in Python 3) hold text as a sequence of characters. A minimal unit in such strings is a Unicode character — an abstract entity that represent an alphabet symbol, a punctuation sign or other symbol included in the Unicode character table. Of course, such symbols have concrete internal binary representation (a computer is a binary machine after all), but this representation is not relevant to us. All you need to know is that “textual” strings contain text units without being tied to concrete binary representation.

Unlike Unicode strings, binary strings (str in Python 2 and bytes in both Python 2 and 3) hold binary data, and a minimal unit of such strings is a byte or 8 bits. Historically, in Python 2 such strings are used for textual data encoded in ASCII or other 1-byte fixed-length encoding (e.g. Windows 1251 for Cyrillic alphabets). However, this approach has its limitation, that is why unicode type was introduced in Python 2, and in Python 3 it was renamed to str and made a default container for textual data.

Unfortunately, Python 2 allows to mix and match Unicode and binary strings in the same context and this creates big problems when porting Python 2 code to Python 3. So in order to successfully port your code to Python 3 you need to carefully consider how textual data are processed in your addon. The following recommendations will help you avoid problems when writing Python 2/3 compatible code.

=== Avoid using binary strings for text ===

Store all your text as Unicode strings. This means that all binary strings received from external sources (textual files, websites, various APIs) need to be decoded using appropriate encoding (UTF-8 in most cases). If you need to use strings literals (although using hardcoded strings for user-facing text in Kodi is strongly discouraged), they need to be defined as unicode strings as well. You can use either
<source lang="python">
from __future__ import unicode_literals
</source>
at the beginning of your modules to enable unicode strings by default as in Python 3 or u-strings, e.g. <code>u'foo'</code> (Python 3 understands them too) — it doesn't really matter as long as you are doing it consistently.

=== Know the libraries you are using ===
Working with Python Standard Library and third-party libraries (including Kodi Python API) you should know that types their functions and methods expect and what types they return. For example, in popular requests library Response.text property returns a Unicode string and Response.content returns a binary string.

If a function/method/property returns a binary string, you need to decode it to a Unicode string using .decode() method. Don’t use unicode class constructor because unicode type does not exist in Python 3.

If you are reading textual files from disk, it is better to use io.open() instead of built-in open(). io.open() can decode file contents to Unicode using specified encoding and works identically both in Python 2 and 3. Example:
<source lang="python">
import io

with io.open('foo.txt', 'r', encoding='utf-8') as fo:
text = fo.read()
</source>

=== Potential pitfalls ===

When writing Python 2/3-compatible code or converting existing Python 2 codebase to compatible code you should pay attention to the following Python language constructions where TypeError exceptions may happen because of incompatible string types if you don’t get your strings in order:

* String concatenations with + operator (<code>'foo' + 'bar'</code>).
* String formatting — both “old style” (<code>'foo %s' % 'bar'</code>) and “new style” (<code>'foo {}'.format('bar')</code>).
* String joining (<code>', '.join(['foo', 'bar'])</code>).

In all those cases always make sure that you work with Unicode strings only.
Another source of subtle bugs is functions and methods that accept and return binary strings (str type) in Python 2 but Unicode strings in Python 3 (again, str type but with different properties). You can use the following utility functions to “normalize” strings that are provided to such functions/methods and received from them:

<source lang="python">
import sys

__all__ = ['PY2', 'py2_encode', 'py2_decode']

PY2 = sys.version_info[0] == 2

def py2_encode(s, encoding='utf-8'):
"""
Encode Python 2 ``unicode`` to ``str``

In Python 3 the string is not changed.
"""
if PY2 and isinstance(s, unicode):
s = s.encode(encoding)
return s

def py2_decode(s, encoding='utf-8'):
"""
Decode Python 2 ``str`` to ``unicode``

In Python 3 the string is not changed.
"""
if PY2 and isinstance(s, str):
s = s.decode(encoding)
return s
</source>

== Helper Libraries and Tools ==

=== 2to3 ===

[https://docs.python.org/2/library/2to3.html 2to3] script is created by Python developers to help converting existing Python 2 code to Python 3. On Windows it is included in Python distribution (Tools subfolder) but on other OSes you man need to install it separately. For example, on Ubuntu it is included in python-examples package. Note that this script is created for conversion from Python 2 to 3, not for writing portable code, so you need to treat its output with care.

=== Modernize ===

https://python-modernize.readthedocs.io/en/latest/ Modernize] script works on top of 2to3 and is supposed to help you to convert your existing Python 2 code to Python 2/3-compatible. However, in my experiments it did not work reliably even in simple cases so I cannot recommend it for usage.

=== Six ===

[https://pythonhosted.org/six/ Six] was the first library developed to simplify Python 2 to 3 migration. It provides a set of wrappers that hide differences between Python 2 and 3 behind its API. It is less intrusive than the following alternative because it does not monkey-patch built-in names, but in order to use Six library effectively you need to learn its API. Six can be used both for writing new Python 2/3-compatible addons and converting existing ones, but it requires good knowledge of Python 2/3 differences to pick necessary Six features that address specific differences.

Six library is included in the Kodi addon repository as '''script.module.six addon'''.

=== Future ===

[http://python-future.org/ Future] library, like Six, was created to simplify porting existing Python 2 code to 3. But it uses a different approach than Six. Future monkey-patches built-in Python objects to make Python 2 behave like Python 3. The advantage of such approach is that code written using Future is close to idiomatic Python 3 and requires little re-work if you decide to drop Python 2 support in the future. However, such approach may cause problems in some rare edge-cases.

Future library also includes <code>futurize</code> command-line utility for converting existing Python 2 code to 2/3-compatible and in my experiments this utility showed good results.
Future library (without additional utilities) is included in the Kodi addon repository as script.module.future addon.

=== PyCharm ===

PyCharm, except for being a really good Python IDE, also provides code compatibility inspection that helps you write Python 2/3-portable code. Open '''Settings''' (Alt+F7) > '''Editor''' > '''Inspections''' > '''Python''' > '''Code compatibility inspections''' and check Python version that you want to support.

PyCharm Community Edition is free and provides all features you need for creating Python addons for Kodi.

== Writing Portable Code ==

Here I’ll give you some tips about how to write portable code:

* Learn the differences between Python 2 and 3. You need to know at least the most important differences between the two major Python versions.

* Use version control system — git or mercurial — to track changes in your code. If you are porting existing code, do it in a separate branch.

* No matter if you are going to write a brand new addon or to port existing addon to Python 3, carefully choose your tools. It is totally possible to write portable code without any helper tools and libraries, but you need to know what you are doing. However, in most cases I’d recommend you to use Future library and its utilities. Carefully read Future documentation.

* Put the following line at the beginning of all your modules:
<source lang="python">
from __future__ import absolute_import, division, unicode_literals
</source>
This will enable respective Python 3 features in your code. You don’t need to import print_function (another Python 3 feature) because in Kodi addons you should to use xbmc.log() to write messages to the Kodi log file.

=== Writing New Addons ===

Check the [http://python-future.org/quickstart.html#if-you-are-writing-code-from-scratch Quick Start Guide] section of Future library documentation. You can also use Six library, but, as it was said previously, you need to learn its API to pick the necessary features to address specific Python version differences, while Future allows to write your code in (almost) idiomatic Python 3.

A brief procedure for writing new Python 2/3-compatible addons with Future library:

# Create a new virtual environment with Python 3 interpreter and activate it.
# Install Future library: <code>pip install future</code>.
# Point your IDE (Integrated Development Environment) to that environment. For example, in PyCharm: '''File''' > '''Setting''' > '''Project''' > '''Project Interpreter'''.
# Put the following line at the beginning of your Python code:
<source lang="python">
from __future__ import absolute_import, division, unicode_literals
from future import standard_library
from future.builtins import *
standard_library.install_aliases()
</source>
Write your addon using Python 3 syntax and standard library names. However, you may still need to use wrappers from Future library for some specific cases (e.g. iterators over <code>dict</code> elements. Read Future docs for more info.

=== Porting Existing Addons ===

A brief procedure for porting existing code to Python 3-compatible with Future library:

# Install Future library into your working Python 2 virtual environment: <code>pip install future</code>.
# Run <code>futurize</code> utility to convert all your scripts and modules to portable code.

Test your new or converted addon in Kodi with Python 2 interpreter and fix all found issues. After that test the addon in Kodi with Python 3 interpreter and again fix all found issues.
You can use [https://github.com/romanvm/kodi.web-pdb Web-PDB] debugger for troubleshooting issues in your code. It is compatible with both Python 2 and 3.

== Links ==

* Python 2 and 3 differences: https://docs.python.org/3.6/whatsnew/3.0.html
* Future library documentation: http://python-future.org/
* Six library documentation: https://pythonhosted.org/six/
* Kodi test builds with Python 3 for Windows: http://mirrors.xbmc.org/test-builds/windows/win32/
* Kodi test builds with Python 3 for Linux Ubuntu: https://launchpad.net/~wsnipex/+archive/ubuntu/kodi-python3/+packages
* “Python 3 migration” section on the Kodi official forum: https://forum.kodi.tv/forumdisplay.php?fid=281

General information about migration to Python 3

2018-01-19T09:56:56Z

Roman V M: /* Writing New Addons */

== Overview ==

Python 3 was released 9 years ago and EOL for Python 2.7 is scheduled for 2020. Currently more and more Python project are switching to Python 3 or 2/3 compatible code. The issue of migrating Kodi’s built-in Python interpreter to Python 3.x was brought up on the Kodi forum several times. Now, thanks to a successful GSOC 2017 project, we have a working Python 3 interpreter for Kodi. That is why on Kodi DevCon 2017 Team Kodi decided that we will switch to Python 3 in the future Kodi release (v.19 preliminarily).

Unfortunately, Python 3.x versions are not backward-compatible with Python 2.x, so we decided to urge addon developers to start porting their addons to Python 3 by making the addons compatible with both Python 2 and 3 and submitting them to the official repo. This way, when Kodi with Python 3 is released, we won’t have a situation when we have no working addons.

== The Process ==

* Starting from Kodi 18 (Leia), only addons that are compatible with both Python 2 and 3 will be accepted to the official addon repository.
* We will provide Kodi builds with Python 3 so that addon developers can tests if their addons are really compatible with Python 3.
* A thread will be created on the Kodi forum so that addon devs can get help with their migration process.

== Python 3 And 2 Differences ==

Below is a brief overview of the main differences between Python 3 and 2.

=== Unicode strings by default ===

In Python 2 built-in <code>str</code> type holds the sequence of bytes so it can be used to store both binary data and textual data in ASCII or any of 8-bit fixed-length encodings (there were plenty). Python 2 also has <code>unicode</code> type that can store text in various writing systems. A minimal unit of a <code>unicode</code> object is a Unicode codepoint (a Unicode character). Both <code>str</code> and <code>unicode</code> objects can be mixed and matched together, for example, concatenated using <code>+</code> operator.

In Python 3 <code>str</code> type holds Unicode characters, and for binary data a new <code>bytes</code> type was added. This type was also added to Python 2.7 to simplify porting to Python 3 but in Python 2.7 bytes is simply an alias for <code>str</code>. The <code>str</code> and <code>bytes</code> types are incompatible, and an attempt to use both together will result in <code>TypeError</code> exception. A <code>bytes</code> object can be converted to <code>str</code> using .decode() method, and a <code>str</code> is converted to <code>bytes</code> using .encode() method.

=== Absolute Import ===
Python 3 uses so called “absolute import” and you can no longer import modules from inside a package using only a module name. You need to either to use either a fully qualified name or a “dotted” notation.

Let’s assume that you have a foo package that contains bar and spam modules, and you need to import bar module from spam. In Python 2 you can do:
<source lang="python">
import bar
</source>

But in Python 3 you need to use either

<source lang="python">
import foo.bar
</source>

or

<source lang="python">
from . import bar
</source>

=== Floating Point Division ===

In Python 2 applying division operator to int numbers produces int result. For example:

<source lang="python">
>>> 4 / 3
1
</source>

In Python 3 division operator always gives you a float result.

<source lang="python">
>>> 4/3
1.3333333333333333
</source>

To get an int result you need to use integer division operator:

<source lang="python">
>>> 4 // 3
1
</source>

=== Print Function ===

In Python 3 print statement has been replaced with print() function. This is not very relevant to Kodi because using print in Python addons is discouraged and xbmc.log() function should be used to write messages to the Kodi log.

=== Default Iterators ===

In Python 3 most function and methods that produce sequences return iterators instead of lists. For example, range() function and .items(), .keys(), .values() of dict type produce iterators instead of lists, and their Python 2 analogues — xrange() function and .iter* methods of dict type — have been removed.

=== No long Integers ===

In Python 3 long integer type has been removed and int type can now hold a number of arbitrary length.

Those are only some of the differences between Python 2 and 3. For a more complete list see [https://docs.python.org/3.6/whatsnew/3.0.html official Python documentation].

== Some Useful Information ==

As it was said above, Python 3 includes a number of incompatible changes, so to avoid situation when after upgrading Kodi for Python 3 support most Python addons will be broken developers should start to port their addons to be compatible with both Python 2 and 3. This is not a trivial task but fortunately there are a number of tools and recommendations to simplify this process. Here I’ll give you some advice and information about tools that will simplify creating portable code that runs on both Python 2 and 3.

=== Know Your Strings! ===

This is the most problematic part of porting Python 2 code to Python 3 so I put it first. One of the most notable differences between Python 2 and 3 is clear separation between “textual” and “binary” strings, that is, between textual data and their binary representation. This difference is often difficult to understand, especially for novice developers (not only in Python), and the fact that in Python 2 you can mix together str (binary data) and unicode (textual data) adds to that difficulty.

There are many explanations in the Internet, but here are the most basic things about strings that you need to know:

“Textual” or Unicode strings (unicode in Python 2 and str in Python 3) hold text as a sequence of characters. A minimal unit in such strings is a Unicode character — an abstract entity that represent an alphabet symbol, a punctuation sign or other symbol included in the Unicode character table. Of course, such symbols have concrete internal binary representation (a computer is a binary machine after all), but this representation is not relevant to us. All you need to know is that “textual” strings contain text units without being tied to concrete binary representation.

Unlike Unicode strings, binary strings (str in Python 2 and bytes in both Python 2 and 3) hold binary data, and a minimal unit of such strings is a byte or 8 bits. Historically, in Python 2 such strings are used for textual data encoded in ASCII or other 1-byte fixed-length encoding (e.g. Windows 1251 for Cyrillic alphabets). However, this approach has its limitation, that is why unicode type was introduced in Python 2, and in Python 3 it was renamed to str and made a default container for textual data.

Unfortunately, Python 2 allows to mix and match Unicode and binary strings in the same context and this creates big problems when porting Python 2 code to Python 3. So in order to successfully port your code to Python 3 you need to carefully consider how textual data are processed in your addon. The following recommendations will help you avoid problems when writing Python 2/3 compatible code.

=== Avoid using binary strings for text ===

Store all your text as Unicode strings. This means that all binary strings received from external sources (textual files, websites, various APIs) need to be decoded using appropriate encoding (UTF-8 in most cases). If you need to use strings literals (although using hardcoded strings for user-facing text in Kodi is strongly discouraged), they need to be defined as unicode strings as well. You can use either
<source lang="python">
from __future__ import unicode_literals
</source>
at the beginning of your modules to enable unicode strings by default as in Python 3 or u-strings, e.g. <code>u'foo'</code> (Python 3 understands them too) — it doesn't really matter as long as you are doing it consistently.

=== Know the libraries you are using ===
Working with Python Standard Library and third-party libraries (including Kodi Python API) you should know that types their functions and methods expect and what types they return. For example, in popular requests library Response.text property returns a Unicode string and Response.content returns a binary string.

If a function/method/property returns a binary string, you need to decode it to a Unicode string using .decode() method. Don’t use unicode class constructor because unicode type does not exist in Python 3.

If you are reading textual files from disk, it is better to use io.open() instead of built-in open(). io.open() can decode file contents to Unicode using specified encoding and works identically both in Python 2 and 3. Example:
<source lang="python">
import io

with io.open('foo.txt', 'r', encoding='utf-8') as fo:
text = fo.read()
</source>

=== Potential pitfalls ===

When writing Python 2/3-compatible code or converting existing Python 2 codebase to compatible code you should pay attention to the following Python language constructions where TypeError exceptions may happen because of incompatible string types if you don’t get your strings in order:

* String concatenations with + operator (<code>'foo' + 'bar'</code>).
* String formatting — both “old style” (<code>'foo %s' % 'bar'</code>) and “new style” (<code>'foo {}'.format('bar')</code>).
* String joining (<code>', '.join(['foo', 'bar'])</code>).

In all those cases always make sure that you work with Unicode strings only.
Another source of subtle bugs is functions and methods that accept and return binary strings (str type) in Python 2 but Unicode strings in Python 3 (again, str type but with different properties). You can use the following utility functions to “normalize” strings that are provided to such functions/methods and received from them:

<source lang="python">
import sys

__all__ = ['PY2', 'py2_encode', 'py2_decode']

PY2 = sys.version_info[0] == 2

def py2_encode(s, encoding='utf-8'):
"""
Encode Python 2 ``unicode`` to ``str``

In Python 3 the string is not changed.
"""
if PY2 and isinstance(s, unicode):
s = s.encode(encoding)
return s

def py2_decode(s, encoding='utf-8'):
"""
Decode Python 2 ``str`` to ``unicode``

In Python 3 the string is not changed.
"""
if PY2 and isinstance(s, str):
s = s.decode(encoding)
return s
</source>

== Helper Libraries and Tools ==

=== 2to3 ===

[https://docs.python.org/2/library/2to3.html 2to3] script is created by Python developers to help converting existing Python 2 code to Python 3. On Windows it is included in Python distribution (Tools subfolder) but on other OSes you man need to install it separately. For example, on Ubuntu it is included in python-examples package. Note that this script is created for conversion from Python 2 to 3, not for writing portable code, so you need to treat its output with care.

=== Modernize ===

https://python-modernize.readthedocs.io/en/latest/ Modernize] script works on top of 2to3 and is supposed to help you to convert your existing Python 2 code to Python 2/3-compatible. However, in my experiments it did not work reliably even in simple cases so I cannot recommend it for usage.

=== Six ===

[https://pythonhosted.org/six/ Six] was the first library developed to simplify Python 2 to 3 migration. It provides a set of wrappers that hide differences between Python 2 and 3 behind its API. It is less intrusive than the following alternative because it does not monkey-patch built-in names, but in order to use Six library effectively you need to learn its API. Six can be used both for writing new Python 2/3-compatible addons and converting existing ones, but it requires good knowledge of Python 2/3 differences to pick necessary Six features that address specific differences.

Six library is included in the Kodi addon repository as '''script.module.six addon'''.

=== Future ===

[http://python-future.org/ Future] library, like Six, was created to simplify porting existing Python 2 code to 3. But it uses a different approach than Six. Future monkey-patches built-in Python objects to make Python 2 behave like Python 3. The advantage of such approach is that code written using Future is close to idiomatic Python 3 and requires little re-work if you decide to drop Python 2 support in the future. However, such approach may cause problems in some rare edge-cases.

Future library also includes <code>futurize</code> command-line utility for converting existing Python 2 code to 2/3-compatible and in my experiments this utility showed good results.
Future library (without additional utilities) is included in the Kodi addon repository as script.module.future addon.

=== PyCharm ===

PyCharm, except for being a really good Python IDE, also provides code compatibility inspection that helps you write Python 2/3-portable code. Open '''Settings''' (Alt+F7) > '''Editor''' > '''Inspections''' > '''Python''' > '''Code compatibility inspections''' and check Python version that you want to support.

PyCharm Community Edition is free and provides all features you need for creating Python addons for Kodi.

== Writing Portable Code ==

Here I’ll give you some tips about how to write portable code:

* Learn the differences between Python 2 and 3. You need to know at least the most important differences between the two major Python versions.

* Use version control system — git or mercurial — to track changes in your code. If you are porting existing code, do it in a separate branch.

* No matter if you are going to write a brand new addon or to port existing addon to Python 3, carefully choose your tools. It is totally possible to write portable code without any helper tools and libraries, but you need to know what you are doing. However, in most cases I’d recommend you to use Future library and its utilities. Carefully read Future documentation.

* Put the following line at the beginning of all your modules:
<source lang="python">
from __future__ import absolute_import, division, unicode_literals
</source>
This will enable respective Python 3 features in your code. You don’t need to import print_function (another Python 3 feature) because in Kodi addons you should to use xbmc.log() to write messages to the Kodi log file.

=== Writing New Addons ===

Check the [http://python-future.org/quickstart.html#if-you-are-writing-code-from-scratch Quick Start Guide] section of Future library documentation. You can also use Six library, but, as it was said previously, you need to learn its API to pick the necessary features to address specific Python version differences, while Future allows to write your code in (almost) idiomatic Python 3.

A brief procedure for writing new Python 2/3-compatible addons with Future library:

# Create a new virtual environment with Python 3 interpreter and activate it.
# Install Future library: <code>pip install future</code>.
# Point your IDE (Integrated Development Environment) to that environment. For example, in PyCharm: '''File''' > '''Setting''' > '''Project''' > '''Project Interpreter'''.
# Put the following line at the beginning of your Python code:
<source lang="python">
from __future__ import absolute_import, division, unicode_literals
from future import standard_library
from future.builtins import *
standard_library.install_aliases()
</source>
Write your addon using Python 3 syntax and standard library names. However, you may still need to use wrappers from Future library for some specific cases (e.g. iterators over <code>dict</dict> elements. Read Future docs for more info.

=== Porting Existing Addons ===

A brief procedure for porting existing code to Python 3-compatible with Future library:

# Install Future library into your working Python 2 virtual environment: <code>pip install future</code>.
# Run <code>futurize</code> utility to convert all your scripts and modules to portable code.

Test your new or converted addon in Kodi with Python 2 interpreter and fix all found issues. After that test the addon in Kodi with Python 3 interpreter and again fix all found issues.
You can use [https://github.com/romanvm/kodi.web-pdb Web-PDB] debugger for troubleshooting issues in your code. It is compatible with both Python 2 and 3.

== Links ==

* Python 2 and 3 differences: https://docs.python.org/3.6/whatsnew/3.0.html
* Future library documentation: http://python-future.org/
* Six library documentation: https://pythonhosted.org/six/
* Kodi test builds with Python 3 for Windows: http://mirrors.xbmc.org/test-builds/windows/win32/
* Kodi test builds with Python 3 for Linux Ubuntu: https://launchpad.net/~wsnipex/+archive/ubuntu/kodi-python3/+packages
* “Python 3 migration” section on the Kodi official forum: https://forum.kodi.tv/forumdisplay.php?fid=281

General information about migration to Python 3

2018-01-19T09:55:38Z

Roman V M: /* Writing New Addons */

== Overview ==

Python 3 was released 9 years ago and EOL for Python 2.7 is scheduled for 2020. Currently more and more Python project are switching to Python 3 or 2/3 compatible code. The issue of migrating Kodi’s built-in Python interpreter to Python 3.x was brought up on the Kodi forum several times. Now, thanks to a successful GSOC 2017 project, we have a working Python 3 interpreter for Kodi. That is why on Kodi DevCon 2017 Team Kodi decided that we will switch to Python 3 in the future Kodi release (v.19 preliminarily).

Unfortunately, Python 3.x versions are not backward-compatible with Python 2.x, so we decided to urge addon developers to start porting their addons to Python 3 by making the addons compatible with both Python 2 and 3 and submitting them to the official repo. This way, when Kodi with Python 3 is released, we won’t have a situation when we have no working addons.

== The Process ==

* Starting from Kodi 18 (Leia), only addons that are compatible with both Python 2 and 3 will be accepted to the official addon repository.
* We will provide Kodi builds with Python 3 so that addon developers can tests if their addons are really compatible with Python 3.
* A thread will be created on the Kodi forum so that addon devs can get help with their migration process.

== Python 3 And 2 Differences ==

Below is a brief overview of the main differences between Python 3 and 2.

=== Unicode strings by default ===

In Python 2 built-in <code>str</code> type holds the sequence of bytes so it can be used to store both binary data and textual data in ASCII or any of 8-bit fixed-length encodings (there were plenty). Python 2 also has <code>unicode</code> type that can store text in various writing systems. A minimal unit of a <code>unicode</code> object is a Unicode codepoint (a Unicode character). Both <code>str</code> and <code>unicode</code> objects can be mixed and matched together, for example, concatenated using <code>+</code> operator.

In Python 3 <code>str</code> type holds Unicode characters, and for binary data a new <code>bytes</code> type was added. This type was also added to Python 2.7 to simplify porting to Python 3 but in Python 2.7 bytes is simply an alias for <code>str</code>. The <code>str</code> and <code>bytes</code> types are incompatible, and an attempt to use both together will result in <code>TypeError</code> exception. A <code>bytes</code> object can be converted to <code>str</code> using .decode() method, and a <code>str</code> is converted to <code>bytes</code> using .encode() method.

=== Absolute Import ===
Python 3 uses so called “absolute import” and you can no longer import modules from inside a package using only a module name. You need to either to use either a fully qualified name or a “dotted” notation.

Let’s assume that you have a foo package that contains bar and spam modules, and you need to import bar module from spam. In Python 2 you can do:
<source lang="python">
import bar
</source>

But in Python 3 you need to use either

<source lang="python">
import foo.bar
</source>

or

<source lang="python">
from . import bar
</source>

=== Floating Point Division ===

In Python 2 applying division operator to int numbers produces int result. For example:

<source lang="python">
>>> 4 / 3
1
</source>

In Python 3 division operator always gives you a float result.

<source lang="python">
>>> 4/3
1.3333333333333333
</source>

To get an int result you need to use integer division operator:

<source lang="python">
>>> 4 // 3
1
</source>

=== Print Function ===

In Python 3 print statement has been replaced with print() function. This is not very relevant to Kodi because using print in Python addons is discouraged and xbmc.log() function should be used to write messages to the Kodi log.

=== Default Iterators ===

In Python 3 most function and methods that produce sequences return iterators instead of lists. For example, range() function and .items(), .keys(), .values() of dict type produce iterators instead of lists, and their Python 2 analogues — xrange() function and .iter* methods of dict type — have been removed.

=== No long Integers ===

In Python 3 long integer type has been removed and int type can now hold a number of arbitrary length.

Those are only some of the differences between Python 2 and 3. For a more complete list see [https://docs.python.org/3.6/whatsnew/3.0.html official Python documentation].

== Some Useful Information ==

As it was said above, Python 3 includes a number of incompatible changes, so to avoid situation when after upgrading Kodi for Python 3 support most Python addons will be broken developers should start to port their addons to be compatible with both Python 2 and 3. This is not a trivial task but fortunately there are a number of tools and recommendations to simplify this process. Here I’ll give you some advice and information about tools that will simplify creating portable code that runs on both Python 2 and 3.

=== Know Your Strings! ===

This is the most problematic part of porting Python 2 code to Python 3 so I put it first. One of the most notable differences between Python 2 and 3 is clear separation between “textual” and “binary” strings, that is, between textual data and their binary representation. This difference is often difficult to understand, especially for novice developers (not only in Python), and the fact that in Python 2 you can mix together str (binary data) and unicode (textual data) adds to that difficulty.

There are many explanations in the Internet, but here are the most basic things about strings that you need to know:

“Textual” or Unicode strings (unicode in Python 2 and str in Python 3) hold text as a sequence of characters. A minimal unit in such strings is a Unicode character — an abstract entity that represent an alphabet symbol, a punctuation sign or other symbol included in the Unicode character table. Of course, such symbols have concrete internal binary representation (a computer is a binary machine after all), but this representation is not relevant to us. All you need to know is that “textual” strings contain text units without being tied to concrete binary representation.

Unlike Unicode strings, binary strings (str in Python 2 and bytes in both Python 2 and 3) hold binary data, and a minimal unit of such strings is a byte or 8 bits. Historically, in Python 2 such strings are used for textual data encoded in ASCII or other 1-byte fixed-length encoding (e.g. Windows 1251 for Cyrillic alphabets). However, this approach has its limitation, that is why unicode type was introduced in Python 2, and in Python 3 it was renamed to str and made a default container for textual data.

Unfortunately, Python 2 allows to mix and match Unicode and binary strings in the same context and this creates big problems when porting Python 2 code to Python 3. So in order to successfully port your code to Python 3 you need to carefully consider how textual data are processed in your addon. The following recommendations will help you avoid problems when writing Python 2/3 compatible code.

=== Avoid using binary strings for text ===

Store all your text as Unicode strings. This means that all binary strings received from external sources (textual files, websites, various APIs) need to be decoded using appropriate encoding (UTF-8 in most cases). If you need to use strings literals (although using hardcoded strings for user-facing text in Kodi is strongly discouraged), they need to be defined as unicode strings as well. You can use either
<source lang="python">
from __future__ import unicode_literals
</source>
at the beginning of your modules to enable unicode strings by default as in Python 3 or u-strings, e.g. <code>u'foo'</code> (Python 3 understands them too) — it doesn't really matter as long as you are doing it consistently.

=== Know the libraries you are using ===
Working with Python Standard Library and third-party libraries (including Kodi Python API) you should know that types their functions and methods expect and what types they return. For example, in popular requests library Response.text property returns a Unicode string and Response.content returns a binary string.

If a function/method/property returns a binary string, you need to decode it to a Unicode string using .decode() method. Don’t use unicode class constructor because unicode type does not exist in Python 3.

If you are reading textual files from disk, it is better to use io.open() instead of built-in open(). io.open() can decode file contents to Unicode using specified encoding and works identically both in Python 2 and 3. Example:
<source lang="python">
import io

with io.open('foo.txt', 'r', encoding='utf-8') as fo:
text = fo.read()
</source>

=== Potential pitfalls ===

When writing Python 2/3-compatible code or converting existing Python 2 codebase to compatible code you should pay attention to the following Python language constructions where TypeError exceptions may happen because of incompatible string types if you don’t get your strings in order:

* String concatenations with + operator (<code>'foo' + 'bar'</code>).
* String formatting — both “old style” (<code>'foo %s' % 'bar'</code>) and “new style” (<code>'foo {}'.format('bar')</code>).
* String joining (<code>', '.join(['foo', 'bar'])</code>).

In all those cases always make sure that you work with Unicode strings only.
Another source of subtle bugs is functions and methods that accept and return binary strings (str type) in Python 2 but Unicode strings in Python 3 (again, str type but with different properties). You can use the following utility functions to “normalize” strings that are provided to such functions/methods and received from them:

<source lang="python">
import sys

__all__ = ['PY2', 'py2_encode', 'py2_decode']

PY2 = sys.version_info[0] == 2

def py2_encode(s, encoding='utf-8'):
"""
Encode Python 2 ``unicode`` to ``str``

In Python 3 the string is not changed.
"""
if PY2 and isinstance(s, unicode):
s = s.encode(encoding)
return s

def py2_decode(s, encoding='utf-8'):
"""
Decode Python 2 ``str`` to ``unicode``

In Python 3 the string is not changed.
"""
if PY2 and isinstance(s, str):
s = s.decode(encoding)
return s
</source>

== Helper Libraries and Tools ==

=== 2to3 ===

[https://docs.python.org/2/library/2to3.html 2to3] script is created by Python developers to help converting existing Python 2 code to Python 3. On Windows it is included in Python distribution (Tools subfolder) but on other OSes you man need to install it separately. For example, on Ubuntu it is included in python-examples package. Note that this script is created for conversion from Python 2 to 3, not for writing portable code, so you need to treat its output with care.

=== Modernize ===

https://python-modernize.readthedocs.io/en/latest/ Modernize] script works on top of 2to3 and is supposed to help you to convert your existing Python 2 code to Python 2/3-compatible. However, in my experiments it did not work reliably even in simple cases so I cannot recommend it for usage.

=== Six ===

[https://pythonhosted.org/six/ Six] was the first library developed to simplify Python 2 to 3 migration. It provides a set of wrappers that hide differences between Python 2 and 3 behind its API. It is less intrusive than the following alternative because it does not monkey-patch built-in names, but in order to use Six library effectively you need to learn its API. Six can be used both for writing new Python 2/3-compatible addons and converting existing ones, but it requires good knowledge of Python 2/3 differences to pick necessary Six features that address specific differences.

Six library is included in the Kodi addon repository as '''script.module.six addon'''.

=== Future ===

[http://python-future.org/ Future] library, like Six, was created to simplify porting existing Python 2 code to 3. But it uses a different approach than Six. Future monkey-patches built-in Python objects to make Python 2 behave like Python 3. The advantage of such approach is that code written using Future is close to idiomatic Python 3 and requires little re-work if you decide to drop Python 2 support in the future. However, such approach may cause problems in some rare edge-cases.

Future library also includes <code>futurize</code> command-line utility for converting existing Python 2 code to 2/3-compatible and in my experiments this utility showed good results.
Future library (without additional utilities) is included in the Kodi addon repository as script.module.future addon.

=== PyCharm ===

PyCharm, except for being a really good Python IDE, also provides code compatibility inspection that helps you write Python 2/3-portable code. Open '''Settings''' (Alt+F7) > '''Editor''' > '''Inspections''' > '''Python''' > '''Code compatibility inspections''' and check Python version that you want to support.

PyCharm Community Edition is free and provides all features you need for creating Python addons for Kodi.

== Writing Portable Code ==

Here I’ll give you some tips about how to write portable code:

* Learn the differences between Python 2 and 3. You need to know at least the most important differences between the two major Python versions.

* Use version control system — git or mercurial — to track changes in your code. If you are porting existing code, do it in a separate branch.

* No matter if you are going to write a brand new addon or to port existing addon to Python 3, carefully choose your tools. It is totally possible to write portable code without any helper tools and libraries, but you need to know what you are doing. However, in most cases I’d recommend you to use Future library and its utilities. Carefully read Future documentation.

* Put the following line at the beginning of all your modules:
<source lang="python">
from __future__ import absolute_import, division, unicode_literals
</source>
This will enable respective Python 3 features in your code. You don’t need to import print_function (another Python 3 feature) because in Kodi addons you should to use xbmc.log() to write messages to the Kodi log file.

=== Writing New Addons ===

Check the [http://python-future.org/quickstart.html#if-you-are-writing-code-from-scratch Quick Start Guide] section of Future library documentation. You can also use Six library, but, as it was said previously, you need to learn its API to pick the necessary features to address specific Python version differences, while Future allows to write your code in (almost) idiomatic Python 3.

A brief procedure for writing new Python 2/3-compatible addons with Future library:

# Create a new virtual environment with Python 3 interpreter and activate it.
# Install Future library: <code>pip install future</code>.
# Point your IDE (Integrated Development Environment) to that environment. For example, in PyCharm: '''File''' > '''Setting''' > '''Project''' > '''Project Interpreter'''.
# Put the following line at the beginning of your Python code:
<source lang="python">
from __future__ import absolute_import, division, unicode_literals
from future import standard_library
from future.builtins import *
standard_library.install_aliases()
</source>
Write your addon using Python 3 syntax and standard library names. However, you may still need to use wrappers from Future library for some specific cases. Read Future docs for more info.

=== Porting Existing Addons ===

A brief procedure for porting existing code to Python 3-compatible with Future library:

# Install Future library into your working Python 2 virtual environment: <code>pip install future</code>.
# Run <code>futurize</code> utility to convert all your scripts and modules to portable code.

Test your new or converted addon in Kodi with Python 2 interpreter and fix all found issues. After that test the addon in Kodi with Python 3 interpreter and again fix all found issues.
You can use [https://github.com/romanvm/kodi.web-pdb Web-PDB] debugger for troubleshooting issues in your code. It is compatible with both Python 2 and 3.

== Links ==

* Python 2 and 3 differences: https://docs.python.org/3.6/whatsnew/3.0.html
* Future library documentation: http://python-future.org/
* Six library documentation: https://pythonhosted.org/six/
* Kodi test builds with Python 3 for Windows: http://mirrors.xbmc.org/test-builds/windows/win32/
* Kodi test builds with Python 3 for Linux Ubuntu: https://launchpad.net/~wsnipex/+archive/ubuntu/kodi-python3/+packages
* “Python 3 migration” section on the Kodi official forum: https://forum.kodi.tv/forumdisplay.php?fid=281

General information about migration to Python 3

2018-01-19T09:55:03Z

Roman V M: /* Writing New Addons */

== Overview ==

Python 3 was released 9 years ago and EOL for Python 2.7 is scheduled for 2020. Currently more and more Python project are switching to Python 3 or 2/3 compatible code. The issue of migrating Kodi’s built-in Python interpreter to Python 3.x was brought up on the Kodi forum several times. Now, thanks to a successful GSOC 2017 project, we have a working Python 3 interpreter for Kodi. That is why on Kodi DevCon 2017 Team Kodi decided that we will switch to Python 3 in the future Kodi release (v.19 preliminarily).

Unfortunately, Python 3.x versions are not backward-compatible with Python 2.x, so we decided to urge addon developers to start porting their addons to Python 3 by making the addons compatible with both Python 2 and 3 and submitting them to the official repo. This way, when Kodi with Python 3 is released, we won’t have a situation when we have no working addons.

== The Process ==

* Starting from Kodi 18 (Leia), only addons that are compatible with both Python 2 and 3 will be accepted to the official addon repository.
* We will provide Kodi builds with Python 3 so that addon developers can tests if their addons are really compatible with Python 3.
* A thread will be created on the Kodi forum so that addon devs can get help with their migration process.

== Python 3 And 2 Differences ==

Below is a brief overview of the main differences between Python 3 and 2.

=== Unicode strings by default ===

In Python 2 built-in <code>str</code> type holds the sequence of bytes so it can be used to store both binary data and textual data in ASCII or any of 8-bit fixed-length encodings (there were plenty). Python 2 also has <code>unicode</code> type that can store text in various writing systems. A minimal unit of a <code>unicode</code> object is a Unicode codepoint (a Unicode character). Both <code>str</code> and <code>unicode</code> objects can be mixed and matched together, for example, concatenated using <code>+</code> operator.

In Python 3 <code>str</code> type holds Unicode characters, and for binary data a new <code>bytes</code> type was added. This type was also added to Python 2.7 to simplify porting to Python 3 but in Python 2.7 bytes is simply an alias for <code>str</code>. The <code>str</code> and <code>bytes</code> types are incompatible, and an attempt to use both together will result in <code>TypeError</code> exception. A <code>bytes</code> object can be converted to <code>str</code> using .decode() method, and a <code>str</code> is converted to <code>bytes</code> using .encode() method.

=== Absolute Import ===
Python 3 uses so called “absolute import” and you can no longer import modules from inside a package using only a module name. You need to either to use either a fully qualified name or a “dotted” notation.

Let’s assume that you have a foo package that contains bar and spam modules, and you need to import bar module from spam. In Python 2 you can do:
<source lang="python">
import bar
</source>

But in Python 3 you need to use either

<source lang="python">
import foo.bar
</source>

or

<source lang="python">
from . import bar
</source>

=== Floating Point Division ===

In Python 2 applying division operator to int numbers produces int result. For example:

<source lang="python">
>>> 4 / 3
1
</source>

In Python 3 division operator always gives you a float result.

<source lang="python">
>>> 4/3
1.3333333333333333
</source>

To get an int result you need to use integer division operator:

<source lang="python">
>>> 4 // 3
1
</source>

=== Print Function ===

In Python 3 print statement has been replaced with print() function. This is not very relevant to Kodi because using print in Python addons is discouraged and xbmc.log() function should be used to write messages to the Kodi log.

=== Default Iterators ===

In Python 3 most function and methods that produce sequences return iterators instead of lists. For example, range() function and .items(), .keys(), .values() of dict type produce iterators instead of lists, and their Python 2 analogues — xrange() function and .iter* methods of dict type — have been removed.

=== No long Integers ===

In Python 3 long integer type has been removed and int type can now hold a number of arbitrary length.

Those are only some of the differences between Python 2 and 3. For a more complete list see [https://docs.python.org/3.6/whatsnew/3.0.html official Python documentation].

== Some Useful Information ==

As it was said above, Python 3 includes a number of incompatible changes, so to avoid situation when after upgrading Kodi for Python 3 support most Python addons will be broken developers should start to port their addons to be compatible with both Python 2 and 3. This is not a trivial task but fortunately there are a number of tools and recommendations to simplify this process. Here I’ll give you some advice and information about tools that will simplify creating portable code that runs on both Python 2 and 3.

=== Know Your Strings! ===

This is the most problematic part of porting Python 2 code to Python 3 so I put it first. One of the most notable differences between Python 2 and 3 is clear separation between “textual” and “binary” strings, that is, between textual data and their binary representation. This difference is often difficult to understand, especially for novice developers (not only in Python), and the fact that in Python 2 you can mix together str (binary data) and unicode (textual data) adds to that difficulty.

There are many explanations in the Internet, but here are the most basic things about strings that you need to know:

“Textual” or Unicode strings (unicode in Python 2 and str in Python 3) hold text as a sequence of characters. A minimal unit in such strings is a Unicode character — an abstract entity that represent an alphabet symbol, a punctuation sign or other symbol included in the Unicode character table. Of course, such symbols have concrete internal binary representation (a computer is a binary machine after all), but this representation is not relevant to us. All you need to know is that “textual” strings contain text units without being tied to concrete binary representation.

Unlike Unicode strings, binary strings (str in Python 2 and bytes in both Python 2 and 3) hold binary data, and a minimal unit of such strings is a byte or 8 bits. Historically, in Python 2 such strings are used for textual data encoded in ASCII or other 1-byte fixed-length encoding (e.g. Windows 1251 for Cyrillic alphabets). However, this approach has its limitation, that is why unicode type was introduced in Python 2, and in Python 3 it was renamed to str and made a default container for textual data.

Unfortunately, Python 2 allows to mix and match Unicode and binary strings in the same context and this creates big problems when porting Python 2 code to Python 3. So in order to successfully port your code to Python 3 you need to carefully consider how textual data are processed in your addon. The following recommendations will help you avoid problems when writing Python 2/3 compatible code.

=== Avoid using binary strings for text ===

Store all your text as Unicode strings. This means that all binary strings received from external sources (textual files, websites, various APIs) need to be decoded using appropriate encoding (UTF-8 in most cases). If you need to use strings literals (although using hardcoded strings for user-facing text in Kodi is strongly discouraged), they need to be defined as unicode strings as well. You can use either
<source lang="python">
from __future__ import unicode_literals
</source>
at the beginning of your modules to enable unicode strings by default as in Python 3 or u-strings, e.g. <code>u'foo'</code> (Python 3 understands them too) — it doesn't really matter as long as you are doing it consistently.

=== Know the libraries you are using ===
Working with Python Standard Library and third-party libraries (including Kodi Python API) you should know that types their functions and methods expect and what types they return. For example, in popular requests library Response.text property returns a Unicode string and Response.content returns a binary string.

If a function/method/property returns a binary string, you need to decode it to a Unicode string using .decode() method. Don’t use unicode class constructor because unicode type does not exist in Python 3.

If you are reading textual files from disk, it is better to use io.open() instead of built-in open(). io.open() can decode file contents to Unicode using specified encoding and works identically both in Python 2 and 3. Example:
<source lang="python">
import io

with io.open('foo.txt', 'r', encoding='utf-8') as fo:
text = fo.read()
</source>

=== Potential pitfalls ===

When writing Python 2/3-compatible code or converting existing Python 2 codebase to compatible code you should pay attention to the following Python language constructions where TypeError exceptions may happen because of incompatible string types if you don’t get your strings in order:

* String concatenations with + operator (<code>'foo' + 'bar'</code>).
* String formatting — both “old style” (<code>'foo %s' % 'bar'</code>) and “new style” (<code>'foo {}'.format('bar')</code>).
* String joining (<code>', '.join(['foo', 'bar'])</code>).

In all those cases always make sure that you work with Unicode strings only.
Another source of subtle bugs is functions and methods that accept and return binary strings (str type) in Python 2 but Unicode strings in Python 3 (again, str type but with different properties). You can use the following utility functions to “normalize” strings that are provided to such functions/methods and received from them:

<source lang="python">
import sys

__all__ = ['PY2', 'py2_encode', 'py2_decode']

PY2 = sys.version_info[0] == 2

def py2_encode(s, encoding='utf-8'):
"""
Encode Python 2 ``unicode`` to ``str``

In Python 3 the string is not changed.
"""
if PY2 and isinstance(s, unicode):
s = s.encode(encoding)
return s

def py2_decode(s, encoding='utf-8'):
"""
Decode Python 2 ``str`` to ``unicode``

In Python 3 the string is not changed.
"""
if PY2 and isinstance(s, str):
s = s.decode(encoding)
return s
</source>

== Helper Libraries and Tools ==

=== 2to3 ===

[https://docs.python.org/2/library/2to3.html 2to3] script is created by Python developers to help converting existing Python 2 code to Python 3. On Windows it is included in Python distribution (Tools subfolder) but on other OSes you man need to install it separately. For example, on Ubuntu it is included in python-examples package. Note that this script is created for conversion from Python 2 to 3, not for writing portable code, so you need to treat its output with care.

=== Modernize ===

https://python-modernize.readthedocs.io/en/latest/ Modernize] script works on top of 2to3 and is supposed to help you to convert your existing Python 2 code to Python 2/3-compatible. However, in my experiments it did not work reliably even in simple cases so I cannot recommend it for usage.

=== Six ===

[https://pythonhosted.org/six/ Six] was the first library developed to simplify Python 2 to 3 migration. It provides a set of wrappers that hide differences between Python 2 and 3 behind its API. It is less intrusive than the following alternative because it does not monkey-patch built-in names, but in order to use Six library effectively you need to learn its API. Six can be used both for writing new Python 2/3-compatible addons and converting existing ones, but it requires good knowledge of Python 2/3 differences to pick necessary Six features that address specific differences.

Six library is included in the Kodi addon repository as '''script.module.six addon'''.

=== Future ===

[http://python-future.org/ Future] library, like Six, was created to simplify porting existing Python 2 code to 3. But it uses a different approach than Six. Future monkey-patches built-in Python objects to make Python 2 behave like Python 3. The advantage of such approach is that code written using Future is close to idiomatic Python 3 and requires little re-work if you decide to drop Python 2 support in the future. However, such approach may cause problems in some rare edge-cases.

Future library also includes <code>futurize</code> command-line utility for converting existing Python 2 code to 2/3-compatible and in my experiments this utility showed good results.
Future library (without additional utilities) is included in the Kodi addon repository as script.module.future addon.

=== PyCharm ===

PyCharm, except for being a really good Python IDE, also provides code compatibility inspection that helps you write Python 2/3-portable code. Open '''Settings''' (Alt+F7) > '''Editor''' > '''Inspections''' > '''Python''' > '''Code compatibility inspections''' and check Python version that you want to support.

PyCharm Community Edition is free and provides all features you need for creating Python addons for Kodi.

== Writing Portable Code ==

Here I’ll give you some tips about how to write portable code:

* Learn the differences between Python 2 and 3. You need to know at least the most important differences between the two major Python versions.

* Use version control system — git or mercurial — to track changes in your code. If you are porting existing code, do it in a separate branch.

* No matter if you are going to write a brand new addon or to port existing addon to Python 3, carefully choose your tools. It is totally possible to write portable code without any helper tools and libraries, but you need to know what you are doing. However, in most cases I’d recommend you to use Future library and its utilities. Carefully read Future documentation.

* Put the following line at the beginning of all your modules:
<source lang="python">
from __future__ import absolute_import, division, unicode_literals
</source>
This will enable respective Python 3 features in your code. You don’t need to import print_function (another Python 3 feature) because in Kodi addons you should to use xbmc.log() to write messages to the Kodi log file.

=== Writing New Addons ===

Check the [http://python-future.org/quickstart.html#if-you-are-writing-code-from-scratch Quick Start Guide] section of Future library documentation. You can also use Six library, but, as it was said previously, you need to learn its API to pick the necessary features to address specific Python version differences, while Future allows to write your code in (almost) idiomatic Python 3.

A brief procedure for writing new Python 2/3-compatible addons with Future library:

# Create a new virtual environment with Python 3 interpreter and activate it.
# Install Future library: <code>pip install future</code>.
# Point your IDE (Integrated Development Environment) to that environment. For example, in PyCharm: '''File''' > '''Setting''' > '''Project''' > '''Project Interpreter'''.
# Put the following line at the beginning of your Python code:
<source lang="python">
from __future__ import absolute_import, division, unicode_literals
from future.builtins import *
from future import standard_library
standard_library.install_aliases()
</source>
Write your addon using Python 3 syntax and standard library names. However, you may still need to use wrappers from Future library for some specific cases. Read Future docs for more info.

=== Porting Existing Addons ===

A brief procedure for porting existing code to Python 3-compatible with Future library:

# Install Future library into your working Python 2 virtual environment: <code>pip install future</code>.
# Run <code>futurize</code> utility to convert all your scripts and modules to portable code.

Test your new or converted addon in Kodi with Python 2 interpreter and fix all found issues. After that test the addon in Kodi with Python 3 interpreter and again fix all found issues.
You can use [https://github.com/romanvm/kodi.web-pdb Web-PDB] debugger for troubleshooting issues in your code. It is compatible with both Python 2 and 3.

== Links ==

* Python 2 and 3 differences: https://docs.python.org/3.6/whatsnew/3.0.html
* Future library documentation: http://python-future.org/
* Six library documentation: https://pythonhosted.org/six/
* Kodi test builds with Python 3 for Windows: http://mirrors.xbmc.org/test-builds/windows/win32/
* Kodi test builds with Python 3 for Linux Ubuntu: https://launchpad.net/~wsnipex/+archive/ubuntu/kodi-python3/+packages
* “Python 3 migration” section on the Kodi official forum: https://forum.kodi.tv/forumdisplay.php?fid=281

General information about migration to Python 3

2018-01-09T16:47:43Z

Roman V M: /* Writing New Addons */

== Overview ==

Python 3 was released 9 years ago and EOL for Python 2.7 is scheduled for 2020. Currently more and more Python project are switching to Python 3 or 2/3 compatible code. The issue of migrating Kodi’s built-in Python interpreter to Python 3.x was brought up on the Kodi forum several times. Now, thanks to a successful GSOC 2017 project, we have a working Python 3 interpreter for Kodi. That is why on Kodi DevCon 2017 Team Kodi decided that we will switch to Python 3 in the future Kodi release (v.19 preliminarily).

Unfortunately, Python 3.x versions are not backward-compatible with Python 2.x, so we decided to urge addon developers to start porting their addons to Python 3 by making the addons compatible with both Python 2 and 3 and submitting them to the official repo. This way, when Kodi with Python 3 is released, we won’t have a situation when we have no working addons.

== The Process ==

* Starting from Kodi 18 (Leia), only addons that are compatible with both Python 2 and 3 will be accepted to the official addon repository.
* We will provide Kodi builds with Python 3 so that addon developers can tests if their addons are really compatible with Python 3.
* A thread will be created on the Kodi forum so that addon devs can get help with their migration process.

== Python 3 And 2 Differences ==

Below is a brief overview of the main differences between Python 3 and 2.

=== Unicode strings by default ===

In Python 2 built-in <code>str</code> type holds the sequence of bytes so it can be used to store both binary data and textual data in ASCII or any of 8-bit fixed-length encodings (there were plenty). Python 2 also has <code>unicode</code> type that can store text in various writing systems. A minimal unit of a <code>unicode</code> object is a Unicode codepoint (a Unicode character). Both <code>str</code> and <code>unicode</code> objects can be mixed and matched together, for example, concatenated using <code>+</code> operator.

In Python 3 <code>str</code> type holds Unicode characters, and for binary data a new <code>bytes</code> type was added. This type was also added to Python 2.7 to simplify porting to Python 3 but in Python 2.7 bytes is simply an alias for <code>str</code>. The <code>str</code> and <code>bytes</code> types are incompatible, and an attempt to use both together will result in <code>TypeError</code> exception. A <code>bytes</code> object can be converted to <code>str</code> using .decode() method, and a <code>str</code> is converted to <code>bytes</code> using .encode() method.

=== Absolute Import ===
Python 3 uses so called “absolute import” and you can no longer import modules from inside a package using only a module name. You need to either to use either a fully qualified name or a “dotted” notation.

Let’s assume that you have a foo package that contains bar and spam modules, and you need to import bar module from spam. In Python 2 you can do:
<source lang="python">
import bar
</source>

But in Python 3 you need to use either

<source lang="python">
import foo.bar
</source>

or

<source lang="python">
from . import bar
</source>

=== Floating Point Division ===

In Python 2 applying division operator to int numbers produces int result. For example:

<source lang="python">
>>> 4 / 3
1
</source>

In Python 3 division operator always gives you a float result.

<source lang="python">
>>> 4/3
1.3333333333333333
</source>

To get an int result you need to use integer division operator:

<source lang="python">
>>> 4 // 3
1
</source>

=== Print Function ===

In Python 3 print statement has been replaced with print() function. This is not very relevant to Kodi because using print in Python addons is discouraged and xbmc.log() function should be used to write messages to the Kodi log.

=== Default Iterators ===

In Python 3 most function and methods that produce sequences return iterators instead of lists. For example, range() function and .items(), .keys(), .values() of dict type produce iterators instead of lists, and their Python 2 analogues — xrange() function and .iter* methods of dict type — have been removed.

=== No long Integers ===

In Python 3 long integer type has been removed and int type can now hold a number of arbitrary length.

Those are only some of the differences between Python 2 and 3. For a more complete list see [https://docs.python.org/3.6/whatsnew/3.0.html official Python documentation].

== Some Useful Information ==

As it was said above, Python 3 includes a number of incompatible changes, so to avoid situation when after upgrading Kodi for Python 3 support most Python addons will be broken developers should start to port their addons to be compatible with both Python 2 and 3. This is not a trivial task but fortunately there are a number of tools and recommendations to simplify this process. Here I’ll give you some advice and information about tools that will simplify creating portable code that runs on both Python 2 and 3.

=== Know Your Strings! ===

This is the most problematic part of porting Python 2 code to Python 3 so I put it first. One of the most notable differences between Python 2 and 3 is clear separation between “textual” and “binary” strings, that is, between textual data and their binary representation. This difference is often difficult to understand, especially for novice developers (not only in Python), and the fact that in Python 2 you can mix together str (binary data) and unicode (textual data) adds to that difficulty.

There are many explanations in the Internet, but here are the most basic things about strings that you need to know:

“Textual” or Unicode strings (unicode in Python 2 and str in Python 3) hold text as a sequence of characters. A minimal unit in such strings is a Unicode character — an abstract entity that represent an alphabet symbol, a punctuation sign or other symbol included in the Unicode character table. Of course, such symbols have concrete internal binary representation (a computer is a binary machine after all), but this representation is not relevant to us. All you need to know is that “textual” strings contain text units without being tied to concrete binary representation.

Unlike Unicode strings, binary strings (str in Python 2 and bytes in both Python 2 and 3) hold binary data, and a minimal unit of such strings is a byte or 8 bits. Historically, in Python 2 such strings are used for textual data encoded in ASCII or other 1-byte fixed-length encoding (e.g. Windows 1251 for Cyrillic alphabets). However, this approach has its limitation, that is why unicode type was introduced in Python 2, and in Python 3 it was renamed to str and made a default container for textual data.

Unfortunately, Python 2 allows to mix and match Unicode and binary strings in the same context and this creates big problems when porting Python 2 code to Python 3. So in order to successfully port your code to Python 3 you need to carefully consider how textual data are processed in your addon. The following recommendations will help you avoid problems when writing Python 2/3 compatible code.

=== Avoid using binary strings for text ===

Store all your text as Unicode strings. This means that all binary strings received from external sources (textual files, websites, various APIs) need to be decoded using appropriate encoding (UTF-8 in most cases). If you need to use strings literals (although using hardcoded strings for user-facing text in Kodi is strongly discouraged), they need to be defined as unicode strings as well. You can use either
<source lang="python">
from __future__ import unicode_literals
</source>
at the beginning of your modules to enable unicode strings by default as in Python 3 or u-strings, e.g. <code>u'foo'</code> (Python 3 understands them too) — it doesn't really matter as long as you are doing it consistently.

=== Know the libraries you are using ===
Working with Python Standard Library and third-party libraries (including Kodi Python API) you should know that types their functions and methods expect and what types they return. For example, in popular requests library Response.text property returns a Unicode string and Response.content returns a binary string.

If a function/method/property returns a binary string, you need to decode it to a Unicode string using .decode() method. Don’t use unicode class constructor because unicode type does not exist in Python 3.

If you are reading textual files from disk, it is better to use io.open() instead of built-in open(). io.open() can decode file contents to Unicode using specified encoding and works identically both in Python 2 and 3. Example:
<source lang="python">
import io

with io.open('foo.txt', 'r', encoding='utf-8') as fo:
text = fo.read()
</source>

=== Potential pitfalls ===

When writing Python 2/3-compatible code or converting existing Python 2 codebase to compatible code you should pay attention to the following Python language constructions where TypeError exceptions may happen because of incompatible string types if you don’t get your strings in order:

* String concatenations with + operator (<code>'foo' + 'bar'</code>).
* String formatting — both “old style” (<code>'foo %s' % 'bar'</code>) and “new style” (<code>'foo {}'.format('bar')</code>).
* String joining (<code>', '.join(['foo', 'bar'])</code>).

In all those cases always make sure that you work with Unicode strings only.
Another source of subtle bugs is functions and methods that accept and return binary strings (str type) in Python 2 but Unicode strings in Python 3 (again, str type but with different properties). You can use the following utility functions to “normalize” strings that are provided to such functions/methods and received from them:

<source lang="python">
import sys

__all__ = ['PY2', 'py2_encode', 'py2_decode']

PY2 = sys.version_info[0] == 2

def py2_encode(s, encoding='utf-8'):
"""
Encode Python 2 ``unicode`` to ``str``

In Python 3 the string is not changed.
"""
if PY2 and isinstance(s, unicode):
s = s.encode(encoding)
return s

def py2_decode(s, encoding='utf-8'):
"""
Decode Python 2 ``str`` to ``unicode``

In Python 3 the string is not changed.
"""
if PY2 and isinstance(s, str):
s = s.decode(encoding)
return s
</source>

== Helper Libraries and Tools ==

=== 2to3 ===

[https://docs.python.org/2/library/2to3.html 2to3] script is created by Python developers to help converting existing Python 2 code to Python 3. On Windows it is included in Python distribution (Tools subfolder) but on other OSes you man need to install it separately. For example, on Ubuntu it is included in python-examples package. Note that this script is created for conversion from Python 2 to 3, not for writing portable code, so you need to treat its output with care.

=== Modernize ===

https://python-modernize.readthedocs.io/en/latest/ Modernize] script works on top of 2to3 and is supposed to help you to convert your existing Python 2 code to Python 2/3-compatible. However, in my experiments it did not work reliably even in simple cases so I cannot recommend it for usage.

=== Six ===

[https://pythonhosted.org/six/ Six] was the first library developed to simplify Python 2 to 3 migration. It provides a set of wrappers that hide differences between Python 2 and 3 behind its API. It is less intrusive than the following alternative because it does not monkey-patch built-in names, but in order to use Six library effectively you need to learn its API. Six can be used both for writing new Python 2/3-compatible addons and converting existing ones, but it requires good knowledge of Python 2/3 differences to pick necessary Six features that address specific differences.

Six library is included in the Kodi addon repository as '''script.module.six addon'''.

=== Future ===

[http://python-future.org/ Future] library, like Six, was created to simplify porting existing Python 2 code to 3. But it uses a different approach than Six. Future monkey-patches built-in Python objects to make Python 2 behave like Python 3. The advantage of such approach is that code written using Future is close to idiomatic Python 3 and requires little re-work if you decide to drop Python 2 support in the future. However, such approach may cause problems in some rare edge-cases.

Future library also includes <code>futurize</code> command-line utility for converting existing Python 2 code to 2/3-compatible and in my experiments this utility showed good results.
Future library (without additional utilities) is included in the Kodi addon repository as script.module.future addon.

=== PyCharm ===

PyCharm, except for being a really good Python IDE, also provides code compatibility inspection that helps you write Python 2/3-portable code. Open '''Settings''' (Alt+F7) > '''Editor''' > '''Inspections''' > '''Python''' > '''Code compatibility inspections''' and check Python version that you want to support.

PyCharm Community Edition is free and provides all features you need for creating Python addons for Kodi.

== Writing Portable Code ==

Here I’ll give you some tips about how to write portable code:

* Learn the differences between Python 2 and 3. You need to know at least the most important differences between the two major Python versions.

* Use version control system — git or mercurial — to track changes in your code. If you are porting existing code, do it in a separate branch.

* No matter if you are going to write a brand new addon or to port existing addon to Python 3, carefully choose your tools. It is totally possible to write portable code without any helper tools and libraries, but you need to know what you are doing. However, in most cases I’d recommend you to use Future library and its utilities. Carefully read Future documentation.

* Put the following line at the beginning of all your modules:
<source lang="python">
from __future__ import absolute_import, division, unicode_literals
</source>
This will enable respective Python 3 features in your code. You don’t need to import print_function (another Python 3 feature) because in Kodi addons you should to use xbmc.log() to write messages to the Kodi log file.

=== Writing New Addons ===

Check the [http://python-future.org/quickstart.html#if-you-are-writing-code-from-scratch Quick Start Guide] section of Future library documentation. You can also use Six library, but, as it was said previously, you need to learn its API to pick the necessary features to address specific Python version differences, while Future allows to write your code in (almost) idiomatic Python 3.

A brief procedure for writing new Python 2/3-compatible addons with Future library:

# Create a new virtual environment with Python 3 interpreter and activate it.
# Install Future library: <code>pip install future</code>.
# Point your IDE (Integrated Development Environment) to that environment. For example, in PyCharm: '''File''' > '''Setting''' > '''Project''' > '''Project Interpreter'''.
# Put the following line at the beginning of your Python code:
<source lang="python">
from __future__ import absolute_import, division, unicode_literals
from builtins import *
from future import standard_library
standard_library.install_aliases()
</source>
Write your addon using Python 3 syntax and standard library names. However, you may still need to use wrappers from Future library for some specific cases. Read Future docs for more info.

=== Porting Existing Addons ===

A brief procedure for porting existing code to Python 3-compatible with Future library:

# Install Future library into your working Python 2 virtual environment: <code>pip install future</code>.
# Run <code>futurize</code> utility to convert all your scripts and modules to portable code.

Test your new or converted addon in Kodi with Python 2 interpreter and fix all found issues. After that test the addon in Kodi with Python 3 interpreter and again fix all found issues.
You can use [https://github.com/romanvm/kodi.web-pdb Web-PDB] debugger for troubleshooting issues in your code. It is compatible with both Python 2 and 3.

== Links ==

* Python 2 and 3 differences: https://docs.python.org/3.6/whatsnew/3.0.html
* Future library documentation: http://python-future.org/
* Six library documentation: https://pythonhosted.org/six/
* Kodi test builds with Python 3 for Windows: http://mirrors.xbmc.org/test-builds/windows/win32/
* Kodi test builds with Python 3 for Linux Ubuntu: https://launchpad.net/~wsnipex/+archive/ubuntu/kodi-python3/+packages
* “Python 3 migration” section on the Kodi official forum: https://forum.kodi.tv/forumdisplay.php?fid=281

HOW-TO:Debug Python Scripts with Web-PDB

2018-01-06T09:27:50Z

Roman V M:

[https://github.com/romanvm/kodi.web-pdb Web-PDB] is a remote web-interface to Python's built-in [https://docs.python.org/2/library/pdb.html PDB] debugger with additional convenience features. It is not tied to any IDE or other software, all you need is a common web-browser, e.g. Chrome or Firefox.
[[File:web-pdb.png]]

Web-PDB for Kodi is available as an addon in the official Kodi addons repo.

== How To Use Web-PDB for Kodi ==

1. Install Web-PDB addon: '''Kodi Add-on repository > Program add-ons > Web-PDB'''.

2. Add <code>script.module.web-pdb</code> to [[addon.xml]] as a dependency:
<syntaxhighlight lang="xml" enclose="div">
<requires>
...
<import addon="script.module.web-pdb" />
<requires>
</syntaxhighlight>

3. Restart Kodi so that it re-reads addon dependencies.

4. Insert the following line into your addon code at the point where you want to start debugging:
<syntaxhighlight lang="python" enclose="div">
import web_pdb; web_pdb.set_trace()
</syntaxhighlight>

The <code>set_trace()</code> call will suspend your addon and open a web-UI at the default port 5555 (port value can be changed). At the same time a notification will be displayed in Kodi, indicating that a debug session is active. The notification also shows web-UI host/port.

5. Enter in your the address bar of your browser: <code>http://<your Kodi machine hostname or IP>:5555</code>, for example <code>http://monty-python:5555</code>. Use <code>localhost</code> as a hostname if you are connecting from the same machine that runs Kodi. If everything is OK, you should see the Web-PDB UI. Now you can use all PDB commands and features. Additional '''Current file''', '''Globals''' and '''Locals''' information boxes help you better track your program runtime state.

== More Information ==

* [https://github.com/romanvm/kodi.web-pdb Web-PDB for Kodi on GitHub].
* [https://docs.python.org/2/library/pdb.html PDB debugger documentation].

File:Web-pdb.png

2018-01-06T09:22:39Z

Roman V M: Web-PDB debugger screenshot

Web-PDB debugger screenshot

General information about migration to Python 3

2018-01-04T11:28:24Z

Roman V M: /* Unicode strings by default */

== Overview ==

Python 3 was released 9 years ago and EOL for Python 2.7 is scheduled for 2020. Currently more and more Python project are switching to Python 3 or 2/3 compatible code. The issue of migrating Kodi’s built-in Python interpreter to Python 3.x was brought up on the Kodi forum several times. Now, thanks to a successful GSOC 2017 project, we have a working Python 3 interpreter for Kodi. That is why on Kodi DevCon 2017 Team Kodi decided that we will switch to Python 3 in the future Kodi release (v.19 preliminarily).

Unfortunately, Python 3.x versions are not backward-compatible with Python 2.x, so we decided to urge addon developers to start porting their addons to Python 3 by making the addons compatible with both Python 2 and 3 and submitting them to the official repo. This way, when Kodi with Python 3 is released, we won’t have a situation when we have no working addons.

== The Process ==

* Starting from Kodi 18 (Leia), only addons that are compatible with both Python 2 and 3 will be accepted to the official addon repository.
* We will provide Kodi builds with Python 3 so that addon developers can tests if their addons are really compatible with Python 3.
* A thread will be created on the Kodi forum so that addon devs can get help with their migration process.

== Python 3 And 2 Differences ==

Below is a brief overview of the main differences between Python 3 and 2.

=== Unicode strings by default ===

In Python 2 built-in <code>str</code> type holds the sequence of bytes so it can be used to store both binary data and textual data in ASCII or any of 8-bit fixed-length encodings (there were plenty). Python 2 also has <code>unicode</code> type that can store text in various writing systems. A minimal unit of a <code>unicode</code> object is a Unicode codepoint (a Unicode character). Both <code>str</code> and <code>unicode</code> objects can be mixed and matched together, for example, concatenated using <code>+</code> operator.

In Python 3 <code>str</code> type holds Unicode characters, and for binary data a new <code>bytes</code> type was added. This type was also added to Python 2.7 to simplify porting to Python 3 but in Python 2.7 bytes is simply an alias for <code>str</code>. The <code>str</code> and <code>bytes</code> types are incompatible, and an attempt to use both together will result in <code>TypeError</code> exception. A <code>bytes</code> object can be converted to <code>str</code> using .decode() method, and a <code>str</code> is converted to <code>bytes</code> using .encode() method.

=== Absolute Import ===
Python 3 uses so called “absolute import” and you can no longer import modules from inside a package using only a module name. You need to either to use either a fully qualified name or a “dotted” notation.

Let’s assume that you have a foo package that contains bar and spam modules, and you need to import bar module from spam. In Python 2 you can do:
<source lang="python">
import bar
</source>

But in Python 3 you need to use either

<source lang="python">
import foo.bar
</source>

or

<source lang="python">
from . import bar
</source>

=== Floating Point Division ===

In Python 2 applying division operator to int numbers produces int result. For example:

<source lang="python">
>>> 4 / 3
1
</source>

In Python 3 division operator always gives you a float result.

<source lang="python">
>>> 4/3
1.3333333333333333
</source>

To get an int result you need to use integer division operator:

<source lang="python">
>>> 4 // 3
1
</source>

=== Print Function ===

In Python 3 print statement has been replaced with print() function. This is not very relevant to Kodi because using print in Python addons is discouraged and xbmc.log() function should be used to write messages to the Kodi log.

=== Default Iterators ===

In Python 3 most function and methods that produce sequences return iterators instead of lists. For example, range() function and .items(), .keys(), .values() of dict type produce iterators instead of lists, and their Python 2 analogues — xrange() function and .iter* methods of dict type — have been removed.

=== No long Integers ===

In Python 3 long integer type has been removed and int type can now hold a number of arbitrary length.

Those are only some of the differences between Python 2 and 3. For a more complete list see [https://docs.python.org/3.6/whatsnew/3.0.html official Python documentation].

== Some Useful Information ==

As it was said above, Python 3 includes a number of incompatible changes, so to avoid situation when after upgrading Kodi for Python 3 support most Python addons will be broken developers should start to port their addons to be compatible with both Python 2 and 3. This is not a trivial task but fortunately there are a number of tools and recommendations to simplify this process. Here I’ll give you some advice and information about tools that will simplify creating portable code that runs on both Python 2 and 3.

=== Know Your Strings! ===

This is the most problematic part of porting Python 2 code to Python 3 so I put it first. One of the most notable differences between Python 2 and 3 is clear separation between “textual” and “binary” strings, that is, between textual data and their binary representation. This difference is often difficult to understand, especially for novice developers (not only in Python), and the fact that in Python 2 you can mix together str (binary data) and unicode (textual data) adds to that difficulty.

There are many explanations in the Internet, but here are the most basic things about strings that you need to know:

“Textual” or Unicode strings (unicode in Python 2 and str in Python 3) hold text as a sequence of characters. A minimal unit in such strings is a Unicode character — an abstract entity that represent an alphabet symbol, a punctuation sign or other symbol included in the Unicode character table. Of course, such symbols have concrete internal binary representation (a computer is a binary machine after all), but this representation is not relevant to us. All you need to know is that “textual” strings contain text units without being tied to concrete binary representation.

Unlike Unicode strings, binary strings (str in Python 2 and bytes in both Python 2 and 3) hold binary data, and a minimal unit of such strings is a byte or 8 bits. Historically, in Python 2 such strings are used for textual data encoded in ASCII or other 1-byte fixed-length encoding (e.g. Windows 1251 for Cyrillic alphabets). However, this approach has its limitation, that is why unicode type was introduced in Python 2, and in Python 3 it was renamed to str and made a default container for textual data.

Unfortunately, Python 2 allows to mix and match Unicode and binary strings in the same context and this creates big problems when porting Python 2 code to Python 3. So in order to successfully port your code to Python 3 you need to carefully consider how textual data are processed in your addon. The following recommendations will help you avoid problems when writing Python 2/3 compatible code.

=== Avoid using binary strings for text ===

Store all your text as Unicode strings. This means that all binary strings received from external sources (textual files, websites, various APIs) need to be decoded using appropriate encoding (UTF-8 in most cases). If you need to use strings literals (although using hardcoded strings for user-facing text in Kodi is strongly discouraged), they need to be defined as unicode strings as well. You can use either
<source lang="python">
from __future__ import unicode_literals
</source>
at the beginning of your modules to enable unicode strings by default as in Python 3 or u-strings, e.g. <code>u'foo'</code> (Python 3 understands them too) — it doesn't really matter as long as you are doing it consistently.

=== Know the libraries you are using ===
Working with Python Standard Library and third-party libraries (including Kodi Python API) you should know that types their functions and methods expect and what types they return. For example, in popular requests library Response.text property returns a Unicode string and Response.content returns a binary string.

If a function/method/property returns a binary string, you need to decode it to a Unicode string using .decode() method. Don’t use unicode class constructor because unicode type does not exist in Python 3.

If you are reading textual files from disk, it is better to use io.open() instead of built-in open(). io.open() can decode file contents to Unicode using specified encoding and works identically both in Python 2 and 3. Example:
<source lang="python">
import io

with io.open('foo.txt', 'r', encoding='utf-8') as fo:
text = fo.read()
</source>

=== Potential pitfalls ===

When writing Python 2/3-compatible code or converting existing Python 2 codebase to compatible code you should pay attention to the following Python language constructions where TypeError exceptions may happen because of incompatible string types if you don’t get your strings in order:

* String concatenations with + operator (<code>'foo' + 'bar'</code>).
* String formatting — both “old style” (<code>'foo %s' % 'bar'</code>) and “new style” (<code>'foo {}'.format('bar')</code>).
* String joining (<code>', '.join(['foo', 'bar'])</code>).

In all those cases always make sure that you work with Unicode strings only.
Another source of subtle bugs is functions and methods that accept and return binary strings (str type) in Python 2 but Unicode strings in Python 3 (again, str type but with different properties). You can use the following utility functions to “normalize” strings that are provided to such functions/methods and received from them:

<source lang="python">
import sys

__all__ = ['PY2', 'py2_encode', 'py2_decode']

PY2 = sys.version_info[0] == 2

def py2_encode(s, encoding='utf-8'):
"""
Encode Python 2 ``unicode`` to ``str``

In Python 3 the string is not changed.
"""
if PY2 and isinstance(s, unicode):
s = s.encode(encoding)
return s

def py2_decode(s, encoding='utf-8'):
"""
Decode Python 2 ``str`` to ``unicode``

In Python 3 the string is not changed.
"""
if PY2 and isinstance(s, str):
s = s.decode(encoding)
return s
</source>

== Helper Libraries and Tools ==

=== 2to3 ===

[https://docs.python.org/2/library/2to3.html 2to3] script is created by Python developers to help converting existing Python 2 code to Python 3. On Windows it is included in Python distribution (Tools subfolder) but on other OSes you man need to install it separately. For example, on Ubuntu it is included in python-examples package. Note that this script is created for conversion from Python 2 to 3, not for writing portable code, so you need to treat its output with care.

=== Modernize ===

https://python-modernize.readthedocs.io/en/latest/ Modernize] script works on top of 2to3 and is supposed to help you to convert your existing Python 2 code to Python 2/3-compatible. However, in my experiments it did not work reliably even in simple cases so I cannot recommend it for usage.

=== Six ===

[https://pythonhosted.org/six/ Six] was the first library developed to simplify Python 2 to 3 migration. It provides a set of wrappers that hide differences between Python 2 and 3 behind its API. It is less intrusive than the following alternative because it does not monkey-patch built-in names, but in order to use Six library effectively you need to learn its API. Six can be used both for writing new Python 2/3-compatible addons and converting existing ones, but it requires good knowledge of Python 2/3 differences to pick necessary Six features that address specific differences.

Six library is included in the Kodi addon repository as '''script.module.six addon'''.

=== Future ===

[http://python-future.org/ Future] library, like Six, was created to simplify porting existing Python 2 code to 3. But it uses a different approach than Six. Future monkey-patches built-in Python objects to make Python 2 behave like Python 3. The advantage of such approach is that code written using Future is close to idiomatic Python 3 and requires little re-work if you decide to drop Python 2 support in the future. However, such approach may cause problems in some rare edge-cases.

Future library also includes <code>futurize</code> command-line utility for converting existing Python 2 code to 2/3-compatible and in my experiments this utility showed good results.
Future library (without additional utilities) is included in the Kodi addon repository as script.module.future addon.

=== PyCharm ===

PyCharm, except for being a really good Python IDE, also provides code compatibility inspection that helps you write Python 2/3-portable code. Open '''Settings''' (Alt+F7) > '''Editor''' > '''Inspections''' > '''Python''' > '''Code compatibility inspections''' and check Python version that you want to support.

PyCharm Community Edition is free and provides all features you need for creating Python addons for Kodi.

== Writing Portable Code ==

Here I’ll give you some tips about how to write portable code:

* Learn the differences between Python 2 and 3. You need to know at least the most important differences between the two major Python versions.

* Use version control system — git or mercurial — to track changes in your code. If you are porting existing code, do it in a separate branch.

* No matter if you are going to write a brand new addon or to port existing addon to Python 3, carefully choose your tools. It is totally possible to write portable code without any helper tools and libraries, but you need to know what you are doing. However, in most cases I’d recommend you to use Future library and its utilities. Carefully read Future documentation.

* Put the following line at the beginning of all your modules:
<source lang="python">
from __future__ import absolute_import, division, unicode_literals
</source>
This will enable respective Python 3 features in your code. You don’t need to import print_function (another Python 3 feature) because in Kodi addons you should to use xbmc.log() to write messages to the Kodi log file.

=== Writing New Addons ===

Check the [http://python-future.org/quickstart.html#if-you-are-writing-code-from-scratch Quick Start Guide] section of Future library documentation. You can also use Six library, but, as it was said previously, you need to learn its API to pick the necessary features to address specific Python version differences, while Future allows to write your code in (almost) idiomatic Python 3.

A brief procedure for writing new Python 2/3-compatible addons with Future library:

# Create a new virtual environment with Python 3 interpreter and activate it.
# Install Future library: <code>pip install future</code>.
# Point your IDE (Integrated Development Environment) to that environment. For example, in PyCharm: '''File''' > '''Setting''' > '''Project''' > '''Project Interpreter'''.
# Put the following line at the beginning of your Python code:
<source lang="python">
from __future__ import absolute_import, division, unicode_literals
from builtins import *
from future import standard_library
standard_library.install_aliases()
</source>
Write your addon using Python 3 syntax and standard library names.

=== Porting Existing Addons ===

A brief procedure for porting existing code to Python 3-compatible with Future library:

# Install Future library into your working Python 2 virtual environment: <code>pip install future</code>.
# Run <code>futurize</code> utility to convert all your scripts and modules to portable code.

Test your new or converted addon in Kodi with Python 2 interpreter and fix all found issues. After that test the addon in Kodi with Python 3 interpreter and again fix all found issues.
You can use [https://github.com/romanvm/kodi.web-pdb Web-PDB] debugger for troubleshooting issues in your code. It is compatible with both Python 2 and 3.

== Links ==

* Python 2 and 3 differences: https://docs.python.org/3.6/whatsnew/3.0.html
* Future library documentation: http://python-future.org/
* Six library documentation: https://pythonhosted.org/six/
* Kodi test builds with Python 3 for Windows: http://mirrors.xbmc.org/test-builds/windows/win32/
* Kodi test builds with Python 3 for Linux Ubuntu: https://launchpad.net/~wsnipex/+archive/ubuntu/kodi-python3/+packages
* “Python 3 migration” section on the Kodi official forum: https://forum.kodi.tv/forumdisplay.php?fid=281

General information about migration to Python 3

2018-01-04T11:27:12Z

Roman V M: /* Unicode strings by default */

== Overview ==

Python 3 was released 9 years ago and EOL for Python 2.7 is scheduled for 2020. Currently more and more Python project are switching to Python 3 or 2/3 compatible code. The issue of migrating Kodi’s built-in Python interpreter to Python 3.x was brought up on the Kodi forum several times. Now, thanks to a successful GSOC 2017 project, we have a working Python 3 interpreter for Kodi. That is why on Kodi DevCon 2017 Team Kodi decided that we will switch to Python 3 in the future Kodi release (v.19 preliminarily).

Unfortunately, Python 3.x versions are not backward-compatible with Python 2.x, so we decided to urge addon developers to start porting their addons to Python 3 by making the addons compatible with both Python 2 and 3 and submitting them to the official repo. This way, when Kodi with Python 3 is released, we won’t have a situation when we have no working addons.

== The Process ==

* Starting from Kodi 18 (Leia), only addons that are compatible with both Python 2 and 3 will be accepted to the official addon repository.
* We will provide Kodi builds with Python 3 so that addon developers can tests if their addons are really compatible with Python 3.
* A thread will be created on the Kodi forum so that addon devs can get help with their migration process.

== Python 3 And 2 Differences ==

Below is a brief overview of the main differences between Python 3 and 2.

=== Unicode strings by default ===

In Python 2 built-in <code>str</code> type holds the sequence of bytes so it can be used to store both binary data and textual data in ASCII or any of 8-bit fixed-length encodings (there were plenty). Python 2 also has <code>unicode</code> type that can store text in various writing systems. A minimal unit of a <code>unicode</code> object is a Unicode codepoint (a Unicode character). Both <code>str</code> and <code>unicode</code> objects can be mixed and matched together, for example, concatenated using <code>+</code> operator.

In Python 3 <code>str</code> type holds Unicode characters, and for binary data a new <code>bytes</code> type was added. This type was also added to Python 2.7 to simplify porting to Python 3 but in Python 2.7 bytes is simply an alias for <code>str</code>. The <code>str</code> and bytes types are incompatible, and an attempt to use both together will result in <code>TypeError</code> exception. A <code>bytes</code> object can be converted to <code>str</code> using .decode() method, and a <code>str</code> is converted to <code>bytes</code> using .encode() method.

=== Absolute Import ===
Python 3 uses so called “absolute import” and you can no longer import modules from inside a package using only a module name. You need to either to use either a fully qualified name or a “dotted” notation.

Let’s assume that you have a foo package that contains bar and spam modules, and you need to import bar module from spam. In Python 2 you can do:
<source lang="python">
import bar
</source>

But in Python 3 you need to use either

<source lang="python">
import foo.bar
</source>

or

<source lang="python">
from . import bar
</source>

=== Floating Point Division ===

In Python 2 applying division operator to int numbers produces int result. For example:

<source lang="python">
>>> 4 / 3
1
</source>

In Python 3 division operator always gives you a float result.

<source lang="python">
>>> 4/3
1.3333333333333333
</source>

To get an int result you need to use integer division operator:

<source lang="python">
>>> 4 // 3
1
</source>

=== Print Function ===

In Python 3 print statement has been replaced with print() function. This is not very relevant to Kodi because using print in Python addons is discouraged and xbmc.log() function should be used to write messages to the Kodi log.

=== Default Iterators ===

In Python 3 most function and methods that produce sequences return iterators instead of lists. For example, range() function and .items(), .keys(), .values() of dict type produce iterators instead of lists, and their Python 2 analogues — xrange() function and .iter* methods of dict type — have been removed.

=== No long Integers ===

In Python 3 long integer type has been removed and int type can now hold a number of arbitrary length.

Those are only some of the differences between Python 2 and 3. For a more complete list see [https://docs.python.org/3.6/whatsnew/3.0.html official Python documentation].

== Some Useful Information ==

As it was said above, Python 3 includes a number of incompatible changes, so to avoid situation when after upgrading Kodi for Python 3 support most Python addons will be broken developers should start to port their addons to be compatible with both Python 2 and 3. This is not a trivial task but fortunately there are a number of tools and recommendations to simplify this process. Here I’ll give you some advice and information about tools that will simplify creating portable code that runs on both Python 2 and 3.

=== Know Your Strings! ===

This is the most problematic part of porting Python 2 code to Python 3 so I put it first. One of the most notable differences between Python 2 and 3 is clear separation between “textual” and “binary” strings, that is, between textual data and their binary representation. This difference is often difficult to understand, especially for novice developers (not only in Python), and the fact that in Python 2 you can mix together str (binary data) and unicode (textual data) adds to that difficulty.

There are many explanations in the Internet, but here are the most basic things about strings that you need to know:

“Textual” or Unicode strings (unicode in Python 2 and str in Python 3) hold text as a sequence of characters. A minimal unit in such strings is a Unicode character — an abstract entity that represent an alphabet symbol, a punctuation sign or other symbol included in the Unicode character table. Of course, such symbols have concrete internal binary representation (a computer is a binary machine after all), but this representation is not relevant to us. All you need to know is that “textual” strings contain text units without being tied to concrete binary representation.

Unlike Unicode strings, binary strings (str in Python 2 and bytes in both Python 2 and 3) hold binary data, and a minimal unit of such strings is a byte or 8 bits. Historically, in Python 2 such strings are used for textual data encoded in ASCII or other 1-byte fixed-length encoding (e.g. Windows 1251 for Cyrillic alphabets). However, this approach has its limitation, that is why unicode type was introduced in Python 2, and in Python 3 it was renamed to str and made a default container for textual data.

Unfortunately, Python 2 allows to mix and match Unicode and binary strings in the same context and this creates big problems when porting Python 2 code to Python 3. So in order to successfully port your code to Python 3 you need to carefully consider how textual data are processed in your addon. The following recommendations will help you avoid problems when writing Python 2/3 compatible code.

=== Avoid using binary strings for text ===

Store all your text as Unicode strings. This means that all binary strings received from external sources (textual files, websites, various APIs) need to be decoded using appropriate encoding (UTF-8 in most cases). If you need to use strings literals (although using hardcoded strings for user-facing text in Kodi is strongly discouraged), they need to be defined as unicode strings as well. You can use either
<source lang="python">
from __future__ import unicode_literals
</source>
at the beginning of your modules to enable unicode strings by default as in Python 3 or u-strings, e.g. <code>u'foo'</code> (Python 3 understands them too) — it doesn't really matter as long as you are doing it consistently.

=== Know the libraries you are using ===
Working with Python Standard Library and third-party libraries (including Kodi Python API) you should know that types their functions and methods expect and what types they return. For example, in popular requests library Response.text property returns a Unicode string and Response.content returns a binary string.

If a function/method/property returns a binary string, you need to decode it to a Unicode string using .decode() method. Don’t use unicode class constructor because unicode type does not exist in Python 3.

If you are reading textual files from disk, it is better to use io.open() instead of built-in open(). io.open() can decode file contents to Unicode using specified encoding and works identically both in Python 2 and 3. Example:
<source lang="python">
import io

with io.open('foo.txt', 'r', encoding='utf-8') as fo:
text = fo.read()
</source>

=== Potential pitfalls ===

When writing Python 2/3-compatible code or converting existing Python 2 codebase to compatible code you should pay attention to the following Python language constructions where TypeError exceptions may happen because of incompatible string types if you don’t get your strings in order:

* String concatenations with + operator (<code>'foo' + 'bar'</code>).
* String formatting — both “old style” (<code>'foo %s' % 'bar'</code>) and “new style” (<code>'foo {}'.format('bar')</code>).
* String joining (<code>', '.join(['foo', 'bar'])</code>).

In all those cases always make sure that you work with Unicode strings only.
Another source of subtle bugs is functions and methods that accept and return binary strings (str type) in Python 2 but Unicode strings in Python 3 (again, str type but with different properties). You can use the following utility functions to “normalize” strings that are provided to such functions/methods and received from them:

<source lang="python">
import sys

__all__ = ['PY2', 'py2_encode', 'py2_decode']

PY2 = sys.version_info[0] == 2

def py2_encode(s, encoding='utf-8'):
"""
Encode Python 2 ``unicode`` to ``str``

In Python 3 the string is not changed.
"""
if PY2 and isinstance(s, unicode):
s = s.encode(encoding)
return s

def py2_decode(s, encoding='utf-8'):
"""
Decode Python 2 ``str`` to ``unicode``

In Python 3 the string is not changed.
"""
if PY2 and isinstance(s, str):
s = s.decode(encoding)
return s
</source>

== Helper Libraries and Tools ==

=== 2to3 ===

[https://docs.python.org/2/library/2to3.html 2to3] script is created by Python developers to help converting existing Python 2 code to Python 3. On Windows it is included in Python distribution (Tools subfolder) but on other OSes you man need to install it separately. For example, on Ubuntu it is included in python-examples package. Note that this script is created for conversion from Python 2 to 3, not for writing portable code, so you need to treat its output with care.

=== Modernize ===

https://python-modernize.readthedocs.io/en/latest/ Modernize] script works on top of 2to3 and is supposed to help you to convert your existing Python 2 code to Python 2/3-compatible. However, in my experiments it did not work reliably even in simple cases so I cannot recommend it for usage.

=== Six ===

[https://pythonhosted.org/six/ Six] was the first library developed to simplify Python 2 to 3 migration. It provides a set of wrappers that hide differences between Python 2 and 3 behind its API. It is less intrusive than the following alternative because it does not monkey-patch built-in names, but in order to use Six library effectively you need to learn its API. Six can be used both for writing new Python 2/3-compatible addons and converting existing ones, but it requires good knowledge of Python 2/3 differences to pick necessary Six features that address specific differences.

Six library is included in the Kodi addon repository as '''script.module.six addon'''.

=== Future ===

[http://python-future.org/ Future] library, like Six, was created to simplify porting existing Python 2 code to 3. But it uses a different approach than Six. Future monkey-patches built-in Python objects to make Python 2 behave like Python 3. The advantage of such approach is that code written using Future is close to idiomatic Python 3 and requires little re-work if you decide to drop Python 2 support in the future. However, such approach may cause problems in some rare edge-cases.

Future library also includes <code>futurize</code> command-line utility for converting existing Python 2 code to 2/3-compatible and in my experiments this utility showed good results.
Future library (without additional utilities) is included in the Kodi addon repository as script.module.future addon.

=== PyCharm ===

PyCharm, except for being a really good Python IDE, also provides code compatibility inspection that helps you write Python 2/3-portable code. Open '''Settings''' (Alt+F7) > '''Editor''' > '''Inspections''' > '''Python''' > '''Code compatibility inspections''' and check Python version that you want to support.

PyCharm Community Edition is free and provides all features you need for creating Python addons for Kodi.

== Writing Portable Code ==

Here I’ll give you some tips about how to write portable code:

* Learn the differences between Python 2 and 3. You need to know at least the most important differences between the two major Python versions.

* Use version control system — git or mercurial — to track changes in your code. If you are porting existing code, do it in a separate branch.

* No matter if you are going to write a brand new addon or to port existing addon to Python 3, carefully choose your tools. It is totally possible to write portable code without any helper tools and libraries, but you need to know what you are doing. However, in most cases I’d recommend you to use Future library and its utilities. Carefully read Future documentation.

* Put the following line at the beginning of all your modules:
<source lang="python">
from __future__ import absolute_import, division, unicode_literals
</source>
This will enable respective Python 3 features in your code. You don’t need to import print_function (another Python 3 feature) because in Kodi addons you should to use xbmc.log() to write messages to the Kodi log file.

=== Writing New Addons ===

Check the [http://python-future.org/quickstart.html#if-you-are-writing-code-from-scratch Quick Start Guide] section of Future library documentation. You can also use Six library, but, as it was said previously, you need to learn its API to pick the necessary features to address specific Python version differences, while Future allows to write your code in (almost) idiomatic Python 3.

A brief procedure for writing new Python 2/3-compatible addons with Future library:

# Create a new virtual environment with Python 3 interpreter and activate it.
# Install Future library: <code>pip install future</code>.
# Point your IDE (Integrated Development Environment) to that environment. For example, in PyCharm: '''File''' > '''Setting''' > '''Project''' > '''Project Interpreter'''.
# Put the following line at the beginning of your Python code:
<source lang="python">
from __future__ import absolute_import, division, unicode_literals
from builtins import *
from future import standard_library
standard_library.install_aliases()
</source>
Write your addon using Python 3 syntax and standard library names.

=== Porting Existing Addons ===

A brief procedure for porting existing code to Python 3-compatible with Future library:

# Install Future library into your working Python 2 virtual environment: <code>pip install future</code>.
# Run <code>futurize</code> utility to convert all your scripts and modules to portable code.

Test your new or converted addon in Kodi with Python 2 interpreter and fix all found issues. After that test the addon in Kodi with Python 3 interpreter and again fix all found issues.
You can use [https://github.com/romanvm/kodi.web-pdb Web-PDB] debugger for troubleshooting issues in your code. It is compatible with both Python 2 and 3.

== Links ==

* Python 2 and 3 differences: https://docs.python.org/3.6/whatsnew/3.0.html
* Future library documentation: http://python-future.org/
* Six library documentation: https://pythonhosted.org/six/
* Kodi test builds with Python 3 for Windows: http://mirrors.xbmc.org/test-builds/windows/win32/
* Kodi test builds with Python 3 for Linux Ubuntu: https://launchpad.net/~wsnipex/+archive/ubuntu/kodi-python3/+packages
* “Python 3 migration” section on the Kodi official forum: https://forum.kodi.tv/forumdisplay.php?fid=281

General information about migration to Python 3

2018-01-04T11:22:46Z

Roman V M: Created page with "== Overview == Python 3 was released 9 years ago and EOL for Python 2.7 is scheduled for 2020. Currently more and more Python project are switching to Python 3 or 2/3 compati..."

== Overview ==

Python 3 was released 9 years ago and EOL for Python 2.7 is scheduled for 2020. Currently more and more Python project are switching to Python 3 or 2/3 compatible code. The issue of migrating Kodi’s built-in Python interpreter to Python 3.x was brought up on the Kodi forum several times. Now, thanks to a successful GSOC 2017 project, we have a working Python 3 interpreter for Kodi. That is why on Kodi DevCon 2017 Team Kodi decided that we will switch to Python 3 in the future Kodi release (v.19 preliminarily).

Unfortunately, Python 3.x versions are not backward-compatible with Python 2.x, so we decided to urge addon developers to start porting their addons to Python 3 by making the addons compatible with both Python 2 and 3 and submitting them to the official repo. This way, when Kodi with Python 3 is released, we won’t have a situation when we have no working addons.

== The Process ==

* Starting from Kodi 18 (Leia), only addons that are compatible with both Python 2 and 3 will be accepted to the official addon repository.
* We will provide Kodi builds with Python 3 so that addon developers can tests if their addons are really compatible with Python 3.
* A thread will be created on the Kodi forum so that addon devs can get help with their migration process.

== Python 3 And 2 Differences ==

Below is a brief overview of the main differences between Python 3 and 2.

=== Unicode strings by default ===

In Python 2 built-in str type holds the sequence of bytes so it can be used to store both binary data and textual data in ASCII or any of 8-bit fixed-length encodings (there were plenty). Python 2 also has unicode type that can store text in various writing systems. A minimal unit of a unicode object is a Unicode codepoint (a Unicode character). Both str and unicode objects can be mixed and matched together, for example, concatenated using + operator.

In Python 3 str type holds Unicode characters, and for binary data a new bytes type was added. This type was also added to Python 2.7 to simplify porting to Python 3 but in Python 2.7 bytes is simply an alias for str. The str and bytes types are incompatible, and an attempt to use both together will result in TypeError exception. A bytes object can be converted to str using .decode() method, and a str is converted to bytes using .encode() method.

=== Absolute Import ===
Python 3 uses so called “absolute import” and you can no longer import modules from inside a package using only a module name. You need to either to use either a fully qualified name or a “dotted” notation.

Let’s assume that you have a foo package that contains bar and spam modules, and you need to import bar module from spam. In Python 2 you can do:
<source lang="python">
import bar
</source>

But in Python 3 you need to use either

<source lang="python">
import foo.bar
</source>

or

<source lang="python">
from . import bar
</source>

=== Floating Point Division ===

In Python 2 applying division operator to int numbers produces int result. For example:

<source lang="python">
>>> 4 / 3
1
</source>

In Python 3 division operator always gives you a float result.

<source lang="python">
>>> 4/3
1.3333333333333333
</source>

To get an int result you need to use integer division operator:

<source lang="python">
>>> 4 // 3
1
</source>

=== Print Function ===

In Python 3 print statement has been replaced with print() function. This is not very relevant to Kodi because using print in Python addons is discouraged and xbmc.log() function should be used to write messages to the Kodi log.

=== Default Iterators ===

In Python 3 most function and methods that produce sequences return iterators instead of lists. For example, range() function and .items(), .keys(), .values() of dict type produce iterators instead of lists, and their Python 2 analogues — xrange() function and .iter* methods of dict type — have been removed.

=== No long Integers ===

In Python 3 long integer type has been removed and int type can now hold a number of arbitrary length.

Those are only some of the differences between Python 2 and 3. For a more complete list see [https://docs.python.org/3.6/whatsnew/3.0.html official Python documentation].

== Some Useful Information ==

As it was said above, Python 3 includes a number of incompatible changes, so to avoid situation when after upgrading Kodi for Python 3 support most Python addons will be broken developers should start to port their addons to be compatible with both Python 2 and 3. This is not a trivial task but fortunately there are a number of tools and recommendations to simplify this process. Here I’ll give you some advice and information about tools that will simplify creating portable code that runs on both Python 2 and 3.

=== Know Your Strings! ===

This is the most problematic part of porting Python 2 code to Python 3 so I put it first. One of the most notable differences between Python 2 and 3 is clear separation between “textual” and “binary” strings, that is, between textual data and their binary representation. This difference is often difficult to understand, especially for novice developers (not only in Python), and the fact that in Python 2 you can mix together str (binary data) and unicode (textual data) adds to that difficulty.

There are many explanations in the Internet, but here are the most basic things about strings that you need to know:

“Textual” or Unicode strings (unicode in Python 2 and str in Python 3) hold text as a sequence of characters. A minimal unit in such strings is a Unicode character — an abstract entity that represent an alphabet symbol, a punctuation sign or other symbol included in the Unicode character table. Of course, such symbols have concrete internal binary representation (a computer is a binary machine after all), but this representation is not relevant to us. All you need to know is that “textual” strings contain text units without being tied to concrete binary representation.

Unlike Unicode strings, binary strings (str in Python 2 and bytes in both Python 2 and 3) hold binary data, and a minimal unit of such strings is a byte or 8 bits. Historically, in Python 2 such strings are used for textual data encoded in ASCII or other 1-byte fixed-length encoding (e.g. Windows 1251 for Cyrillic alphabets). However, this approach has its limitation, that is why unicode type was introduced in Python 2, and in Python 3 it was renamed to str and made a default container for textual data.

Unfortunately, Python 2 allows to mix and match Unicode and binary strings in the same context and this creates big problems when porting Python 2 code to Python 3. So in order to successfully port your code to Python 3 you need to carefully consider how textual data are processed in your addon. The following recommendations will help you avoid problems when writing Python 2/3 compatible code.

=== Avoid using binary strings for text ===

Store all your text as Unicode strings. This means that all binary strings received from external sources (textual files, websites, various APIs) need to be decoded using appropriate encoding (UTF-8 in most cases). If you need to use strings literals (although using hardcoded strings for user-facing text in Kodi is strongly discouraged), they need to be defined as unicode strings as well. You can use either
<source lang="python">
from __future__ import unicode_literals
</source>
at the beginning of your modules to enable unicode strings by default as in Python 3 or u-strings, e.g. <code>u'foo'</code> (Python 3 understands them too) — it doesn't really matter as long as you are doing it consistently.

=== Know the libraries you are using ===
Working with Python Standard Library and third-party libraries (including Kodi Python API) you should know that types their functions and methods expect and what types they return. For example, in popular requests library Response.text property returns a Unicode string and Response.content returns a binary string.

If a function/method/property returns a binary string, you need to decode it to a Unicode string using .decode() method. Don’t use unicode class constructor because unicode type does not exist in Python 3.

If you are reading textual files from disk, it is better to use io.open() instead of built-in open(). io.open() can decode file contents to Unicode using specified encoding and works identically both in Python 2 and 3. Example:
<source lang="python">
import io

with io.open('foo.txt', 'r', encoding='utf-8') as fo:
text = fo.read()
</source>

=== Potential pitfalls ===

When writing Python 2/3-compatible code or converting existing Python 2 codebase to compatible code you should pay attention to the following Python language constructions where TypeError exceptions may happen because of incompatible string types if you don’t get your strings in order:

* String concatenations with + operator (<code>'foo' + 'bar'</code>).
* String formatting — both “old style” (<code>'foo %s' % 'bar'</code>) and “new style” (<code>'foo {}'.format('bar')</code>).
* String joining (<code>', '.join(['foo', 'bar'])</code>).

In all those cases always make sure that you work with Unicode strings only.
Another source of subtle bugs is functions and methods that accept and return binary strings (str type) in Python 2 but Unicode strings in Python 3 (again, str type but with different properties). You can use the following utility functions to “normalize” strings that are provided to such functions/methods and received from them:

<source lang="python">
import sys

__all__ = ['PY2', 'py2_encode', 'py2_decode']

PY2 = sys.version_info[0] == 2

def py2_encode(s, encoding='utf-8'):
"""
Encode Python 2 ``unicode`` to ``str``

In Python 3 the string is not changed.
"""
if PY2 and isinstance(s, unicode):
s = s.encode(encoding)
return s

def py2_decode(s, encoding='utf-8'):
"""
Decode Python 2 ``str`` to ``unicode``

In Python 3 the string is not changed.
"""
if PY2 and isinstance(s, str):
s = s.decode(encoding)
return s
</source>

== Helper Libraries and Tools ==

=== 2to3 ===

[https://docs.python.org/2/library/2to3.html 2to3] script is created by Python developers to help converting existing Python 2 code to Python 3. On Windows it is included in Python distribution (Tools subfolder) but on other OSes you man need to install it separately. For example, on Ubuntu it is included in python-examples package. Note that this script is created for conversion from Python 2 to 3, not for writing portable code, so you need to treat its output with care.

=== Modernize ===

https://python-modernize.readthedocs.io/en/latest/ Modernize] script works on top of 2to3 and is supposed to help you to convert your existing Python 2 code to Python 2/3-compatible. However, in my experiments it did not work reliably even in simple cases so I cannot recommend it for usage.

=== Six ===

[https://pythonhosted.org/six/ Six] was the first library developed to simplify Python 2 to 3 migration. It provides a set of wrappers that hide differences between Python 2 and 3 behind its API. It is less intrusive than the following alternative because it does not monkey-patch built-in names, but in order to use Six library effectively you need to learn its API. Six can be used both for writing new Python 2/3-compatible addons and converting existing ones, but it requires good knowledge of Python 2/3 differences to pick necessary Six features that address specific differences.

Six library is included in the Kodi addon repository as '''script.module.six addon'''.

=== Future ===

[http://python-future.org/ Future] library, like Six, was created to simplify porting existing Python 2 code to 3. But it uses a different approach than Six. Future monkey-patches built-in Python objects to make Python 2 behave like Python 3. The advantage of such approach is that code written using Future is close to idiomatic Python 3 and requires little re-work if you decide to drop Python 2 support in the future. However, such approach may cause problems in some rare edge-cases.

Future library also includes <code>futurize</code> command-line utility for converting existing Python 2 code to 2/3-compatible and in my experiments this utility showed good results.
Future library (without additional utilities) is included in the Kodi addon repository as script.module.future addon.

=== PyCharm ===

PyCharm, except for being a really good Python IDE, also provides code compatibility inspection that helps you write Python 2/3-portable code. Open '''Settings''' (Alt+F7) > '''Editor''' > '''Inspections''' > '''Python''' > '''Code compatibility inspections''' and check Python version that you want to support.

PyCharm Community Edition is free and provides all features you need for creating Python addons for Kodi.

== Writing Portable Code ==

Here I’ll give you some tips about how to write portable code:

* Learn the differences between Python 2 and 3. You need to know at least the most important differences between the two major Python versions.

* Use version control system — git or mercurial — to track changes in your code. If you are porting existing code, do it in a separate branch.

* No matter if you are going to write a brand new addon or to port existing addon to Python 3, carefully choose your tools. It is totally possible to write portable code without any helper tools and libraries, but you need to know what you are doing. However, in most cases I’d recommend you to use Future library and its utilities. Carefully read Future documentation.

* Put the following line at the beginning of all your modules:
<source lang="python">
from __future__ import absolute_import, division, unicode_literals
</source>
This will enable respective Python 3 features in your code. You don’t need to import print_function (another Python 3 feature) because in Kodi addons you should to use xbmc.log() to write messages to the Kodi log file.

=== Writing New Addons ===

Check the [http://python-future.org/quickstart.html#if-you-are-writing-code-from-scratch Quick Start Guide] section of Future library documentation. You can also use Six library, but, as it was said previously, you need to learn its API to pick the necessary features to address specific Python version differences, while Future allows to write your code in (almost) idiomatic Python 3.

A brief procedure for writing new Python 2/3-compatible addons with Future library:

# Create a new virtual environment with Python 3 interpreter and activate it.
# Install Future library: <code>pip install future</code>.
# Point your IDE (Integrated Development Environment) to that environment. For example, in PyCharm: '''File''' > '''Setting''' > '''Project''' > '''Project Interpreter'''.
# Put the following line at the beginning of your Python code:
<source lang="python">
from __future__ import absolute_import, division, unicode_literals
from builtins import *
from future import standard_library
standard_library.install_aliases()
</source>
Write your addon using Python 3 syntax and standard library names.

=== Porting Existing Addons ===

A brief procedure for porting existing code to Python 3-compatible with Future library:

# Install Future library into your working Python 2 virtual environment: <code>pip install future</code>.
# Run <code>futurize</code> utility to convert all your scripts and modules to portable code.

Test your new or converted addon in Kodi with Python 2 interpreter and fix all found issues. After that test the addon in Kodi with Python 3 interpreter and again fix all found issues.
You can use [https://github.com/romanvm/kodi.web-pdb Web-PDB] debugger for troubleshooting issues in your code. It is compatible with both Python 2 and 3.

== Links ==

* Python 2 and 3 differences: https://docs.python.org/3.6/whatsnew/3.0.html
* Future library documentation: http://python-future.org/
* Six library documentation: https://pythonhosted.org/six/
* Kodi test builds with Python 3 for Windows: http://mirrors.xbmc.org/test-builds/windows/win32/
* Kodi test builds with Python 3 for Linux Ubuntu: https://launchpad.net/~wsnipex/+archive/ubuntu/kodi-python3/+packages
* “Python 3 migration” section on the Kodi official forum: https://forum.kodi.tv/forumdisplay.php?fid=281