Official Kodi Wiki - User contributions [en]

Archive:Debug Python Scripts with Eclipse

2015-12-29T10:04:36Z

Calebzor: /* Identify where PyDev remote debugger Python files are */

{{mininav|[[Development]]|[[Add-on development]]|[[Python development]]}}

Here is a method you can adopt in order to use the Eclipse (multi-platforms) remote debugger with your Python scripts/plugin.

Note: Follow this link [[HOW-TO debug Python Scripts|Debugging Python Scripts on Linux with WinPDB]] for more information if you prefer to use WinPDB instead.

Note: The information provided herein is also applicable to user with [http://www.aptana.com/ Aptana Studio 3] installation which come pre-installed with pydev. For this case, you can skip to [[#Configure Pydev and Eclipse|Configure Pydev and Eclipse]] after the Aptana Studio 3 installation.

== Prerequisite ==
* Any platform able to run [http://www.eclipse.org/downloads/ Eclipse IDE] (Windows 32bit-64bit, Mac Carbon 32bit-64bit, Mac Cocoa 32bit-64bit, Linux 32bit-64bit): it is the platform from where you will run the debugger IDE
* Python interpreter 2.4 (use the same version as XBMC or version <= 2.7 ) install on the machine running Eclipse
* Any platform running XBMC (Xbox, Linux, MACOSX, Windows): it is the platform running Python scripts/plugin connecting with the remote debugger.
:XBMC installed on Ubuntu (should work just fine on Windows and Mac too)
* [http://pydev.org/ Pydev] plugin for Eclipse IDE: Pydev is a Python IDE for Eclipse, which may be used in Python, Jython and IronPython development.

== Prepare and Setup Eclipse ==

=== Install Eclipse ===
* Download [http://www.eclipse.org/downloads/ Eclipse IDE] and install it
Note: in our example Eclipse Galileo (3.5.1) is used

=== Install Pydev in Eclipse ===
* In Eclipse go in: ''Help'' -> ''Install New Software...''
* Click on the ''Add'' button and add Pydev website as a source: http://pydev.org/updates
* You will see the window below:
[[File:EclipsePydevUpdate.png]]
* Check ''PyDev'' box and follow the steps in order to install PyDev
Note: in our example PyDev 1.5.5

=== Configure Pydev and Eclipse ===

==== Add Python Interpreter Path ====
* Go in ''Window'' -> ''Preferences''
* ''Pydev'' then ''Interpreter - Python''
* Click on button ''New...''
* In the new prompt, enter the ''Interpreter Name: Python'', and Browse for your Python Interpreter 2.4 ''python.exe'' file and select it. After that you should see something like this:
[[File:EclipsePydevPref.png|none]]
This will help in order to have Python code completion (except for XBMC modules)

==== Add Pydev view to Eclipse ====
* On the top right corner click on the View button on select ''Others''
[[File:EclipsePydevViewMenu.png]]
* Select Pydev then OK, a PyDev View Button should be now available on the top right corner
[[File:EclipsePydevViewMenu2.png]]
Use this view for your developments in Python for XBMC

==== Add Pydev Start/Stop debug server buttons ====
* In the menu bar select ''Window'' -> ''Customize perspective...'', a window ''Customize Perspective - PyDev'' will open
* Select ''Command Groups Availability'' Tab
* Check ''Pydev Debug'' box and OK
* You should see 2 new buttons in your toolbar:
[[File:EclipsePydevDebugSrvButtons.png]]

==== Setting the remote debugger port (optional) ====
If you do not want to use the default port (usually 5678) you can set it up:
* Go to ''Window'' -> ''Preferences''
* ''Pydev'' then ''Debug'' you can define here the port and the timeout you want

==== Setting up Predefined Completions (optional) ====
Many developers are accustom to using predefined completions when writing their code (eg. type-ahead). The standard Python completions are enabled by default when using these instructions, but the standard XBMC components will not be recognized appropriately (xbmc, xbmcgui, xbmcplugin, etc).

In order to overcome this, you can run the [http://forum.kodi.tv/showthread.php?t=79070 PyDev Predefined Completions Creator addon] and [http://pydev.org/manual_101_interpreter.html#PyDevInterpreterConfiguration-PredefinedCompletions setup Eclipse] to reference this file:
# Download the 'PyDev Predefined Completions Creator' Addon (eg. System > Settings > Addons > XBMC.org > Program Add-Ons)
# Navigate to the newly installed Addon (eg. Programs from the home screen)
# Open the context menu for newly installed addon and select 'Add-on Settings'
# Set the location to save the help docs to and save the new settings (This can be anywhere on your system - we'll configure it in Eclipse later)
# Select the add-on to run it (the addon should run and close quickly)
# Open Eclipse and select Window > Preferences > PyDev > Interpreter - Python
# From the 'Predefined' tab, click the new button and select the folder you created in the step above
# Press Apply / OK to ensure the changes are saved

== Create a Python script/plugin for XBMC in Eclipse ==

=== Create a new PyDev Project ===
Most of the time when you are doing development, likely that you are running XBMC and Eclipse on the same platform; (The installed debugger described in this article actually can also work with a remote XBMC).<br\>
Let's say you already have scripts installed in XBMC script folder that you want to create a new PyDev project e.g.
<pre>
C:\XBMC\addons\myscript1
C:\XBMC\addons\myscript2
</pre>

With myscript1 & myscript2 being sub-directories of ''C:\XBMC\addons'' each containing the Default.py scripts.<br\>
First you need to define the Python project workspace:
* ''File'' -> ''Switch Workspace'' -> ''Other...''
* ''Browse'' to the parent directory of your script module i.e. ''C:\XBMC\addons'' in this example, select and OK
Then proceed to create the Python project:
* ''File'' -> ''New'' -> ''PyDev Project''
: OR select below in case ''PyDev Project'' is not displayed
* ''File'' -> ''New'' -> ''Others''; and Select ''Pydev'' then ''PyDev Project''; Click ''Next''
* Enter the name of your project e.g. xbmc_addons
* Uncheck option ''Use Default''
* Browse to the directory of your scripts and select i.e. C:\XBMC\addons.
* Set ''Python' as Project Type
* Select ''2.4 (XBMC Python version)'' for the grammar
* Select ''Don't configure PYTHONPATH (to be done manually later on)''
* Click on ''Finish'' button

=== Set the Python Path ===
* Right click on your new project and select ''Properties''
* Select ''PyDev - PYTHONPATH'' and click on ''Add source folder''
* Select the source directory of your script, /xbmc_addons in our example

=== Import File System ===
Assuming you want to add in all the directories under C:\XBMC\addons
* Right click on your new project and select ''Import...''
* Select ''General'' and click on ''File System''; then ''Next''
* Browse to the source directory of your script (or plugin) e.g. C:\XBMC\addons
* Check the directory and files to be added
* Select ''Into folder: xbmc_addons''
* Select ''Options | <<Advanced | Create links in workspace''
* Uncheck both the options i.e. ''Create virtual folders'' & ''Create link locations relative to;...''
* Click on ''Finish'' button
Your source code is now recognized by Eclipse and you will have access to neat functionality such as code completion, link between source files, etc.

=== Add Pydev Python source code (pysrc) to XBMC ===
XBMC needs to know where to find the PyDev remote python source file in order to communicate with the remote debugger of Eclipse.

==== Identify where PyDev remote debugger Python files are ====
In later versions, you need to have started the debug server from Eclipse at least once for the following folder to show up.

Python source files should be in you Eclipse installation directory:
<pre>C:\...\eclipse\plugins\org.python.pydev.debug_1.5.5.2010030420\pysrc\</pre>
In our case we use PyDev 1.5.5. With newer versions of PyDev, the directory structure changed.
<pre>C:\...\eclipse\plugins\org.python.pydev_3.6.0.201406232321\pysrc\</pre>
Look for org.python.pydev.debug (not core)

Instructions for Linux users:

{{Collapse top}}
In new Eclipse versions on Linux, the standard eclipse installation (/usr/share/eclipse/plugins) does not contain the pydev files any more.
The easiest solution is to install the [[Add-on:Pydevd|pydevd-addon]].<br />
To access this from your addon, your addon.xml has to contain <code><import addon="script.module.pydevd" version="3.4.1"/></code>.
{{Collapse bottom}}

==== Modify PyDev source in order to support XBMC paths ====
The pysrc code that came with Eclipse-pydev does not support XBMC paths (special://... or Q:/...) and the module would not find them making it impossible to insert breakpoints in such files.

In order to add the support of XBMC paths, you will need to do a simple modification to the file ''pydevd_file_utils.py'' consisting on adding a line to each procedure that had a filename parameter (4 procedures in total).
Add a line like

def _NormFile(filename):
filename = xbmc.translatePath(filename)
try:
...

The other three procedures are:
* def _NormFile(filename):
* def NormFileToServer(filename):
* def NormFileToClient(filename):

of course you need the import directive at the top of the file:

import xbmc

at the beginning of the module...

==== Choose how XBMC will refer to Pydev debugging module ====

You have 3 options as below:
* Option 1 - Refer from our python script directly the directory where pysrc is installed
* Option 2 - Copy pysrc in our script directory and refer it from our script
* Option 3 - Copy pysrc in XBMC Python install
* Option 4 - Use the pydev from the [[Add-on:Pydevd|pydevd-addon]]

I am only going to show you the steps for ''Option 3'':

==== Copy pysrc in XBMC Python install ====
===== Option 3: (Suggested for Windows users) =====
* Copy the content of pysrc to:
<pre>XBMC\system\python\Lib\pysrc\</pre>
* Add an empty __init__.py file in it

===== Option 4: (Suggested for Linux users) =====
Use the [[Add-on:Pydevd|pydevd-addon]] as described above. No action necessary. Option 3 does not work on Linux, because /usr/share/kodi/system/python/Lib is ignored.

=== Add to your python script the code for remote debug ===
At the beginning of you Python script/plugin (usually Default.py) add the following code:
<pre>
REMOTE_DBG = True

# append pydev remote debugger
if REMOTE_DBG:
# Make pydev debugger works for auto reload.
# Note pydevd module need to be copied in XBMC\system\python\Lib\pysrc
try:
import pysrc.pydevd as pydevd # with the addon script.module.pydevd, only use `import pydevd`
# stdoutToServer and stderrToServer redirect stdout and stderr to eclipse console
pydevd.settrace('localhost', stdoutToServer=True, stderrToServer=True)
except ImportError:
sys.stderr.write("Error: " +
"You must add org.python.pydev.debug.pysrc to your PYTHONPATH.")
sys.exit(1)
</pre>

If this doesn't work and you are still getting the PYTHONPATH error you can try to:

Replace:

<pre>
import pysrc.pydevd as pydevd
</pre>

in the code snippet above with:

<pre>
import sys
sys.path.append('full_path_to_pysrc')
import pydevd
</pre>

In the case you have changed the default port you can replace the line with pydevd.settrace by this one:
<pre>
pydevd.settrace('localhost', port=5678, stdoutToServer=True, stderrToServer=True)
</pre>

* Use REMOTE_DBG in order to activate or not the remote debugging
* port specifies the listening port number use between the client and the server
* Replace localhost by an IP address if XBMC and Eclipse are on 2 different machines
* stdoutToServer and stderrToServer redirect stdout and stderr to eclipse console, you won't have to open xbmc.log file

=== Run ===
* Go in Debug view (Eclipse: [[File:changeperspective.png]] / Debug ) and press on the start server button: [[File:EclipsePydevDebugSrvButtons.png]]

* You will see in the console logs like:
<pre>Debug Server at port: 5678</pre>

* Start XBMC and run your script/plugin, you will see it as "Running" but paused

* In Eclipse (Debug View) click on the Resume button (or F8), then you will see the program running in XBMC

* You are now fully set up: You can set break-point, watch on variables, run step by step etc

* Enjoy!

=== Screenshot ===

[[File:EclipsePydevDebugger.png]]

== See also ==
* [[Add-ons]]
* [[:Category:All add-ons]]
* [[3rd party add-on repositories]]
'''Development:'''
* [[Add-on development]]
* [[Addon Settings]]
* [[Python development]]
* [[Add-on repositories]]
* [[Official add-on repository]]

[[Category:Python]]
[[Category:How-to]]
[[Category:Add-on development]]

Archive:Debug Python Scripts with Eclipse

2015-12-29T09:59:51Z

Calebzor: /* Add to your python script the code for remote debug */

{{mininav|[[Development]]|[[Add-on development]]|[[Python development]]}}

Here is a method you can adopt in order to use the Eclipse (multi-platforms) remote debugger with your Python scripts/plugin.

Note: Follow this link [[HOW-TO debug Python Scripts|Debugging Python Scripts on Linux with WinPDB]] for more information if you prefer to use WinPDB instead.

Note: The information provided herein is also applicable to user with [http://www.aptana.com/ Aptana Studio 3] installation which come pre-installed with pydev. For this case, you can skip to [[#Configure Pydev and Eclipse|Configure Pydev and Eclipse]] after the Aptana Studio 3 installation.

== Prerequisite ==
* Any platform able to run [http://www.eclipse.org/downloads/ Eclipse IDE] (Windows 32bit-64bit, Mac Carbon 32bit-64bit, Mac Cocoa 32bit-64bit, Linux 32bit-64bit): it is the platform from where you will run the debugger IDE
* Python interpreter 2.4 (use the same version as XBMC or version <= 2.7 ) install on the machine running Eclipse
* Any platform running XBMC (Xbox, Linux, MACOSX, Windows): it is the platform running Python scripts/plugin connecting with the remote debugger.
:XBMC installed on Ubuntu (should work just fine on Windows and Mac too)
* [http://pydev.org/ Pydev] plugin for Eclipse IDE: Pydev is a Python IDE for Eclipse, which may be used in Python, Jython and IronPython development.

== Prepare and Setup Eclipse ==

=== Install Eclipse ===
* Download [http://www.eclipse.org/downloads/ Eclipse IDE] and install it
Note: in our example Eclipse Galileo (3.5.1) is used

=== Install Pydev in Eclipse ===
* In Eclipse go in: ''Help'' -> ''Install New Software...''
* Click on the ''Add'' button and add Pydev website as a source: http://pydev.org/updates
* You will see the window below:
[[File:EclipsePydevUpdate.png]]
* Check ''PyDev'' box and follow the steps in order to install PyDev
Note: in our example PyDev 1.5.5

=== Configure Pydev and Eclipse ===

==== Add Python Interpreter Path ====
* Go in ''Window'' -> ''Preferences''
* ''Pydev'' then ''Interpreter - Python''
* Click on button ''New...''
* In the new prompt, enter the ''Interpreter Name: Python'', and Browse for your Python Interpreter 2.4 ''python.exe'' file and select it. After that you should see something like this:
[[File:EclipsePydevPref.png|none]]
This will help in order to have Python code completion (except for XBMC modules)

==== Add Pydev view to Eclipse ====
* On the top right corner click on the View button on select ''Others''
[[File:EclipsePydevViewMenu.png]]
* Select Pydev then OK, a PyDev View Button should be now available on the top right corner
[[File:EclipsePydevViewMenu2.png]]
Use this view for your developments in Python for XBMC

==== Add Pydev Start/Stop debug server buttons ====
* In the menu bar select ''Window'' -> ''Customize perspective...'', a window ''Customize Perspective - PyDev'' will open
* Select ''Command Groups Availability'' Tab
* Check ''Pydev Debug'' box and OK
* You should see 2 new buttons in your toolbar:
[[File:EclipsePydevDebugSrvButtons.png]]

==== Setting the remote debugger port (optional) ====
If you do not want to use the default port (usually 5678) you can set it up:
* Go to ''Window'' -> ''Preferences''
* ''Pydev'' then ''Debug'' you can define here the port and the timeout you want

==== Setting up Predefined Completions (optional) ====
Many developers are accustom to using predefined completions when writing their code (eg. type-ahead). The standard Python completions are enabled by default when using these instructions, but the standard XBMC components will not be recognized appropriately (xbmc, xbmcgui, xbmcplugin, etc).

In order to overcome this, you can run the [http://forum.kodi.tv/showthread.php?t=79070 PyDev Predefined Completions Creator addon] and [http://pydev.org/manual_101_interpreter.html#PyDevInterpreterConfiguration-PredefinedCompletions setup Eclipse] to reference this file:
# Download the 'PyDev Predefined Completions Creator' Addon (eg. System > Settings > Addons > XBMC.org > Program Add-Ons)
# Navigate to the newly installed Addon (eg. Programs from the home screen)
# Open the context menu for newly installed addon and select 'Add-on Settings'
# Set the location to save the help docs to and save the new settings (This can be anywhere on your system - we'll configure it in Eclipse later)
# Select the add-on to run it (the addon should run and close quickly)
# Open Eclipse and select Window > Preferences > PyDev > Interpreter - Python
# From the 'Predefined' tab, click the new button and select the folder you created in the step above
# Press Apply / OK to ensure the changes are saved

== Create a Python script/plugin for XBMC in Eclipse ==

=== Create a new PyDev Project ===
Most of the time when you are doing development, likely that you are running XBMC and Eclipse on the same platform; (The installed debugger described in this article actually can also work with a remote XBMC).<br\>
Let's say you already have scripts installed in XBMC script folder that you want to create a new PyDev project e.g.
<pre>
C:\XBMC\addons\myscript1
C:\XBMC\addons\myscript2
</pre>

With myscript1 & myscript2 being sub-directories of ''C:\XBMC\addons'' each containing the Default.py scripts.<br\>
First you need to define the Python project workspace:
* ''File'' -> ''Switch Workspace'' -> ''Other...''
* ''Browse'' to the parent directory of your script module i.e. ''C:\XBMC\addons'' in this example, select and OK
Then proceed to create the Python project:
* ''File'' -> ''New'' -> ''PyDev Project''
: OR select below in case ''PyDev Project'' is not displayed
* ''File'' -> ''New'' -> ''Others''; and Select ''Pydev'' then ''PyDev Project''; Click ''Next''
* Enter the name of your project e.g. xbmc_addons
* Uncheck option ''Use Default''
* Browse to the directory of your scripts and select i.e. C:\XBMC\addons.
* Set ''Python' as Project Type
* Select ''2.4 (XBMC Python version)'' for the grammar
* Select ''Don't configure PYTHONPATH (to be done manually later on)''
* Click on ''Finish'' button

=== Set the Python Path ===
* Right click on your new project and select ''Properties''
* Select ''PyDev - PYTHONPATH'' and click on ''Add source folder''
* Select the source directory of your script, /xbmc_addons in our example

=== Import File System ===
Assuming you want to add in all the directories under C:\XBMC\addons
* Right click on your new project and select ''Import...''
* Select ''General'' and click on ''File System''; then ''Next''
* Browse to the source directory of your script (or plugin) e.g. C:\XBMC\addons
* Check the directory and files to be added
* Select ''Into folder: xbmc_addons''
* Select ''Options | <<Advanced | Create links in workspace''
* Uncheck both the options i.e. ''Create virtual folders'' & ''Create link locations relative to;...''
* Click on ''Finish'' button
Your source code is now recognized by Eclipse and you will have access to neat functionality such as code completion, link between source files, etc.

=== Add Pydev Python source code (pysrc) to XBMC ===
XBMC needs to know where to find the PyDev remote python source file in order to communicate with the remote debugger of Eclipse.

==== Identify where PyDev remote debugger Python files are ====
Python source files should be in you Eclipse installation directory:
<pre>C:\...\eclipse\plugins\org.python.pydev.debug_1.5.5.2010030420\pysrc\</pre>
In our case we use PyDev 1.5.5. With newer versions of PyDev, the directory structure changed.
<pre>C:\...\eclipse\plugins\org.python.pydev_3.6.0.201406232321\pysrc\</pre>
Look for org.python.pydev.debug (not core)

Instructions for Linux users:

{{Collapse top}}
In new Eclipse versions on Linux, the standard eclipse installation (/usr/share/eclipse/plugins) does not contain the pydev files any more.
The easiest solution is to install the [[Add-on:Pydevd|pydevd-addon]].<br />
To access this from your addon, your addon.xml has to contain <code><import addon="script.module.pydevd" version="3.4.1"/></code>.
{{Collapse bottom}}

==== Modify PyDev source in order to support XBMC paths ====
The pysrc code that came with Eclipse-pydev does not support XBMC paths (special://... or Q:/...) and the module would not find them making it impossible to insert breakpoints in such files.

In order to add the support of XBMC paths, you will need to do a simple modification to the file ''pydevd_file_utils.py'' consisting on adding a line to each procedure that had a filename parameter (4 procedures in total).
Add a line like

def _NormFile(filename):
filename = xbmc.translatePath(filename)
try:
...

The other three procedures are:
* def _NormFile(filename):
* def NormFileToServer(filename):
* def NormFileToClient(filename):

of course you need the import directive at the top of the file:

import xbmc

at the beginning of the module...

==== Choose how XBMC will refer to Pydev debugging module ====

You have 3 options as below:
* Option 1 - Refer from our python script directly the directory where pysrc is installed
* Option 2 - Copy pysrc in our script directory and refer it from our script
* Option 3 - Copy pysrc in XBMC Python install
* Option 4 - Use the pydev from the [[Add-on:Pydevd|pydevd-addon]]

I am only going to show you the steps for ''Option 3'':

==== Copy pysrc in XBMC Python install ====
===== Option 3: (Suggested for Windows users) =====
* Copy the content of pysrc to:
<pre>XBMC\system\python\Lib\pysrc\</pre>
* Add an empty __init__.py file in it

===== Option 4: (Suggested for Linux users) =====
Use the [[Add-on:Pydevd|pydevd-addon]] as described above. No action necessary. Option 3 does not work on Linux, because /usr/share/kodi/system/python/Lib is ignored.

=== Add to your python script the code for remote debug ===
At the beginning of you Python script/plugin (usually Default.py) add the following code:
<pre>
REMOTE_DBG = True

# append pydev remote debugger
if REMOTE_DBG:
# Make pydev debugger works for auto reload.
# Note pydevd module need to be copied in XBMC\system\python\Lib\pysrc
try:
import pysrc.pydevd as pydevd # with the addon script.module.pydevd, only use `import pydevd`
# stdoutToServer and stderrToServer redirect stdout and stderr to eclipse console
pydevd.settrace('localhost', stdoutToServer=True, stderrToServer=True)
except ImportError:
sys.stderr.write("Error: " +
"You must add org.python.pydev.debug.pysrc to your PYTHONPATH.")
sys.exit(1)
</pre>

If this doesn't work and you are still getting the PYTHONPATH error you can try to:

Replace:

<pre>
import pysrc.pydevd as pydevd
</pre>

in the code snippet above with:

<pre>
import sys
sys.path.append('full_path_to_pysrc')
import pydevd
</pre>

In the case you have changed the default port you can replace the line with pydevd.settrace by this one:
<pre>
pydevd.settrace('localhost', port=5678, stdoutToServer=True, stderrToServer=True)
</pre>

* Use REMOTE_DBG in order to activate or not the remote debugging
* port specifies the listening port number use between the client and the server
* Replace localhost by an IP address if XBMC and Eclipse are on 2 different machines
* stdoutToServer and stderrToServer redirect stdout and stderr to eclipse console, you won't have to open xbmc.log file

=== Run ===
* Go in Debug view (Eclipse: [[File:changeperspective.png]] / Debug ) and press on the start server button: [[File:EclipsePydevDebugSrvButtons.png]]

* You will see in the console logs like:
<pre>Debug Server at port: 5678</pre>

* Start XBMC and run your script/plugin, you will see it as "Running" but paused

* In Eclipse (Debug View) click on the Resume button (or F8), then you will see the program running in XBMC

* You are now fully set up: You can set break-point, watch on variables, run step by step etc

* Enjoy!

=== Screenshot ===

[[File:EclipsePydevDebugger.png]]

== See also ==
* [[Add-ons]]
* [[:Category:All add-ons]]
* [[3rd party add-on repositories]]
'''Development:'''
* [[Add-on development]]
* [[Addon Settings]]
* [[Python development]]
* [[Add-on repositories]]
* [[Official add-on repository]]

[[Category:Python]]
[[Category:How-to]]
[[Category:Add-on development]]