Add-on unicode paths - Revision history

Karellen: Page outdated

2019-12-03T22:04:17Z

Page outdated

← Older revision		Revision as of 22:04, 3 December 2019
Line 1:		Line 1:
	{{mininav\| [[Add-on development]] }}		{{mininav\| [[Add-on development]] }}

			{{outdated\|This page or section has not been updated in a long time but many parts still apply up to v18. '''v19 introduces Python 3''' and this page no longer applies to v19 and later. This page may be archived at a later date.}}

	This page describes how to prevent common problems with non latin characters in Kodi or Add-on paths.		This page describes how to prevent common problems with non latin characters in Kodi or Add-on paths.

Ned Scott: xbmc -> kodi

2015-06-17T21:49:00Z

xbmc -> kodi

← Older revision		Revision as of 21:49, 17 June 2015
Line 3:		Line 3:

	== Unicode paths ==		== Unicode paths ==
	If you want to write an add-on which is able to work with paths like 'd:\apps\éîäß\' or 'opt/~~xbmc~~/àí' at first you should read http://docs.python.org/2/howto/unicode.html#python-2-x-s-unicode-support		If you want to write an add-on which is able to work with paths like 'd:\apps\éîäß\' or 'opt/Kodi/àí' at first you should read http://docs.python.org/2/howto/unicode.html#python-2-x-s-unicode-support

	After reading, you should know: "Software (Python) should only work with unicode strings internally, converting to a particular encoding on output. (or input)". To make string literals unicode by default, add		After reading, you should know: "Software (Python) should only work with unicode strings internally, converting to a particular encoding on output. (or input)". To make string literals unicode by default, add
Line 13:		Line 13:
	at the top of the module. See https://docs.python.org/2/reference/simple_stmts.html#future for details.		at the top of the module. See https://docs.python.org/2/reference/simple_stmts.html#future for details.

	~~XBMC~~ outputs UTF-8 encoded strings. Input can be unicode or UTF-8 encoded, but there are rumors that some functions don't work with unicode input parameters.		Kodi outputs UTF-8 encoded strings. Input can be unicode or UTF-8 encoded, but there are rumors that some functions don't work with unicode input parameters.

	Therefore the simplest way to deal with non-ASCII characters is to pass every parameter as UTF-8 encoded string to Kodi and to convert ~~XBMC~~'s UTF-8 output back to unicode.		Therefore the simplest way to deal with non-ASCII characters is to pass every parameter as UTF-8 encoded string to Kodi and to convert Kodi's UTF-8 output back to unicode.

	=== File functions in Python ===		=== File functions in Python ===
Line 43:		Line 43:
	path = addon.getAddonInfo('path').decode('utf-8')		path = addon.getAddonInfo('path').decode('utf-8')
	</syntaxhighlight>		</syntaxhighlight>
	~~XBMC~~'s getAddonInfo returns an UTF-8 encoded string and we decode it an unicode.		Kodi's getAddonInfo returns an UTF-8 encoded string and we decode it an unicode.
	=== Browse dialog ===		=== Browse dialog ===
	<syntaxhighlight enclose="div" lang="python">		<syntaxhighlight enclose="div" lang="python">
Line 91:		Line 91:
	'''Development:'''		'''Development:'''
	* [[Add-on development]]		* [[Add-on development]]

	{{frodo updated}}		{{frodo updated}}

NedBot: Bot: Automated text replacement (- XBMC + Kodi ); cosmetic changes

2015-05-04T08:23:27Z

Bot: Automated text replacement (- XBMC + Kodi ); cosmetic changes

← Older revision		Revision as of 08:23, 4 May 2015
Line 1:		Line 1:
	{{mininav\| [[Add-on development]] }}		{{mininav\| [[Add-on development]] }}
	This page describes how to prevent common problems with non latin characters in ~~XBMC~~ or Add-on paths.		This page describes how to prevent common problems with non latin characters in Kodi or Add-on paths.

	== Unicode paths ==		== Unicode paths ==
Line 15:		Line 15:
	XBMC outputs UTF-8 encoded strings. Input can be unicode or UTF-8 encoded, but there are rumors that some functions don't work with unicode input parameters.		XBMC outputs UTF-8 encoded strings. Input can be unicode or UTF-8 encoded, but there are rumors that some functions don't work with unicode input parameters.

	Therefore the simplest way to deal with non-ASCII characters is to pass every parameter as UTF-8 encoded string to ~~XBMC~~ and to convert XBMC's UTF-8 output back to unicode.		Therefore the simplest way to deal with non-ASCII characters is to pass every parameter as UTF-8 encoded string to Kodi and to convert XBMC's UTF-8 output back to unicode.

	=== File functions in Python ===		=== File functions in Python ===
Line 91:		Line 91:
	'''Development:'''		'''Development:'''
	* [[Add-on development]]		* [[Add-on development]]
			{{frodo updated}}

	[[Category:Add-on development]]		[[Category:Add-on development]]
	[[Category:Development]]		[[Category:Development]]
	[[Category:Python]]		[[Category:Python]]
	~~{{frodo updated}}~~

Takoi: /* Useful functions */ This goes against all the above advices. If done correctly, one should never need this. It's better to be aware of what is unicode and what is not at all times.

2014-11-11T13:10:50Z

Useful functions: This goes against all the above advices. If done correctly, one should never need this. It's better to be aware of what is unicode and what is not at all times.

← Older revision		Revision as of 13:10, 11 November 2014
Line 86:		Line 86:

	However, it's highly recommender to never mix byte strings and unicode strings in your program, in which case the 'if isinstanceof' is unnecessary.		However, it's highly recommender to never mix byte strings and unicode strings in your program, in which case the 'if isinstanceof' is unnecessary.

	~~== Useful functions ==~~
	~~=== smart_unicode and smart_utf8 ===~~
	~~Because you cannot decode an unicode or encode a string it makes sense to have a function which works with unicodes and strings:~~
	~~<syntaxhighlight enclose="div" lang="python">~~
	~~def smart_unicode(s):~~
	~~"""credit : sfaxman"""~~
	~~if not s:~~
	~~return ''~~
	~~try:~~
	~~if not isinstance(s, basestring):~~
	~~if hasattr(s, '__unicode__'):~~
	~~s = unicode(s)~~
	~~else:~~
	~~s = unicode(str(s), 'UTF-8')~~
	~~elif not isinstance(s, unicode):~~
	~~s = unicode(s, 'UTF-8')~~
	~~except:~~
	~~if not isinstance(s, basestring):~~
	~~if hasattr(s, '__unicode__'):~~
	~~s = unicode(s)~~
	~~else:~~
	~~s = unicode(str(s), 'ISO-8859-1')~~
	~~elif not isinstance(s, unicode):~~
	~~s = unicode(s, 'ISO-8859-1')~~
	~~return s~~
	~~</syntaxhighlight>~~
	~~You can use smart_unicode to ensure that the return type is an unicode!~~
	~~<syntaxhighlight enclose="div" lang="python">~~
	~~def smart_utf8(s):~~
	~~return smart_unicode(s).encode('utf-8')~~
	~~</syntaxhighlight>~~
	~~And smart_utf8 to pass parameters to XBMC.~~

	~~=== Notification ===~~
	~~<syntaxhighlight enclose="div" lang="python">~~
	~~def show_notification(title, message, timeout=2000, image=""):~~
	~~if image == "":~~
	~~command = 'Notification(%s,%s,%s)' % (smart_utf8(title), smart_utf8(message), timeout)~~
	~~else:~~
	~~command = 'Notification(%s,%s,%s,%s)' % (smart_utf8(title), smart_utf8(message), timeout, smart_utf8(image))~~
	~~xbmc.executebuiltin(command)~~
	~~</syntaxhighlight>~~
	The show_notification function uses the smart_utf8 function to ensure that every string parameter passed to XBMC is an UTF-8 encoded string. Therefore you can call show_notification with strings or unicodes!


	== See also ==		== See also ==

Takoi: /* Unicode paths */

2014-11-11T12:24:54Z

Unicode paths

← Older revision		Revision as of 12:24, 11 November 2014
Line 5:		Line 5:
	If you want to write an add-on which is able to work with paths like 'd:\apps\éîäß\' or 'opt/xbmc/àí' at first you should read http://docs.python.org/2/howto/unicode.html#python-2-x-s-unicode-support		If you want to write an add-on which is able to work with paths like 'd:\apps\éîäß\' or 'opt/xbmc/àí' at first you should read http://docs.python.org/2/howto/unicode.html#python-2-x-s-unicode-support

	After reading you know: "Software (Python) should only work with unicode strings internally, converting to a particular encoding on output. (or input)".		After reading, you should know: "Software (Python) should only work with unicode strings internally, converting to a particular encoding on output. (or input)". To make string literals unicode by default, add

			<syntaxhighlight enclose="div" lang="python">
			from __future__ import unicode_literals
			</syntaxhighlight>

			at the top of the module. See https://docs.python.org/2/reference/simple_stmts.html#future for details.

	XBMC outputs UTF-8 encoded strings. Input can be unicode or UTF-8 encoded, but there are rumors that some functions don't work with unicode input parameters.		XBMC outputs UTF-8 encoded strings. Input can be unicode or UTF-8 encoded, but there are rumors that some functions don't work with unicode input parameters.

Line 29:		Line 36:
	file function with utf-8 encoded strings		file function with utf-8 encoded strings
	</syntaxhighlight>		</syntaxhighlight>


	== Examples ==		== Examples ==

Ned Scott:

2014-09-18T11:37:12Z

← Older revision		Revision as of 11:37, 18 September 2014
Line 23:		Line 23:
	==== Conclusion ====		==== Conclusion ====
	Since your add-on should work with all supported OS, use the following approach:		Since your add-on should work with all supported OS, use the following approach:
	<~~pre~~>		<syntaxhighlight enclose="div" lang="python">
	if sys.platform.startswith('win'):		if sys.platform.startswith('win'):
	file function with unicodes		file function with unicodes
	else:		else:
	file function with utf-8 encoded strings		file function with utf-8 encoded strings
	</~~pre~~>		</syntaxhighlight>


Line 34:		Line 34:
	=== Addon path ===		=== Addon path ===
	The first path an add-on has to deal with is it's own add-on path:		The first path an add-on has to deal with is it's own add-on path:
			<syntaxhighlight enclose="div" lang="python">
	path = addon.getAddonInfo('path').decode('utf-8')		path = addon.getAddonInfo('path').decode('utf-8')
			</syntaxhighlight>
	XBMC's getAddonInfo returns an UTF-8 encoded string and we decode it an unicode.		XBMC's getAddonInfo returns an UTF-8 encoded string and we decode it an unicode.
	=== Browse dialog ===		=== Browse dialog ===
			<syntaxhighlight enclose="div" lang="python">
	dialog = xbmcgui.Dialog()		dialog = xbmcgui.Dialog()
	directory = dialog.browse(0, 'Title' , 'pictures').decode('utf-8')		directory = dialog.browse(0, 'Title' , 'pictures').decode('utf-8')
			</syntaxhighlight>
	dialog.browse() returns an UTF-8 encoded string which perhaps contains some non latin characters. Therefore decode it to unicode!		dialog.browse() returns an UTF-8 encoded string which perhaps contains some non latin characters. Therefore decode it to unicode!
	=== Path joins ===		=== Path joins ===
			<syntaxhighlight enclose="div" lang="python">
	os.path.join(path, filename)		os.path.join(path, filename)
			</syntaxhighlight>
	If path and filename are unicodes then everthing will work as expected.		If path and filename are unicodes then everthing will work as expected.
	But what will happen if filename is an UTF-8 encoded string which contains "öäü.jpg"?		But what will happen if filename is an UTF-8 encoded string which contains "öäü.jpg"?

	Python always uses unicodes to join a string with an unicode. Therefore Python will decode the string with it's default encoding (ascii).		Python always uses unicodes to join a string with an unicode. Therefore Python will decode the string with it's default encoding (ascii).
			<syntaxhighlight enclose="div" lang="python">
	os.path.join(path, filename.decode('ascii'))		os.path.join(path, filename.decode('ascii'))
			</syntaxhighlight>
	Due to the missing öäü within the ASCII codepage you'll get an unicode exception! That's the reason why you must explicitly convert the string to unicode!		Due to the missing öäü within the ASCII codepage you'll get an unicode exception! That's the reason why you must explicitly convert the string to unicode!
			<syntaxhighlight enclose="div" lang="python">
	os.path.join(path, filename.decode('utf-8'))		os.path.join(path, filename.decode('utf-8'))
			</syntaxhighlight>

	=== Logging ===		=== Logging ===
	"print" and xbmc.log does not support unicode. Always encode unicode strings to utf-8.		"print" and xbmc.log does not support unicode. Always encode unicode strings to utf-8.

	<~~pre~~>		<syntaxhighlight enclose="div" lang="python">
	print message.encode('utf-8')		print message.encode('utf-8')
	</~~pre~~>		</syntaxhighlight>

	Alternatively, the following function can be used, where msg can be everything from string to unicode to class:		Alternatively, the following function can be used, where msg can be everything from string to unicode to class:

	<~~pre~~>		<syntaxhighlight enclose="div" lang="python">
	def log(msg, level=xbmc.LOGDEBUG):		def log(msg, level=xbmc.LOGDEBUG):
	plugin = "My nice plugin"		plugin = "My nice plugin"
Line 67:		Line 77:

	xbmc.log("[%s] %s" % (plugin, msg.__str__()), level)		xbmc.log("[%s] %s" % (plugin, msg.__str__()), level)
	</~~pre~~>		</syntaxhighlight>

	However, it's highly recommender to never mix byte strings and unicode strings in your program, in which case the 'if isinstanceof' is unnecessary.		However, it's highly recommender to never mix byte strings and unicode strings in your program, in which case the 'if isinstanceof' is unnecessary.
Line 105:		Line 115:

	=== Notification ===		=== Notification ===
	<~~pre~~>		<syntaxhighlight enclose="div" lang="python">
	def show_notification(title, message, timeout=2000, image=""):		def show_notification(title, message, timeout=2000, image=""):
	if image == "":		if image == "":
Line 112:		Line 122:
	command = 'Notification(%s,%s,%s,%s)' % (smart_utf8(title), smart_utf8(message), timeout, smart_utf8(image))		command = 'Notification(%s,%s,%s,%s)' % (smart_utf8(title), smart_utf8(message), timeout, smart_utf8(image))
	xbmc.executebuiltin(command)		xbmc.executebuiltin(command)
	</~~pre~~>		</syntaxhighlight>
	The show_notification function uses the smart_utf8 function to ensure that every string parameter passed to XBMC is an UTF-8 encoded string. Therefore you can call show_notification with strings or unicodes!		The show_notification function uses the smart_utf8 function to ensure that every string parameter passed to XBMC is an UTF-8 encoded string. Therefore you can call show_notification with strings or unicodes!

Ned Scott: /* smart_unicode and smart_utf8 */

2014-09-18T11:34:30Z

smart_unicode and smart_utf8: <syntaxhighlight>

← Older revision		Revision as of 11:34, 18 September 2014
Line 74:		Line 74:
	=== smart_unicode and smart_utf8 ===		=== smart_unicode and smart_utf8 ===
	Because you cannot decode an unicode or encode a string it makes sense to have a function which works with unicodes and strings:		Because you cannot decode an unicode or encode a string it makes sense to have a function which works with unicodes and strings:
	<~~pre~~>		<syntaxhighlight enclose="div" lang="python">
	def smart_unicode(s):		def smart_unicode(s):
	"""credit : sfaxman"""		"""credit : sfaxman"""
Line 96:		Line 96:
	s = unicode(s, 'ISO-8859-1')		s = unicode(s, 'ISO-8859-1')
	return s		return s
	</~~pre~~>		</syntaxhighlight>
	You can use smart_unicode to ensure that the return type is an unicode!		You can use smart_unicode to ensure that the return type is an unicode!
	<~~pre~~>		<syntaxhighlight enclose="div" lang="python">
	def smart_utf8(s):		def smart_utf8(s):
	return smart_unicode(s).encode('utf-8')		return smart_unicode(s).encode('utf-8')
	</~~pre~~>		</syntaxhighlight>
	And smart_utf8 to pass parameters to XBMC.		And smart_utf8 to pass parameters to XBMC.

Takoi: move ' file functions' under 'unicode paths'. rewrite openelec (linux) section

2014-09-18T09:57:54Z

move ' file functions' under 'unicode paths'. rewrite openelec (linux) section

@@ Line 9: / Line 9: @@
 Therefore the simplest way to deal with non-ASCII characters is to pass every parameter as UTF-8 encoded string to XBMC and to convert XBMC's UTF-8 output back to unicode.
 == Examples ==
@@ Line 93: / Line 114: @@
 </pre>
 The show_notification function uses the smart_utf8 function to ensure that every string parameter passed to XBMC is an UTF-8 encoded string. Therefore you can call show_notification with strings or unicodes!
 == See also ==

Takoi: /* Conclusion */ catching everything is bad idea. Also, a lot of file function doest raise exception

2014-09-18T09:41:20Z

Conclusion: catching everything is bad idea. Also, a lot of file function doest raise exception

← Older revision		Revision as of 09:41, 18 September 2014
Line 106:		Line 106:
	Since your add-on should work with all supported OS, use the following approach:		Since your add-on should work with all supported OS, use the following approach:
	<pre>		<pre>
	~~try~~:		if sys.platform.startswith('win'):
	file function with unicodes		file function with unicodes
	~~except~~:		else:
	~~try:~~		file function with utf-8 encoded strings
	file function with utf-8 encoded strings
	~~except:~~
	~~fatal error~~
	</pre>		</pre>

Takoi: merge logging and logging

2014-09-18T09:30:48Z

merge logging and logging

@@ Line 28: / Line 28: @@
 Due to the missing öäü within the ASCII codepage you'll get an unicode exception! That's the reason why you must explicitly convert the string to unicode!
   os.path.join(path, filename.decode('utf-8'))
 === Logging ===
-Don't use "print message" because if message contains non latin character you'll get an unicode exception.
+"print" and xbmc.log does not support unicode. Always encode unicode strings to utf-8.
-Instead:
- print message.encode('utf-8')
+<pre>
-which requires that message must be a unicode!
+    print message.encode('utf-8')
 == Useful functions ==
 === smart_unicode and smart_utf8 ===
@@ Line 65: / Line 82: @@
 </pre>
 And smart_utf8 to pass parameters to XBMC.
 === Notification ===
 <pre>