Archive:Optimize Load Time of Plugins with Persistence Scripts

Plugins are powerful and easy to write, but that convenience can come with a severe performance cost, particularly in the form of long load times between menus. These load times occur because plugins lack persistence. Every time a user launches a plugin or chooses a menu entry within the plugin, XBMC runs the plugin from scratch, draws the new menu, and then closes the script. Global or class variables used throughout the plugin (often on multiple screens) have to be re-initialized over and over again.

To take full advantage of the plugin design, you should optimize your plugin with this in mind, loading only the information you need on each pass. In some cases (such as plugins that rely on large custom databases), your plugin will require a repeated initialization that costs several seconds every time the user selects a menu item. In such cases, you can significantly speed up navigation by running a Persistence server script in the background, and accessing it from the plugin when you need it.

Overview

This tutorial will teach you how to use a third-party script, Persistence, developed by Alexpoet as a utility tool. The tutorial is divided into two major sections, Theory and Implementation. The Theory section explains how Persistence works, in case you want to customize it for your personal uses or develop your own custom tool.

If you just want to use Persistence to optimize your own script, you can skip ahead to the Implementation section which explains how to install the server and how to configure your plugin or script as a Persistence client.

Theory

Persistence is a server script that runs in the XBMC background. Any script or plugin that is configured as a Persistence client can create and register a class instance with the server. The server will hold that class instance in memory, and provide your clients with instant access to it.

The comm Module

To do this, the Persistence server establishes a socket connection and sits in a continuous loop, listening for client requests on a specified port (the default value is 51999).

Persistence uses a module named comm.py to handle client/server communication. The entire contents of the comm module (without comments) look like this:

import os, cPickle
from struct import *
from socket import *

def send(sock, data):
    data = cPickle.dumps(data)
    datasize = len(data)
    sock.send(pack("<i", datasize))
    sock.send(data)

def recv(sock):
    datasize = unpack("<i", sock.recv(4))[0]
    pick = cPickle.loads(sock.recv(datasize))
    return pick

These two functions (available on both the client and server side) uniformly convert all Python objects into strings using the cPickle library, then transfer them over an established socket connection and convert them back into Python objects.

NOTE: The comm module simplifies client/server interactions and enhances the readability of code. Since it's a separate module, though, comm.py must be included in both client and server folders. If you want to simplify your file management, you can add these two functions into Persistence.py and ClassHandler.py, a module you'll include in your plugin folder.

Sample Code: Unpersist.py

You can see exactly how to implement the comm module by looking at Unpersist.py, a script included in the Persistence folder. Its contents look like this:

import comm
from socket import *

commport = 51999

host = gethostname()
sock = socket(AF_INET, SOCK_STREAM)
sock.connect((host, commport))
comm.send(sock, "break")

This script is a utility client, and about the simplest Persistence client you could build. It imports the custom comm module, and a handful of globals from the socket module -- a built-in that allows interaction over comm ports.

Then it creates a socket connection, sock, and passes it to comm.send, along with the Python object it wants to send. In this case, the Python object is a simple string, the command "break," which interrupts the Persistence server's loop. In seven lines, Unpersist.py connects to the Persistence server, sends the kill command, and then exits.

Of course, the code for a functioning client will be more complicated.

The Client/Server Architecture

The Persistence server acts as a background container for Python classes. A Persistence client uses the ClassHandler module to fetch and maintain a local copy of the class instance from the server.

For example, consider a plugin that displays queries from a massive SQL database. A plugin that imports the SQLite module and then reads in even a medium-sized SQL database from a file can take 5-10 seconds on a PC, or significantly longer on an XBox.

To speed that up you could make a class:

class SQLHolder:
    def __init__(self):
        self.db = sqlite2.connect(filename, detect_types=sqlite2.PARSE_DECLTYPES)

When the class is first created it loads the database which will still take 5-10 seconds. But your plugin can save the initialized class like so:

sqlHolder = SQLHolder()
handler = ClassHandler.ClassHandler()
handler.save(sqlHolder)

That last line registers the class instance with ClassHandler, which then sends a copy of it to the Persistence server. You can then browse up and down the plugin's folder structure without ever waiting for the database to load again.

ClassHandler provides four useful functions for handling your persistent classes:

    def exists(self, classDefinition):

Provides a class definition (like SQLHolder above), and ClassHandler will return an existing instance of it if it exists, and None if it doesn't.

    def new(self, classDefinition, initArgs):

Creates a new instance of a class, immediately registering it with ClassHandler and with the server. Functionally equivalent to creating a class instance locally and then calling ClassHandler.save.

    def update(self, classInstance):

Updates a class instance from the server. Useful in longer-running scripts to verify that the local instance is up-to-date with the server instance. Returns the current server instance (just like classHandler.exists), but requires a local instance as a parameter (rather than the class definition).

    def save(self, classInstance):

Commits the local instance of a class to the server. This should be called immediately after making any changes to a class instance registered with ClassHandler.