Porting GLib Applications to Python
So part of a project I am doing for my university involves writing python bindings for a library called GUPnP. GUPnP is a UPnP framework written by Zeeshan Ali. It is written using GLib and GObject and uses the GLib based Libsoup http backend.On day one Zeeshan pointed me to GObject Introspection (GIR). GIR is this really awesome peace of work that is able to scan c code and generate what is essentially a compressed api/marshalling definitition file. There are then libraries written for several languages which understand these files (called 'typelibs') and are capable of marshalling data back and fourth between languages. Until recently the only option for doing this translation was a project called PyBank. PyBank has recently been merged with PyGObject which provides GObject support and also GIR support (thanks Simon [svdlinden]!).So, I wrote my proposal for school and got sponsorship to visit the Maemo Summit. There I met Zeeshan Ali (great guy, btw!) and also got a free Nokia N900. When I finally got over hacking the N900 with widgets galore (stay tuned for another post about that...) I sat down and started working on GUPnP-Python via GIR.So to build a GIR typelib there are two applications to help you along. The first is g-ir-scanner. The following command (only half complete -- those hardcoded paths need to be pkg-config-erized) was able to generate bindings for GUPnP. Notice that it is required that all of GUPnP's dependencies already have GIR support (and hence typelibs) installed in the proper locations before GUPnP would scan and compile. I had to add GIR to both GSSDP and libsoup before GUPnP would get bindings. (Patches to those libs imminent...) Also note the explicit addition of -I and --include for all of the includes. (-I required to properly scan the file for symbols, and --include to add the include to the intermediate .gir file so the eventual .typelib can find external symbols).Modifications to Makefile.am needed to get the demo working (does not include all headers yet):
SCANNER_BIN = g-ir-scannerSCANNER_ARGS = -v --add-include-path=.GLIB_INCLUDEDIR=`pkg-config --variable=includedir glib-2.0`/glib-2.0GLIB_LIBDIR=`pkg-config --variable=libdir glib-2.0`GLIB_LIBRARY=glib-2.0GUPnP-1.0.gir: env PYTHONPATH=.:.. env LPATH=.libs 577331 577808 577331SCANNER_BIN) 577331 577808 577331SCANNER_ARGS) --namespace GUPnP --nsversion=1.0 --noclosure --output --strip-prefix= --libtool="577331 577808 577331LIBTOOL)" --library=gupnp-1.0 -I/usr/include/libsoup-2.4 -I/usr/local/include/gssdp-1.0 -I/usr/include/libxml2 -I/usr/local/include/gobject-introspection-1.0 -I577331 577808 577331GLIB_INCLUDEDIR) -I577331 577808 577331GLIB_LIBDIR)/glib-2.0/include --include=GObject-2.0 --include=GSSDP-1.0 --include=libsoup-2.4 --include=libxml2-2.0 --pkg gupnp-1.0 gupnp-control-point.h gupnp-context.h gupnp-resource-factory.h gupnp-device-proxy.h gupnp-service-proxy.h gupnp-device-info.h gupnp-service-info.h gupnp-service-introspection.hGUPnP-1.0.typelib: GUPnP-1.0.gir g-ir-compiler --includedir=. 577331 577808 577331G_IR_COMPILER_OPTS) -o GUPnP-1.0.gir
Once we finally had GUPnP-1.0.gir (the actual scanned API in an XML format) and the .typelib (essentially a compiled and compressed version of the .gir) generated and put in place it was time to fire up pygobject!
>>> from gi.repository import GUPnP>>> print GUPnP
Success! Now to try and actually write a meaningful app. I felt the best way to do this was to first get a C based application using GUPnP and re-write it in python. So I grabbed some sample code from the GUPnP website and modified it a bit.
#include #include "libgupnp/gupnp.h"static GMainLoop *main_loop;static voiddevice_available_cb (GUPnPControlPoint *cp, GUPnPDeviceProxy *proxy){GUPnPDeviceInfo* gupnp_device_info = GUPNP_DEVICE_INFO(proxy); g_print("Device model name: %s", gupnp_device_info_get_model_name(gupnp_device_info));}int main(int argc, char** argv){GUPnPContext *context;GUPnPControlPoint *cp;/* Required initialisation */g_thread_init (NULL); g_type_init ();/* Create a new GUPnP Context. By here we are using the default GLib main context, and connecting to the current machine's default IP on an automatically generated port. */context = gupnp_context_new (NULL, NULL, 0, NULL);/* Create a Control Point targeting WAN IP Connection services */cp = gupnp_control_point_new (context, "upnp:rootdevice");/* The service-proxy-available signal is emitted when any services which match our target are found, so connect to it */g_signal_connect (cp, "device-proxy-available", G_CALLBACK (device_available_cb), NULL);/* Tell the Control Point to start searching */ gssdp_resource_browser_set_active (GSSDP_RESOURCE_BROWSER (cp), TRUE);/* Enter the main loop. This will start the search and result in callbacks to service_proxy_available_cb. */ main_loop = g_main_loop_new (NULL, FALSE);g_main_loop_run (main_loop);/* Clean up */g_main_loop_unref (main_loop);g_object_unref (cp);g_object_unref (context);return 0;}
And compiled..
gcc -I/usr/local/include/gupnp-1.0 -I/usr/local/include/gssdp-1.0 -I/usr/include/libxml2 -I/usr/include/libsoup-2.4 -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/include/gtk-2.0 -I/usr/lib/gtk-2.0/include -I/usr/include/atk-1.0 -I/usr/include/cairo -I/usr/include/pango-1.0 -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/include/pixman-1 -I/usr/include/freetype2 -I/usr/include/libpng12 -pthread -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I../.. -DDATA_DIR="/usr/local/share/gupnp-tools" -g -O2 -Wall -o test -pthread -Wl,--export-dynamic -L/usr/local/lib /usr/local/lib/libgupnp-1.0.so -L/usr/lib -luuid /usr/local/lib/libgssdp-1.0.so /usr/lib/libsoup-2.4.so /usr/lib/libxml2.so /usr/lib/libgnutls.so /usr/lib/libtasn1.so /usr/lib/libgcrypt.so /usr/lib/libgpg-error.so /usr/lib/libgtk-x11-2.0.so /usr/lib/libgdk-x11-2.0.so /usr/lib/libatk-1.0.so /usr/lib/libgdk_pixbuf-2.0.so /usr/lib/libgio-2.0.so /usr/lib/libpangocairo-1.0.so /usr/lib/libpangoft2-1.0.so /usr/lib/libcairo.so /usr/lib/libpixman-1.so /usr/lib/libglitz-glx.so /usr/lib/libGL.so /usr/lib/libXext.so /usr/lib/libglitz.so /usr/lib/libpng12.so /usr/lib/libXrender.so /usr/lib/libX11.so /usr/lib/libXau.so /usr/lib/libXdmcp.so /usr/lib/libpango-1.0.so -lm /usr/lib/libfontconfig.so /usr/lib/libfreetype.so -lz /usr/lib/libexpat.so /usr/lib/libgobject-2.0.so /usr/lib/libgmodule-2.0.so -ldl /usr/lib/libgthread-2.0.so -lpthread -lrt /usr/lib/libglib-2.0.so -pthread test1.c0./test1Device model name: MediaTomb
Success! It found my local media serve and correctly printed the model name.I decided to start small with the python application. Lets first just make a UPnPContext
from gi.repository import GLib, GUPnP# Get a default maincontextmain_ctx = GLib.main_context_default()# Bind to eth0 in the maincontext on any portctx = GUPnP.UPnPContext(main_ctx, "eth0", 0)
This, however, failed miserably:
** (process:20844): WARNING **: (pygi-argument.c:1818):_pygi_argument_release: runtime check failed: (!is_pointer || transfer == GI_TRANSFER_NOTHING)GLib-ERROR **: The thread system is not yet initialized.aborting...Aborted
(That warning will persist even when we eventually get it working. I plan to file a bug). Well, the error it gives is pretty clear, 'thread system not initialized'. Alright, lets initialize the thread system:
from gi.repository import GLib, GUPnPGLib.thread_init(None)# Get a default maincontextmain_ctx = GLib.main_context_default()# Bind to eth0 in the maincontext on any portctx = GUPnP.UPnPContext(main_ctx, "eth0", 0)---Traceback (most recent call last):File "bad.py", line 3, in GLib.thread_init(None) File "gi/types.py", line 34, in function return info.invoke(*args)RuntimeError: Could not locate g_thread_init: `g_thread_init': /usr/local/lib/libglib-2.0.so.0: undefined symbol: g_thread_init
Well, thats even worse. A quick strace and ldd confirmed that libglib-2.0 does indeed not contain the symbol g_thread_init. Whats funny is that normally pygobject doesn't allow you to call functions that doesn't exist in C through GIR. It complains that the function doesn't exist at all -- not that its not in the shared object! I stopped into #pybank on IRC and had a lengthy discussion with user malept who pointed out that the symbol is in fact in libgthreads, and that I was probably the only person using libgthreads + pygobject + GIR. Ah!It took a couple hours of headbanging but eventually I discovered that GObject also can initialize threads...
from gi.repository import GLib, GUPnP, GObjectGObject.threads_init()# Get a default maincontextmain_ctx = GLib.main_context_default()# Bind to eth0 in the maincontext on any portctx = GUPnP.UPnPContext(main_ctx, "eth0", 0)
That worked, the program exited normally. At this point coding the rest of the app using the C code as a model was fairly simple (a bunch of guess and check on how the C -> python-ization at times worked out well, for example in C its gssdp_resource_browser_set_active() and in python its GSSDP.ResourceBrowser.set_active()). Compare the following (working) python code to the original C. Which would you rather write? :)
from gi.repository import GLib, GUPnP, GSSDP, GObjectdef device_available(cp, device):print "GOT A DEVICE! Its model name is :", device.get_model_name()# Note: glib.thread_init() doesn't work here, have to use the gobject callGObject.threads_init()# Get a default maincontextmain_ctx = GLib.main_context_default()# Bind to eth0 in the maincontext on any portctx = GUPnP.UPnPContext(main_ctx, "eth0", 0)# Pretend to be a root device (Zeeshan needs to document these uri things...)cp = GUPnP.UPnPControlPoint(ctx, "upnp:rootdevice")# Use glib style .connect() as a callback on the controlpoint to listen for new devicescp.connect("device-proxy-available", device_available)# "Tell the Control Point to Start Searching"GSSDP.ResourceBrowser.set_active(cp, True)# Enter the main loop which begins the work and facilitates callbacksGObject.MainLoop().run()# Be cheeky, and intentionally NOT clean up memory because python is awesomeprint "You'll never know I am here!"--test.py** (process:21327): WARNING **: (pygi-argument.c:1818):_pygi_argument_release: runtime check failed: (!is_pointer || transfer == GI_TRANSFER_NOTHING)GOT A DEVICE! Its model name is : MediaTomb
Ah, victory. The plan now is to finish the GUPnP bindings (include the rest of the headers), clean up the patches to GSSDP, Libsoup and GUPnP and get them upstream. Then I'll write a few more demo apps to ensure the bindings work properly (and report/fix bugs) and then on with the rest of the project (and more N900 hacking of course!)