Sometimes I have to learn a lesson over and over again before I get a glimmer of what’s going on. I spent some time again today trying to get the MySQL client library module for Python working. I need it to be able to talk w/ a MySQL database for an online store. I’d rather use PostgreSQL but in this case, I don’t have a choice.
I downloaded MySQL OS X package and ran the installation program. It did what it was supposed to – install the software under /usr/local. The MySQL client runs and all seems right with the world. I download and install the MySQL Python client module and it builds as expected. But when I try to run Django, I get this error:
django.core.exceptions.ImproperlyConfigured: Error loading MySQLdb module: dynamic module does not define init function (init_mysql)
Grumble, grumble, grumble.
I try to load the MySQL Python module directly:
peter@drag:~> python Python 2.6.1 (r261:67515, Feb 11 2010, 00:51:29) [GCC 4.2.1 (Apple Inc. build 5646)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import _mysql Traceback (most recent call last): File "", line 1, in File "build/bdist.macosx-10.6-universal/egg/_mysql.py", line 7, in File "build/bdist.macosx-10.6-universal/egg/_mysql.py", line 6, in __bootstrap__ ImportError: dynamic module does not define init function (init_mysql)
I checked the build output. Checked the install output. Everything looks fine. This should work. Searching the web doesn’t help much since the references I find are from 2007 and 2008. The patches that they describe have already been incorporated into the MySQL Python module.
About this time, I realize that I’ve solved this problem before. It has to do with bits. I run:
peter@drag:~> python Python 2.6.1 (r261:67515, Feb 11 2010, 00:51:29) [GCC 4.2.1 (Apple Inc. build 5646)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import platform >>> platform.architecture() ('64bit', '') >>>
I look at the original MySQL disk image that I downloaded and see that its a 32bit application. Mixing a 32bit dynamic library with a 64bit executable doesn’t work.
After downloading the 64bit MySQL disk image, installing it and rebuilding the MySQL Python module, everything works. Now its on to writing code for the online store. Yea.
Django is pretty good at creating a database driven website. The documentation is clear and the tutorials show how to use the framework to create web based applications. But one part that I wish was a bit more straight forward is running scripts outside the web server. The issue is that Django code expects to have a certain environment configured and setup for the framework. With this in place, you can preform tasks like polling an IMAP server for incoming email messages or monitoring a directory for new files or whatever else needs to be done. There are several posts online to help you get the environment setup here, here and here. But some of them seem not to work correctly because of the changes to Django for the 1.0 release or other reasons.
I have a fairly straight forward example of how to setup the Django environment and allow the rest of your code to access the Django framework for your web application. Its remarkably simple and straight forward.
Suppose that I’ve created a Django project in my tmp directory called demo_scripts and within that project, I create an app called someapp.
peter@fog:~/tmp> django-admin-2.5.py startproject demo_scripts peter@fog:~/tmp> cd demo_scripts/ peter@fog:~/tmp/demo_scripts> django-admin-2.5.py startapp someapp peter@fog:~/tmp/demo_scripts>
I create a model in someapp that looks like:
from django.db import models class Foo(models.Model): name = models.CharField(max_length=21, unique=True, help_text="Name of the foo.") def __unicode__(self): return self.name class Meta: ordering = ('name',)
Next step is to sync the database:
peter@fog:~/tmp/demo_scripts> ./manage.py syncdb Creating table auth_permission Creating table auth_group Creating table auth_user Creating table auth_message Creating table django_content_type Creating table django_session Creating table django_site Creating table someapp_foo You just installed Django's auth system, which means you don't have any superusers defined. Would you like to create one now? (yes/no): yes Username (Leave blank to use 'peter'): E-mail address: firstname.lastname@example.org Password: Password (again): Superuser created successfully. Installing index for auth.Permission model Installing index for auth.Message model peter@fog:~/tmp/demo_scripts>
And add some initial data to the database:
peter@fog:~/tmp/demo_scripts> ./manage.py shell Python 2.5.4 (r254:67916, May 1 2009, 17:14:50) [GCC 4.0.1 (Apple Inc. build 5490)] on darwin Type "help", "copyright", "credits" or "license" for more information. (InteractiveConsole) >>> from someapp.models import Foo >>> Foo(name='A Foo').save() >>> Foo(name='Another Foo').save() >>> peter@fog:~/tmp/demo_scripts>
Now we can write a standalone script to do something with the data model. For simplicity’s sake, I’ll just print out all the Foo objects. The script is going to live in a new directory called scripts. Here’s the source:
#! /usr/bin/env python #coding:utf-8 import sys import os import datetime sys.path.insert(0, os.path.expanduser('~/tmp/demo_scripts')) os.environ['DJANGO_SETTINGS_MODULE'] = 'settings' from someapp.models import * print Foo.objects.all()
When I run the script, it prints the array of the two Foo objects that I previously created:
peter@fog:~/tmp/demo_scripts> ./scripts/show_foo.py [<Foo: A Foo>, <Foo: Another Foo>] peter@fog:~/tmp/demo_scripts>
Lines 8 and 9 are the critical lines in the script code. The first adds the project directory to the Python system path so that the settings module can be found. The second tells the Django code which module to import to determine the project settings.
Django has an excellent user management and authentication system built into the framework. With it you can easily create users that can be authenticated against the website. But there are times when you just need to authenticate against a different system. In the case of an app I recently developed, I originally wanted to authenticate against an OS X Server. The OpenDirectory service on OS X Server is an LDAP server, under the hood you’ll find slapd from OpenLDAP running. So should be pretty straight forward to create an authentication module that uses Python’s LDAP module. And this article from the Carthage WebDev site shows you how to do it.
After I got the ldap_auth.py module working on my site, I realized the site would be better served if the authentication happened against Google Apps. Since Google Apps is currently being used by the organization for email, calendaring and sharing documents, everyone already has an account there. And with the ldap_auth.py module from Carthage Webdev, I thought it would be pretty simple to provide a google_auth.py module.
On a side note, I’m using Python 2.5 as installed via MacPorts. Before I could use the gdata APIs, I had to install py25-socket-ssl.
The APIs are pretty well documented via the examples from the Python Developer’s Guide. Here’s how I’m authenticating a Django project with users on Google Apps.
To start, there are three configuration variables that I added to the Django project’s settings.py module:
# Google Apps Settings GAPPS_DOMAIN = 'your_domain.com' GAPPS_USERNAME = 'name_of_an_admin_user' GAPPS_PASSWORD = 'admin_users_password'
These will allow the module to authenticate against Google Apps and ask for specific details about the user.
Here’s the code for google_auth.py:
import logging from django.contrib.auth.models import User from django.conf import settings from gdata.apps.service import AppsService, AppsForYourDomainException from gdata.docs.service import DocsService from gdata.service import BadAuthentication logging.debug('GoogleAppsBackend') class GoogleAppsBackend: """ Authenticate against Google Apps """ def authenticate(self, username=None, password=None): logging.debug('GoogleAppsBackend.authenticate: %s - %s' % (username, '*' * len(password))) admin_email = '%s@%s' % (settings.GAPPS_USERNAME, settings.GAPPS_DOMAIN) email = '%s@%s' % (username, settings.GAPPS_DOMAIN) try: # Check user's password logging.debug('GoogleAppsBackend.authenticate: gdocs') gdocs = DocsService() gdocs.email = email gdocs.password = password gdocs.ProgrammaticLogin() # Get the user object logging.debug('GoogleAppsBackend.authenticate: gapps') gapps = AppsService(email=admin_email, password=settings.GAPPS_PASSWORD, domain=settings.GAPPS_DOMAIN) gapps.ProgrammaticLogin() guser = gapps.RetrieveUser(username) logging.debug('GoogleAppsBackend.authenticate: user - %s' % username) user, created = User.objects.get_or_create(username=username) if created: logging.debug('GoogleAppsBackend.authenticate: created') user.email = email user.last_name = guser.name.family_name user.first_name = guser.name.given_name user.is_active = not guser.login.suspended == 'true' user.is_superuser = guser.login.admin == 'true' user.is_staff = True user.save() except BadAuthentication: logging.debug('GoogleAppsBackend.authenticate: BadAuthentication') return None except AppsForYourDomainException: logging.debug('GoogleAppsBackend.authenticate: AppsForYourDomainException') return None return user def get_user(self, user_id): user = None try: logging.debug('GoogleAppsBackend.get_user') user = User.objects.get(pk=user_id) except User.DoesNotExist: logging.debug('GoogleAppsBackend.get_user - DoesNotExist') return None return user
It was pretty easy to write and debug this code using the ldap_auth.py module as a working example.
One downside to this code is that any newly created users in the Django auth database don’t have any rights. So if the Django project is expecting to be able to dynamically change the contents based on the rights that the user has, the account will have to manually modified via the Django admin interface. Not too bad, but annoying.