Category: Programming

Arduino vs Servos

The eyes on Exuro are basically 2 axis gimbals with a servo driving each axis. Never having built anything like this before, it was a bit of a learning experience. One of them was more of a re-learning experience in that I knew this but didn’t remember. On a small microcontroller board like an Arduino, there is a limited amount of power that can be drawn by anything connected to it.

In first working with the Arduino, I connected a single servo so that I could work through the initial hardware setup and some basic programming of the Arduino to make the servo arm move. Ground and power lines were connected to ground and +5v on the Arduino. The control line was connected to pin 9. It went pretty much as expected and was it was fun to see changes in software cause actions in the physical world.

Building on the initial success, I added a 2nd servo so I could start to see to the interactions and hardware layout needed to by the gimbals. Again, the ground and power lines were connected to the Arduino ground and +5v. Control was connected to pin 11. Mostly everything went smoothly but every once in a while, the Arduino would reset. It didn’t happen often so I didn’t spend much time tracing the problem. Until I added the 3rd servo. Ground and power lines were again connected to ground and +5v on the Arduino and the control line to pin 12.

With the 3rd connected and all 3 servos trying to move, the Arduino would reset. It would reset almost immediately after the servos started moving. I changed the code to only move 2 of the servos and things went back to working as expected. Every once in a while the Arduino would reset. But adding the 3rd would cause a reset every time. About this time it dawned on me that I might be trying to draw too much power through the Arduino.

So I dug through my collection of scrap wall warts till I found one that put out 5v at 2 amps. Since each servo wants around 0.5 amps, this should work. So I disconnected the power and ground lines for the 3 servos and connected them to the +5v and ground wires of the wall wart. I added a wire connecting the Arduino ground with the wall wart ground and powered up the Arduino. All three servos started moved their control arms back and forth without any more resets of the Arduino. Yippie!

Tags :

Arduino Switches

Part of the Exuro project includes the setup for a momentary contact switch that can be used to trigger the poofer. This way the system can either be run as a donation machine with the bill reader being used to trip the poofer or it can be manually controlled via a momentary contact switch.

My first attempt at connecting a switch was done by just connecting one switch terminal to ground and the other to the Arduino pin 2. The code was pretty straight forward:

int relay_state = LOW;
 
void toggle_relay()
{
  relay_state = !relay_state;
}
 
void setup() {
    Serial.begin(57600);
    attachInterrupt(0, toggle_relay, CHANGE); 
}

This had an number of unfortunate problems. The biggest one was that every once in a while the interrupt would be triggered all by itself . Without doing anything, the poofer relay would trigger. That wasn’t good. (The code for the poofer relay isn’t shown here.) It turns out that the problem was the electrical circuit. Since the switch was just connected between ground and pin 2, the wire running to the switch would act as an antenna and cause the hardware to think that the interrupt should be triggered. To resolve this problem, a 1k resistor was placed between pin 2 and ground. This ensured that the electrical signal traveling over the switch wire was stable.

The second annoying problem was that every once in a while an interrupt would be missed and the relay_state would be wrong. The toggle_relay code merely sets the relay_state to the inverse of what it was. That is if it’s a 1 then it becomes a 0 and if it’s a 0 then it becomes a 1. So if an interrupt is missed then the relay_state would be wrong and the poofer relay would be be the inverse of what it was supposed to be.

To resolve that, the toggle_relay code was changed to:

void toggle_relay()
{
  int val = digitalRead(2);
 
  if (val == LOW) {
    relay_state = LOW;
  } else {
    relay_state = HIGH;
  }
}

This way the relay_state is always set to the state of the switch as read off pin 2. With these two changes, adding a 1k resistor and directly reading the pin, the system is much more stable and works as needed. Yea!

Many thanks to CTP for the pointer to using a resister to stabilize the switch.

Tags : ,

OS X and the mystery bits

Sometimes I have to learn a lesson over and over again before I get a glimmer of what’s going on. I spent some time again today trying to get the MySQL client library module for Python working. I need it to be able to talk w/ a MySQL database for an online store. I’d rather use PostgreSQL but in this case, I don’t have a choice.

I downloaded MySQL OS X package and ran the installation program. It did what it was supposed to – install the software under /usr/local. The MySQL client runs and all seems right with the world. I download and install the MySQL Python client module and it builds as expected. But when I try to run Django, I get this error:

 
django.core.exceptions.ImproperlyConfigured: Error loading MySQLdb module: dynamic module does not define init function (init_mysql)

Grumble, grumble, grumble.

I try to load the MySQL Python module directly:

peter@drag:~> python
Python 2.6.1 (r261:67515, Feb 11 2010, 00:51:29)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import _mysql
Traceback (most recent call last):
  File "", line 1, in
  File "build/bdist.macosx-10.6-universal/egg/_mysql.py", line 7, in
  File "build/bdist.macosx-10.6-universal/egg/_mysql.py", line 6, in __bootstrap__
ImportError: dynamic module does not define init function (init_mysql)

I checked the build output. Checked the install output. Everything looks fine. This should work. Searching the web doesn’t help much since the references I find are from 2007 and 2008. The patches that they describe have already been incorporated into the MySQL Python module.

About this time, I realize that I’ve solved this problem before. It has to do with bits. I run:

peter@drag:~> python
Python 2.6.1 (r261:67515, Feb 11 2010, 00:51:29) 
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform
>>> platform.architecture()
('64bit', '')
>>>

I look at the original MySQL disk image that I downloaded and see that its a 32bit application. Mixing a 32bit dynamic library with a 64bit executable doesn’t work.

After downloading the 64bit MySQL disk image, installing it and rebuilding the MySQL Python module, everything works. Now its on to writing code for the online store. Yea.

Tags : , ,

Bad data and the wonder of RegEx

Regular expressions aren’t something that I use regularly but I think that’s about the change. I came up a programming world where C, Cobol and Fortran were just the best thing considering that the alternative was writing assembler. And in that world, regular expressions don’t really exist. The basic toolkit I developed internally for solving programming problems didn’t include it. To this day, I sometimes still think about iterating over a character sequence in memory until a NULL is found. But in the modern programming world, languages like Python have tools built into the language and standard libraries that provide a lot of help to a developer. One of those tools is regular expression matching.

I’m currently working on a project to convert a FileMaker 5.5 database to SalesForce. Inside the FileMaker database, the data is in a very strange and odd states because there was no data validation done. So a date field can contain strings like ‘1/1/01’, ‘6/24/2007’ or ‘9-1-2005’ or ‘2/27’ or ‘Jan 1.’ These different formats cause a world of grief in trying to move the data to a database that expects a date to be structured. Something like 1/1/2001. It doesn’t know about the various format of our dates and it rejects these as invalid. So the dates had to be filtered into something more standard.

I started to write some code that searches the strings for ‘/’ and ‘-‘ and the text names of months. And naturally, it quickly became a rats nest of nested if and conditions that made understanding the code very cumbersome. So I went looking for another way to solve the problem. I remembered that Python had this module called re that provides regular expression processing but I hadn’t really used it and wasn’t sure about how it would work. So I start searching the web to find some help on using regular expressions and the re module. What I found was just wonderful. A.M. Kuchling wrote up the fantastic Regular Expression HOWTO. In almost no time flat I was starting to put together a regular expression that would match most the permutations of dates (those based on ‘1/1/01’) that I’ve seen in our database and a second that would match the ones that had the months written out. The quickest way that I found to test the regular expressions was using Python’s unittest module. I would edit the regular expression, run the unittests, re-editing, re-run, etc until it worked and all the tests past.

The code that I developed looks like this:

import unittest
import re
 
 
class TestDateRegEx(unittest.TestCase):
    def test_slashes(self):
        slashes = re.compile(r'(d{1,2})[/-](d{1,2})[/-](d{2,4})$')
 
        self.assertEqual(slashes.match('1/1/10').group(),     '1/1/10')
        self.assertEqual(slashes.match('1/1/10').group(1),    '1')
        self.assertEqual(slashes.match('1/1/10').group(2),    '1')
        self.assertEqual(slashes.match('1/1/10').group(3),    '10')
        self.assertEqual(slashes.match('01/1/10').group(),    '01/1/10')
        self.assertEqual(slashes.match('11/1/10').group(),    '11/1/10')
        self.assertEqual(slashes.match('11/13/10').group(),   '11/13/10')
        self.assertEqual(slashes.match('1/1/2010').group(),   '1/1/2010')
        self.assertEqual(slashes.match('12/1/2010').group(),  '12/1/2010')
        self.assertEqual(slashes.match('1/21/2010').group(),  '1/21/2010')
        self.assertEqual(slashes.match('1/21/2010').group(1), '1')
        self.assertEqual(slashes.match('1/21/2010').group(2), '21')
        self.assertEqual(slashes.match('1/21/2010').group(3), '2010')
 
        self.assertEqual(slashes.match('111/1/10'),    None)
        self.assertEqual(slashes.match('11/111/10'),   None)
        self.assertEqual(slashes.match('11/11/10100'), None)
 
        self.assertEqual(slashes.match('1-1-10').group(),     '1-1-10')
        self.assertEqual(slashes.match('01-1-10').group(),    '01-1-10')
        self.assertEqual(slashes.match('11-1-10').group(),    '11-1-10')
        self.assertEqual(slashes.match('11-13-10').group(),   '11-13-10')
        self.assertEqual(slashes.match('1-1-2010').group(),   '1-1-2010')
        self.assertEqual(slashes.match('12-1-2010').group(),  '12-1-2010')
        self.assertEqual(slashes.match('1-21-2010').group(),  '1-21-2010')
        self.assertEqual(slashes.match('1-21-2010').group(1), '1')
        self.assertEqual(slashes.match('1-21-2010').group(2), '21')
        self.assertEqual(slashes.match('1-21-2010').group(3), '2010')
 
        self.assertEqual(slashes.match('111-1-10'),    None)
        self.assertEqual(slashes.match('11-111-10'),   None)
        self.assertEqual(slashes.match('11-11-10100'), None)
 
        self.assertEqual(slashes.match('1'),           None)
        self.assertEqual(slashes.match('1/'),          None)
        self.assertEqual(slashes.match('1/1'),         None)
        self.assertEqual(slashes.match('1'),           None)
        self.assertEqual(slashes.match('1/'),          None)
        self.assertEqual(slashes.match('1/1'),         None)
        self.assertEqual(slashes.match('Jan 1'),       None)
        self.assertEqual(slashes.match('Jan 1, 2010'), None)
 
 
    def test_names(self):
        names = re.compile(r'(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)s(d{1,2})(,s(d{2,4}))?$',
                           re.IGNORECASE)
 
        self.assertEqual(names.match('Jan 1').group(),         'Jan 1')
        self.assertEqual(names.match('Jan 1').group(1),        'Jan')
        self.assertEqual(names.match('Jan 1').group(2),        '1')
        self.assertEqual(names.match('feb 21').group(),        'feb 21')
        self.assertEqual(names.match('feb 21').group(1),       'feb')
        self.assertEqual(names.match('feb 21').group(2),       '21')
        self.assertEqual(names.match('Jan 1, 2010').group(),   'Jan 1, 2010')
        self.assertEqual(names.match('Jan 1, 2010').group(1),  'Jan')
        self.assertEqual(names.match('Jan 1, 2010').group(2),  '1')
        self.assertEqual(names.match('Jan 1, 2010').group(4),  '2010')
        self.assertEqual(names.match('MAR 14, 2010').group(),  'MAR 14, 2010')
        self.assertEqual(names.match('MAR 14, 2010').group(1), 'MAR')
        self.assertEqual(names.match('MAR 14, 2010').group(2), '14')
        self.assertEqual(names.match('MAR 14, 2010').group(4), '2010')
 
        self.assertEqual(names.match('MA 20, 2010'), None)
        self.assertEqual(names.match('xyz 2,'),      None)
        self.assertEqual(names.match('jan 2,'),      None)
 
 
if __name__ == '__main__':
    unittest.main()

The use of regular expressions and the ability to match and group the matches makes the code needed to clean up the dates much simpler and a lot easier to maintain. It might take a moment next time I need to parse strings to think about using regular expressions and I’m pretty sure I’ll have to refer to the howto a couple more times but I’m really happy with how powerful and how much simpler text processing can be by using them.

Tags : ,

Web service monitoring w/ Nagios and JSON

I’m using Nagios to act as a watch dog for my network and the various services that live on it. Nagios does the job pretty well. It lets me know when there’s a problem, when things are back to normal and generally keeps on eye on things for me.

The checks that Nagios performs are done through a series of check commands. These commands are your typical Unix style program with the exceptions that they produce a single line of text that describes the state of the item being checked and the exit value let’s Nagios know what’s going on.

So for instance, to check the health of the web service on the localhost:

peter@sybil:~$ /usr/lib/nagios/plugins/check_http -H localhost
HTTP OK HTTP/1.1 200 OK - 361 bytes in 0.001 seconds |time=0.001021s;;;0.000000 size=361B;;;0
peter@sybil:~$ echo $?
2
peter@sybil:~$

The single line of text that is displayed follows a specific format. It starts with the prefix of what’s being tested, HTTP. Next is the status, OK. This can be OK, WARNING, CRITICAL or UNKNOWN. Everything after the status is eye candy that provide details that are specific to the test being done. Nagios doesn’t really care about it but it does provide important details when looking at problems that may be occurring.

Writing these check program in Python is pretty straight forward.

I recently had a situation where our ISP moved our web servers from one physical machine to another. This resulted in the credit cards processing for our online store to fail. The payment provider uses the IP address of the server as part of the authentication process when submitting credit cards for processing. Since the server changed, the IP address changed. Things went around in circles for a while until we figured out the problem and gave the new IP address to the payment
provider.

I thought is would be a good additional Nagios check for the store web site to check on the IP address of the physical server. Unfortunately, the ISP doesn’t provide access to the IP address. But they do provide access to the hostname.

To get the hostname, I added a simple CGI program that determines the hostname and then packages it up as a JSON data structure.

#!/usr/bin/env python
 
"""
Bundle the hostname up as a JSON data structure.
 
Copyright (c) 2009 Peter Kropf. All rights reserved.
"""
 
import cgi
import popen2
import sys
sys.path.insert(1, '/home/crucible/tools/lib/python2.4/site-packages')
sys.path.insert(1, '/home/crucible/tools/lib/python2.4/site-packages/simplejson-2.0.9-py2.4-linux-x86_64.egg')
 
import simplejson as json
 
field = cgi.FieldStorage()
print "Content-Type: application/json\n\n"
 
r, w, e = popen2.popen3('hostname')
host = r.readline()
r.close()
w.close()
e.close()
 
fields = {'hostname': host.split('n')[0]}
 
print json.dumps(fields)

There’s a couple of things to note. Since the ISP provides a very restrictive environment, I have to add the location of the simplejson module before it can be imported. It’s a bit annoying but it does work.

On the Nagios service side, I created a new check program called check_json. It takes the name of a field, the expected value and the URI from which to pull the JSON data.

#! /usr/bin/env python
 
"""
Nagios plugin to check a value returned from a uri in json format.
 
Copyright (c) 2009 Peter Kropf. All rights reserved.
 
Example:
 
Compare the "hostname" field in the json structure returned from
http://store.example.com/hostname.py against a known value.
 
    ./check_json hostname buenosaires http://store.example.com/hostname.py
"""
 
 
import urllib2
import simplejson
import sys
from optparse import OptionParser
 
prefix = 'JSON'
 
class nagios:
    ok       = (0, 'OK')
    warning  = (1, 'WARNING')
    critical = (2, 'CRITICAL')
    unknown  = (3, 'UNKNOWN')
 
 
def exit(status, message):
    print prefix + ' ' + status[1] + ' - ' + message
    sys.exit(status[0])
 
 
parser = OptionParser(usage='usage: %prog field_name expected_value uri')
options, args = parser.parse_args()
 
 
if len(sys.argv) < 3:
    exit(nagios.unknown, 'missing command line arguments')
 
 
field = args[0]
value = args[1]
uri = args[2]
 
try:
    j = simplejson.load(urllib2.urlopen(uri))
except urllib2.HTTPError, ex:
    exit(nagios.unknown, 'invalid uri')
 
if field not in j:
    exit(nagios.unknown, 'field: ' + field + ' not present')
 
if j[field] != value:
    exit(nagios.critical, j[field] + ' != ' + value)
 
exit(nagios.ok, j[field] + ' == ' + value)

Some checking is done to ensure that the JSON data can be retrieved, that the needed field is in the data and then that the field’s value matches what’s expected.

These examples show the basic testing that’s done and the return values:

peter@sybil:~$ /usr/lib/nagios/plugins/check_json hostname buenosaires http://store.thecrucible.org/hostname.py
JSON OK - buenosaires == buenosaires
peter@sybil:~$ echo $?
0
peter@sybil:~$ /usr/lib/nagios/plugins/check_json hostname buenosaires http://store.thecrucible.org/hostname.p
JSON UNKNOWN - invalid uri
peter@sybil:~$ echo $?
3
peter@sybil:~$ /usr/lib/nagios/plugins/check_json hostname buenosairs http://store.thecrucible.org/hostname.py
JSON CRITICAL - buenosaires != buenosairs
peter@sybil:~$ echo $?
2
peter@sybil:~$ /usr/lib/nagios/plugins/check_json ostname buenosaires http://store.thecrucible.org/hostname.py
JSON UNKNOWN - field: ostname not present
peter@sybil:~$ echo $?
3
peter@sybil:~$

Once the Nagios server is configured with the new command, the hostname on the server can be monitored and hopefully ease any problems that may occur then next time things change at the ISP.

More details on Nagios can be found at http://nagios.org and on developing check program at http://nagiosplug.sourceforge.net/developer-guidelines.html.

Tags : ,