inside Habbie's mind

This script fixes the CSV up in a few ways:

• it puts a single quote character in front of all timestamp fields - without it, Numbers will interpret the dates and, for me, it is confused about day field vs. month field
• it sorts the CSV by start time, ascending instead of descending

Code: (download here)

#!/usr/bin/env python
import csv
import sys
import operator

r = csv.reader(sys.stdin)
rows=[]
for row in r:
        row[5] = "'%s" % row[5]
        row[6] = "'%s" % row[6]
        row[7] = "'%s" % row[7]
        rows.append(row)

rows = [rows[0]] + sorted(
       rows[1:],
       key=operator.itemgetter(5)
)

w = csv.writer(sys.stdout)
for row in rows:
        w.writerow(row)

A simplified version of my workflow would look like this:

def post(entry):
  title = entry.title
  print "posting [%s]" % title
  api.PostUpdate(title) # api is a twitter Api object

feed = feedparser.parse(config["feed"])
for e in reversed(feed.entries):
  if not e.id in seen:
    post(e)

It’s the print statement. All the APIs I’m using have zero trouble with unicode, but print wants to encode for your terminal and it’ll usually assume that that is ASCII. My ‘debugging’ output actually broke the program. My workaround is to say title.encode("ascii","replace")

Brend on #python pointed out to me that the issue is not, exactly, print. The issue is interpolating title into a non-unicode string. Depending on environment, using print on the unicode object might in fact work. For those environments, saying print u"posting [%s]" % title could help. In my case however, I ran into the issue from cron with no locale set at all, so dumbing the string down to ascii is still the right thing to do.

Initially I had a more elaborate version based on the exact connect() code in the higher class, but this simpler version works just fine. Incidentally, what the xen xm commandline tool uses works identically ;)

class UHTTPConnection(httplib.HTTPConnection):
    """Subclass of Python library HTTPConnection that
       uses a unix-domain socket.
    """
 
    def __init__(self, path):
        httplib.HTTPConnection.__init__(self, 'localhost')
        self.path = path
 
    def connect(self):
        sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
        sock.connect(self.path)
        self.sock = sock

This small hack plus the fine PHP serialization classes from Scott Hurring are the basis of OpenPanel.coreclient, which allows for very easy provisioning and querying of OpenPanel/opencore data. We use it mostly for unit testing.