Dave Dash

The Python textcluster Package

2010-07-08T00:00:00-07:00

Earlier I wrote about finding the most common Firefox issues. I had wanted to automate that process and continually find these issues. Unfortunately I never had time to do this.

When they announced Firefox Input, I thought about doing this again... just with Firefox Input data but then I went on paternity leave and time kind of crept away. But I mentioned the idea this week and it piqued some interest.

So I found myself with a bit of time to work on it. The first stage was releasing a python library called textcluster.

textcluster takes the work I did earlier and makes it a bit more general purpose. The idea is I can do something like this:

docs = (
        'Every good boy does fine.',
        'Every good girl does well.',
        'Cats eat rats.',
        "Rats don't sleep.",
        )

c = Corpus()
for doc in docs:
    c.add(doc)

print c.cluster()

Which results in:

[
    (
        "Rats don't sleep.",
        {'Cats eat rats.': 0.21353467285253394}
    ),
    (
        'Every good girl does well.',
        {'Every good boy does fine.': 0.32030200927880093}
    )
]

The number is the "similarity" between the strings relative to the entire document corpus.

My next trick is to see if I can run this memory-intensive calculation over a data-set of 25,000 opinions submitted. If I can we can get some interesting data about what people think of the new Firefox beta.

Firefox Input, powered by Sphinx

2010-07-06T00:00:00-07:00

Thursday, I decided to take a half-day for my sanity, but saw an email about how Whoosh wasn't going to cut it for Firefox Input. I was CC'd about this and there was mention that Sphinx might be possible.

Sphinx is my hammer, and everything is a nail. So I said, let's do this. That translated into me spending my weekend, soothing my newborn and working on Sphinx. Luckily this was easy, since AMO and SUMO are both running Sphinx in a similar Django environment.

In order to move quickly, I copied code from the Zamboni project to Firefox Input. Even our deployment into staging and production wasn't done by our usual "Sphinx guy" in IT. Ultimately, everything landed in place.

So try it out and file bugs or let me know if searches don't go as planned.

Firebug Lite and symfony

2007-06-02T00:00:00-07:00

I wrote a symfony plugin to enable the Firebug Lite javascript in your symfony app (in much the same way that web_debug works).

Firebug, Gmail Manager and Adblock are all plugins I use regularly in Firefox. However, I've been getting tired of Firefox's hunger for my resources, so I want to be as browser independent as possible. So this plugin allows me (and you) to embed some of the debugging functionality of Firebug in other browsers.