Dave Dash

tunneling and mysql

2008-10-22T00:00:00-07:00

I have a mysql server for reviewsby.us that requires tunneling, and after establishing a tunnel like so:

deveshistan:~ dash* ssh user@host -L3307:database_server:3306

I thought I could just do this:

mysql -u mysql_user -P3307 -h localhost

But I was wrong, localhost should be 127.0.0.1, otherwise mysql will do things in socket mode:

mysql -u mysql_user -P3307 -h 127.0.0.1

Why code rewrites are often coupled with redesigns

2008-08-13T00:00:00-07:00

I've done some rewrites of code, and they usually are coupled with redesigns.

Redesigns and rewrites are tricky. With web sites existing users tend to prefer incremental changes with each. Changing a design element or a feature are pretty much equivalent when its done incrementally.

Usually small changes are like surgery. You make a small change, you possibly announce it, and people look at it and usually say yes, this change is better. But mostly a small change can go unnoticed.

Tiny atomic changes are nice and they improve the product.

A rewrite of code however is a discrete change. They take forever. Second system and all that. They inevitably involve a redesign as well. They tend to be received with mixed reviews.

Generally a design of a site is grafted onto code. Sometimes by templating languages, sometimes in a tightly coupled system. In either case there's usually two options:

Make a faithful port of the design.
Redesign

The latter is generally preferred since this saves a lot of time. Generally there are bugs with a design, and there is a design backlog. A faithful port of the design would mean porting every broken HTML code, every non user-friendly element, etc.

Unfortunately this leaves web developers with a scary situation: Launching a product which changes peoples functional and UI experience.

The good thing is redesigns usually mean evolutionary leaps. Meaning, iterations can happen faster, both in design and functionality. We can easily please new customers, and we can quickly bring current customers up to speed.

In my spare time, I've been rewriting code that I wrote in symfony (PHP) into django (python). It's a tiresome long process that can easily kill a side project.

postgresql and python in osx

2008-08-10T00:00:00-07:00

I want to start dabbling with postgreSQL on OS X. After several SVN checkouts, binary packages, etc, I've realize the easiest path to success is just installing from fink unstable.

Running:

fink install psycopg2-py2.5

will get you everything you need, python, the pyscopg2 driver and even postgresql.

Unfortunately I had MacPython binary and a postgres binary which I couldn't easily install (and if anybody knows how to uninstall at least the former, I'm all ears). I decided to just merge the site-packages for both the fink installed python2.5 and the MacPython:

$ rm -rf /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/
$ ln -s /sw/lib/python2.5/site-packages /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/

Now if I installed more python packages via fink they will work in MacPython.

Optimization heuristics

2008-02-26T00:00:00-08:00

I decided to play version control private-eye today when my coworker mentioned that we make a system call to check the time several times per request on a few specific pages.

My analysis was we didn't need to have the calls but it had me thinking...

The micro-caching heuristic

One way of optimizing code is by calculating expensive operations once and storing them in a specific location. It's a micro-cache if you will of data.

One obvious example ends up being loops:

The inefficient example computes count($arr) n times where n is the size of the array $arr. The latter computes it once.

When we write and review code we should look out for loops and repeated code. These elements can be pulled out.

With some data lookups like system calls or queries we can apply a request-wide caching. In a symfony app, for example, we might need to calculate the time stamp in several locations throughout the code. Rather than calling time() throughout the code, we can create a site wide accessible attribute that has the same value.

For example, we can subclass sfContext and add a method called getTime() which runs time() once and only once per request and stores it internally. Whenever part of the app needs the time stamp, myContext::getInstance()->getTime() (or an equivalent shortcut) can be called. Of course, I haven't tested this, and myContext might not be the best place for this, but a similar strategy will work.

-dd

How to code... or do anything.

2008-02-01T00:00:00-08:00

Imagine the most ideal way for something to work, assume that it's going to work, and then make it work like that.

It's amazing how much bad coupled code exists for no good reason.

How to code... or do anything.

2008-02-01T00:00:00-08:00

Imagine the most ideal way for something to work, assume that it's going to work, and then make it work like that.

It's amazing how much bad coupled code exists for no good reason.

reusability

2008-01-30T00:00:00-08:00

[tags]django, plugins, apps, projects, symfony[/tags]

A project is a collection of settings for an instance of Django, including database configuration, Django-specific options, and application-specific settings.

The Django Book, Chapter 2:

A few people have been asking for more comparisons between symfony and Django. For me it's a great way of understanding Django and python as well as symfony and PHP.

Reusability is at the core of Django, not an afterthought. The only unique part of an app is the settings and the views. Everything else is an application that can exist independently of your app. It's nice and decoupled.

This wouldn't be impossible to do in symfony. Each module could be designed from the start as a plugin. Complete with its own set of models and default templates. The configuration of a project/app could then make the web app unique.

Right now the bulk of my symfony models are tightly coupled to their apps. It's a little confusing, but there isn't a direct correlation between Django projects, Django apps and symfony Projects, apps and modules. Each kind of overlaps one another.

Templating

2008-01-28T00:00:00-08:00

[tags]smarty, symfony, php, python, django[/tags]

I used to use the Smarty templating system quite heavily in my PHP apps. Then after switching over to symfony, I broke that habit.

PHP is a templating language for better or worse. Smarty was a way of saying that "well things could be easier", but that's against the grain of what PHP fundamentally is.

One frustration of PHP is this. A "pure" PHP file with no HTML looks like this:

PHP is an HTML templating system, by default everything in PHP is just stuff waiting to be sent untouched to the browser. So in essence, PHP web apps aren't really web apps... they are very complicated templates all talking to each other.

So you'll note in the above example, even a pure PHP file must begin with <?php, and signal that we're actually doing code. In fact, to prevent any unintended whitespace being sent to the browser, a PHP only file will by convention omit the closing ?>.

So my shiny new Django framework is a breath of fresh air. There's no <?py tags, it's all just python. And template files are explicitly template files parsed by a template engine built on top of a scripting language.

I can easily see PHP evolving into something similar one day. Where all PHP files are just PHP by default. It would be more in line with the way MVC development is going.

YUI Autocomplete the easy way

2008-01-27T00:00:00-08:00

[tags]yui, autocomplete, javascript, jquery, symfony[/tags]

I'm redoing the UI for reviewsby.us and for the Javascript library I am standardizing on is YUI. The trick with YUI is that it's very verbose and very configurable.

Unfortunately to do simple things you have to write quite a lot. Which, coming from a jQuery background, is not what I'm used to. Yahoo provides a lot of useful examples for Auto Complete, I'll provide you with another (built in symfony, but anything in any server-side language will work). You'll need:

A data source
Some YUI
Some javascript of your own
And some HTML

We're building a tool to grab random items from our database and put them into our Autocomplete field. We also want to capture and id of these items and set them in our form.

Here we go...

Data Source

YUI gives you a lot of options with data sources. In many cases a remote script that returns JSON will be best. JSON is nice, compact, human debuggable and machine-readable.

We can make a searchJSON method that simply takes your database results and wraps them in to a formatted array and then using PHP's json_encode we can return a nice JSON object.

The ResultSet and Result keys are important as we'll see later.

We can serve this JSON method very easily in symfony like so:

Note that we use renderText rather than a separate ajaxListSuccess.php file.

YUI

There's some YUI code that you need. It's described well in the Autocomplete documentation.

Add the following to your page:

http://yui.yahooapis.com/2.4.1/build/yahoo-dom-event/yahoo-dom-event.js" type="text/javascript
http://yui.yahooapis.com/2.4.1/build/connection/connection-min.js type="text/javascript
http://yui.yahooapis.com/2.4.1/build/autocomplete/autocomplete-min.js
http://yui.yahooapis.com/2.4.1/build/autocomplete/assets/skins/sam/autocomplete.css

These are the bare YUI requirements to get Autocomplete working.

Javascript

There's a good deal of Javascript that you'll need to provide on your own. Here's what I wrote:

var MA = {};
    MA.autocomplete = function()
    {
      var e = YAHOO.util.Event;
      var w = YAHOO.widget;
  
      return {
        init: function()
        {
           e.onAvailable("myInput", this.fnHandler);
        },
        fnHandler: function()
        {
          var rDS = new w.DS_XHR("/ajax/object/list", ["ResultSet.Result","Title"]);

rDS.maxCacheEntries    = 60; 
          rDS.queryMatchContains = true;

var rAC = new w.AutoComplete("myInput","myACContainer", rDS);

rAC.formatResult = function(item, query) 
          {
            return item[1].Title;
          };
      
          rAC.forceSelection           = true; 
          rAC.allowBrowserAutocomplete = false; 
      
          rAC.itemSelectEvent.subscribe(
            function(sType, aArgs) 
            { 
              var data = aArgs[2];
              document.getElementById("object_id").value = aArgs[2][1]['Id'];
            }
          ); 
        },
    
      }
    }();

MA.autocomplete.init();

MA is my own private namespace. /ajax/object/list is the url of our data source we defined earlier. myInput, myACContainer and object_id are all IDs of elements in our DOM which we'll look at next.

HTML

Okay I went through this backwards, now we have the HTML.

This is all you need:

The End

That's it. It might seem like a lot, but YUI offers a tested working solution that works and is very customizable.

python: relative paths

2008-01-27T00:00:00-08:00

So I started yesterday with Django, and I decided I didn't want to futz with creating another mysql database that I'd need to manage, etc. Instead I'll just use sqlite.

I wanted to keep my sqlite database within my project regardless of where I might move my project later. So I did this:

I confused a lot of people on IRC, but it's really quite easy:

__file__ is the filename of the current script, very similar to PHP's __FILE__
os.path.abspath calculates the absolute path, hence the absolute path of the current file
os.path.join does all the nasty business of joining paths together and figuring out what type of slashes are needed, etc.
'data/db.sqlite' is a string

So really all we were doing is creating a relative path, but setting it absolutely.

GeoDjango

2008-01-27T00:00:00-08:00

[tags]geodjango, django, gis[/tags]

I just discovered GeoDjango. The existence of this is awesome.

The GIS layer of reviewsby.us is incredibly complicated, and if I do another project of that magnitude, it'll be nice to know that a framework exists to tackle problems. It looks functional, but not quite documentation complete for usage, but if I endeavor to learn more about GIS, I'm going to dive-in.

django experiment: Day 1 and Model Inheritence

2008-01-27T00:00:00-08:00

[tags]django, inheritance[/tags]

I spent a few hours diving into Django. I knew enough python to get around, and also to be dangerous.

Here's a summary of what I've learned.

Model Inheritence

So far I've learned that ModelInheritence is not quite ready. In other words, you can't define in your models.py something like this:

The model itself will work, and it will save, but things like Page.objects.all() will return empty lists.

Where I'm still shaky

I... * still have to learn how to create templates. * want better understanding of the available field types. * want to find a good CMS plugin * want to know if there's the idea of environments in Django (like in symfony) * want to override .save() on my models * want to learn how to have self-referencing models (E.g. a hierarchical category)

I'm going to keep on plugging along, it's really easy to figure out django.

python: it begins

2008-01-26T00:00:00-08:00

[tags]php, python, symfony, frameworks, programming[/tags]

The problem I have with python, is I really like what I can do with it, and it makes me hate that I have to use PHP.

PHP works. I've used it for over 10 years without problem. My web development methodology was this: I want to do x, but I need to first learn y. Well one day y became PHP/FI and then PHP 3 and then PHP 4 and then PHP5. It was a natural progression.

I came from a (what was at the time) traditional C/C++/Java background and decided that PHP while amateurish was still a better solution for web development. A few years later I was proven right by companies like Yahoo! and Facebook.

I did have my issues, and I worked through them, symfony and other frameworks like it, have made PHP development so much better, but eventually you get let in on little secrets. Secrets like other scripting languages.

In the last year I dabbled with ruby, but decided I didn't want to look at rails. A few people pointed me in the direction of python and Django as an alternative to PHP alternatives. So I've spent time learning python.

Yesterday and today I gave myself a real project. I started hacking away at del.icio.us for a Winter Hack Day at Yahoo! I would write a few lines of code, and then realize that I can write it a little better a little cleaner using some neat little python constructs. It's enjoyable in the way that PHP is not.

It might just be a phase, but I sincerely doubt it. A lot of people I know who use python love it. Many have been using it for years (even while I secretly thought, "python? really?" Well, I'm starting to think they've been onto something. We'll see where I end up.

python: it begins

2008-01-26T00:00:00-08:00

[tags]php, python, symfony, frameworks, programming[/tags]

The problem I have with python, is I really like what I can do with it, and it makes me hate that I have to use PHP.

Python, Named arguments: Pure Genius

2007-12-21T00:00:00-08:00

I decided I want to learn python, if only to learn Django and to "get" what all the python hub-bub is about.

Python's named arguments in function calls is pure genius. Let me explain.

In PHP, and many other languages you can define a function as such:

function foo($a = 2, $b = 2)
{
    return pow($a,$b);
}

If you follow, foo() will give you 4. foo(3) is 9 and foo(99,0) is 1. In python we can do the same thing, but it'll pay to use some better variable names:

def foo(base=2, exponent=2):
    return base**exponent

Similarly foo() will give you 4. foo(3) is 9 and foo(99,0) is 1. But what if we forgot what the order was? Did base come first or was it exponent? We can do this:

foo(exponent=99, base=2)

Since base and exponent both have default values, we can even omit base and let it use the default:

foo(exponent=10)

This means rather than passing an $options array to my functions and checking whether an option was set or not, I can just specify which options I want in my function call. Or instead of remembering the order of the arguments, I can use whatever order suits me. Or instead of calling a function like bar(null, null, null, 2) I can just skip those first three arguments all together.

A side effect of this, is now there's a real use, even for simple functions, to give your variables easy to remember names.

Python, Named arguments: Pure Genius

2007-12-21T00:00:00-08:00

I decided I want to learn python, if only to learn Django and to "get" what all the python hub-bub is about.

Python's named arguments in function calls is pure genius. Let me explain.

In PHP, and many other languages you can define a function as such:

function foo($a = 2, $b = 2)
{
    return pow($a,$b);
}

If you follow, foo() will give you 4. foo(3) is 9 and foo(99,0) is 1. In python we can do the same thing, but it'll pay to use some better variable names:

def foo(base=2, exponent=2):
    return base**exponent

Similarly foo() will give you 4. foo(3) is 9 and foo(99,0) is 1. But what if we forgot what the order was? Did base come first or was it exponent? We can do this:

foo(exponent=99, base=2)

Since base and exponent both have default values, we can even omit base and let it use the default:

foo(exponent=10)

A side effect of this, is now there's a real use, even for simple functions, to give your variables easy to remember names.

Better than ORM Object Persistence

2007-12-20T00:00:00-08:00

After talking to people about the benefits and disadvantages of various ORMs... and reading up a little on non RDMBSs like Amazon SimpleDB I came to the realization that ORM is really a hack to get RDBMSs to work as a storage for objects.

I'm being liberal with the term hack. It really does work for a lot of situations, but it's not very elegant. The workflow is more or less this:

You create a database.
You create some objects
You define tables to store attributes of objects
You establish a mapping

There's a lot that goes into database definition. It would be nice to breakout from this line of thinking and do things a bit differently:

Create database
Save named (e.g. Person, Place, Log, Restaurant, Rating, Review) serialized objects to the database.

Let the database learn from the saving of the object how to define the type. In fact, it should be flexible and let us have mixmatched types, etc.

Let the database index everything, and keep the indexes up to date, but prioritize it based on usage. In other words if we usually query a database of Persons in order by a field called lastname, then make sure that index on lastname is always at hand. We should be able to query this data back out of storage based on how we stored it.

We should also be able to reach other objects in the database in a similar manner.

The key here is letting the database layer do the heavy-thinking about what to do with serialized data, versus having some middle layer take care of it.

...

so I might just be naive about data and databases. But if this idea is worthwhile and some database people can validate me, I'd be willing to work on it.

jQuery shortcut functions and jQuery plugins

2007-09-15T00:00:00-07:00

[tags]js, javascript, readability, yui, jquery, shortcuts[/tags]

One of the the things that's kept me away from the YUI javascript libraries is longhand. Or rather, the appeal to jQuery (and Prototype) was the almight $ shorthand.

Of course there are times when it pays to write in long hand. Marc Grabanski has written a fantastic calendar plugin which uses a lot of jQuery shorthand. $(), $.extend and whatnot are great for quick code, but disaster when you need to be in maximum compatibility mode. For example, use jQuery with some other libraries and you can get disaster. It's better to opt for the long-hand jQuery and jQuery.extend methods. After some search and replaces, I may have gotten things right, but I think I need to talk to Marc Grabanski and make sure the changes I made make sense.

So remember, long hand is great when other developers are using stuff and namespace conflicts are afoot ;)

Symfony Camp: Ajax and Zend, what would you like to know?

2007-08-16T00:00:00-07:00

[tags]symfonyCamp, symfony, netherlands, ajax, zend search lucene, zsl, jquery[/tags]

I've been asked to speak at SymfonyCamp (symfony['camp']) next month (you should all go if you can) and I thought I'd present as well as I could on Ajax and the Zend Framework Bridge (including Zend Search Lucene).

If you're attending the camp and/or would like to hear about these topics please let me know any specific questions you might have about "symfony and Ajax" and "symfony and Zend" and I'll try to address them in my presentations.

If you are unable to go fear not, I'll try to post my notes on this site.

Using sfDoctrine to match allowed email domains

2007-07-09T00:00:00-07:00

[tags]php, propel, doctrine, validators, symfony, optiopt, startup[/tags]

I'm a co-founder at an online finance web site and I'm in charge with building out the site. Our rollout strategy is to let a a handful of companies at a time, so we're limiting registration based on your company's email address.

I decided to follow the bandwagon and use PHP Doctrine. We'll define two objects, Company and CompanyDomain:

Company:
  tableName: company
  columns:
    name: {type: string(100)}
    created_at: timestamp
    updated_at: timestamp

CompanyDomain:
  tableName: company_domain
  columns:
    company_id:
      foreignClass: Company
      cascadeDelete: true
    pattern: {type: string(100)}

Where a CompanyDomain represents a domain name for a company. E.g. Motorola might have both motorola.com and mot.com.

We can validate a signup form and see if we've got an email address from a domain we recognize. I like using the validationXXX methods in an action class for specific validation. I made one called validateSignup:

public function validateSignup()
{
    if ($this->isPost())
    {
        $email = $this->getRequestParameter('company_email');
        if (!$company = CompanyTable::match($email))
        {
            $this->getRequest()->setError('company_email', 'Your work email doesn\'t appear to belong to any of the registered companies.');
            return false;
        }
        return true;
    }
}

In our CompanyTable we create a static function CompanyTable::match:

static public function match($email)
{
    $company = Doctrine_Query::create()->from('Company c, c.CompanyDomains d')->where('? LIKE CONCAT(\'%\', pattern)', $email)->execute()->getFirst();

    return $company;
}

Note that we transparently do a join in the ->from() statement. If there's a match we get a company object (which we can associate with the new user) otherwise we get a null object.

Enjoy.

Boosting terms in Zend Search Lucene

2007-05-29T00:00:00-07:00

[tags]Zend, Zend Search Lucene, Search, Lucene, php, symfony, zsl[/tags]

Boosting terms — some fields are better than others

Lucene supports boosting or weighting terms. For example, if I search for members of a web site, and I type in Dash, I want people with the name Dash to take precendence over somebody who has a hobby of running the 50-yard Dash.

If we look at our generateZSLDocument() method we defined we just need to adjust a few lines:

Should be turned into:

This is pretty straight forward way to add weight (1.5 times the weight of a normal term) and you can customize it to the needs of your site.

Finding things using Zend Search Lucene in symfony

2007-05-23T00:00:00-07:00

[tags]Zend, Zend Search Lucene, Search, Lucene, php, symfony, zsl[/tags]

This is part of an on going series about the Zend Search Lucene libraries and symfony. We'll pretty everything up when we're done =)

We now know how to manipulate the index via our model classes. But let's actually do something useful with our search engine... let's search!

[tags]Zend, Zend Search Lucene, Search, Lucene, php, symfony, zsl[/tags]

At the time of this writing we're dealing with Propel which uses Peer classes which are meant for dealing with multiple objects¹. This is the perfect place for a ::search() method. In otherwords, UserPeer::search('dave'); should query Lucene for users matching "dave". Let's make that happen:

What we're doing is retrieving our Lucene index. Somewhere between tutorials we wrote this Peer function to handle that:

If our index is missing we'll conveniently create it on the fly. We then use the Zend Search Lucene API to retrieve the matching hits in this index and then use some Propel trickery to retrieve by an array of primary keys.

It's now simple to use ::search() functions in the same manner as you use ::doSelect().

At this point you should be able to create a basic symfony app that can utilize a Lucene index.

The examples refer to using Propel, but it's trivial to adapt this to sfDoctrine ↩

Equal height columns with jQuery

2007-05-22T00:00:00-07:00

[tags]css, jQuery, layout, javascript, equal, columns, equal columns[/tags]

I've seen a few examples of how to equalize column heights using javascript, and none of them seem appealing:

jquery.equalizecols.js
- This required a few other libraries, and I wanted more flexibility (e.g. where the column should grow in order to equalize)
Project 7
- The Project 7 approach was the most interesting, but the code seemed a bit messy and not so open source friendly (even thought it might have been). It would let you specify which element was to grow inside a column.
Nifty Corners
- I had trouble with the syntax, but I liked how it just created a new element out of thin air...

So I wrote my own:

$("#col1, #col2").equalizeCols();

will equalize the columns as expected

$("#col1, #col2").equalizeCols("p,p");

will equalize the columns and add the extra space after the p tag in #col1 or #col2 (whichever is shorter).

Here's our function:

This requires jQuery of course, and it hasn't been tested much.

Debugging yaml configuration with the symfony web debugger

2007-05-21T00:00:00-07:00

[tags]symfony, yaml, configuration, web debug, debug[/tags] There's no doubt that yaml configuration files in symfony can be troublesome to debug.

The symfony web debugger comes to the rescue.

Let's say you're app.yml has the following:

app:
  admin:
    email: me@example.com

In your development environment you can look at click on "vars & config" and under settings see all the sfConfig::getable variables. You'll find the above listed as:

app_admin_email: me@example.com

This technique is very handy when trying to figure out what key you need to use in sfConfig::get().

The Lucene Search Index and symfony

2007-04-23T00:00:00-07:00

[tags]Zend, Zend Search Lucene, Search, Lucene, php, symfony, zsl, index[/tags]

This article is meant to followup sfZendPlugin where we learn a newer way of obtaining the Zend Framework.

In this tutorial we're going to delve into the Lucene index. Zend Search Lucene relies on building a Lucene index. This is a directory that contains files that can be indexed and queried by Lucene or other ports. In our example we'll be creating a search for user profiles.

We'll want to store in our app.yml the precise location of this index file so we can refer to it in our app¹.

Here's an example:

all:
  search:
    user_index: /tmp/myapp.user.lucene.index

Now when we need to refer to the index we can do sfConfig::get('app_search_user_index').

Index Something

Let's try a user search where we can find a user by their name or email address. It's fairly simple to accomplish, and hardly requires the use of ZSL, but by using ZSL we can easily extend it to do a full-text search of a user's profile or any other textual data.

Each "thing" stored in the index is a Lucene "document". Each document then consists of several "fields" (Zend_Search_Lucene_Field objects). In our example, each document will be an individual user and the fields will be relevant attributes of the user (username, first name, last name, email, the text of their profile).

Initially we'll want to populate our index. We may also want to regularly reindex all the users at once to optimize the search performance. Since reindexing involves multiple users it would make sense to have a static reindex method in our UserPeer class².

Very simply, we're creating a new index, getting all the users, adding a document to the index and then committing the index (to disk). You might have noticed that there's a strange function, User::generateZSLDocument(). This function contains all the magic. In order to not repeat ourselves we keep the internals of making a document for the Lucene index in the User class itself. Let's look at it:

We're really just dumping the relevant search terms into this document. The beauty of keeping this code internalized in the User class is we can reuse it later if we need to index a single User at a time.

A couple things to note. Zend_Search_Lucene_Field::Keyword allows us to store data that we can lookup later. We store the User::id in a field called uid since id is a reserved word for the index and we can't access it from Zend Search Lucene.

In a batch script or a reindex action we can now just call UserPeer::reindex() and have a working search index for our users.

Storing things in app.yml is great for indexes that don't need to be searched in multiple applications. ↩
Since we're using a Lucene index, which has an open documented structure, we aren't limited to just using Zend Search Lucene or Apache Lucene (java). We can mix and match and read and write to the same index file. For very large indexes (65,000+ documents), I rewrote a Java application to index all the documents at once as PHP would time out during such a task. ↩

Tips for symfony and Subversion

2007-04-17T00:00:00-07:00

[tags]symfony, subversion[/tags]

There's some tricks you can do to running a symfony project with subversion:

Ignoring files in `cache/` and `log/`

The first thing you can do (and this is well documented in the askeet tutorial) is ignore files in cache/ and log/. These files are specific to each instance of your app and don't contain anything that needs to be in version control.

Run the following:

cd $SF_ROOT
rm -rf log/* cache/*
svn propedit svn:ignore log
svn propedit svn:ignore cache

svn propedit will bring up a text editor, in both instances you want to save the following:

Ignore other auto-generated files

Eric Sink wrote an excellent tutorial on source control. In his chapter on repositories he recommends checking in only hand edited source code. If a property file generates another file, check in the property file, not the auto-generated result. This not only keeps your repository clean, it prevents a lot of unnecessary check-ins.

If you use propel for your ORM layer there are a few files you can ignore using svn propedit svn:ignore {dirname}.

In $SF_ROOT/config we can ignore:

*schema-transformed.xml

These are xml files that propel generates from schema.xml (or schema.yml).

In $SF_ROOT/data/sql we can ignore:

lib.model.schema.sql
plugins.*.sql
sqldb.map

These are created from schema.xml (or schema.yml) as well.

The real savings will come with your model. The propel model creates customizable php classes in lib/model which inherit from auto-generated files in lib/om there are also auto-generated map files in `lib/map'

We can run from $SF_ROOT:

svn propedit svn:ignore lib/model/om
svn propedit svn:ignore lib/model/map

and enter

for both properties.

If you've mistakenly checked in some of these files you will need to remove them from your repository via svn delete.

Linking the symfony Library

I prefer to embed the symfony library into each of my symfony apps rather than relying on a shared PEAR library. This lets me run multiple versions of symfony without much fuss. With subversion we can use the svn:externals property to directly link our app with the symfony subversion repository.

At first this sounds like danger, but externals can be linked to specific revisions. However, the symfony team tags their repository with version numbers. To get this to work we need to do 3 things. (UPDATE: See Fabien's comment about using the lib/vendor directory)

Modify config/config.php to look for symfony internally. Just open it up and change it so it says this:
<?php $sf_symfony_lib_dir = dirname(<strong>FILE</strong>).'/../lib/symfony'; $sf_symfony_data_dir = dirname(<strong>FILE</strong>).'/../data/symfony';
Run svn propedit svn:externals lib from $SF_ROOT and enter:
```
 symfony http://svn.symfony-project.com/tags/RELEASE_1_0_2/lib/
```
or whatever version of symfony you want to link to, at the time of this post, RELEASE_1_0_2 is fairly fresh.
Run svn propedit svn:externals data from $SF_ROOT and enter:
```
 symfony http://svn.symfony-project.com/tags/RELEASE_1_0_2/data/
```
or whatever version of symfony you want to link to, at the time of this post.

Now when you do svn update you'll have the symfony library all linked up. Furthermore this keeps all the developers on the same version of symfony.

Also you may want to start running symfony using ./symfony versus symfony. The former looks at your configuration settings to determine which symfony command to use, the latter is generally linked to your system wide command (which is generally the PEAR installed command).

Linking to symfony Plugins

I have my hands in a number of symfony plugins because I work on a lot of projects which tend to share a lot of similar functionality. Many of the plugins are in early stages of development, so I find it helpful to have them linked from svn as well. This way I can get the benefits of any new functionality and if the occasion should arise, I can contribute any useful changes I make.

To link to the plugins you run svn propedit svn:externals plugins and enter one plugin per line in the following format:

{plugin_name} -r{REVISION} {URL}

For one of my projects I use:

sfPrototypeTooltipPlugin http://svn.symfony-project.com/plugins/sfPrototypeTooltipPlugin
sfGuardPlugin http://svn.symfony-project.com/plugins/sfGuardPlugin
sfZendPlugin http://svn.symfony-project.com/plugins/sfZendPlugin

I've omitted the revision, because I live dangerously and want to use the latest $HEAD.

sfZendPlugin

2007-04-10T00:00:00-07:00

[tags]Zend, Zend Search Lucene, Search, Lucene, php, symfony, zsl, plugins[/tags]

I originally intended to rewrite my Zend Search Lucene tutorial, but Peter Van Garderen covered the bulk of what's changed and I was too busy developing search functionality for lyro.com (not to mention finding inconsistencies with the Zend Search Lucene port and Lucene) to finish the tutorial. So I broke it up into smaller pieces.

I packaged Zend Framework into a symfony plugin. symfony is easily extended using plugins.

You can obtain this from subversion with the following command (from your /plugins directory):

svn export http://svn.symfony-project.com/plugins/sfZendPlugin

symfony has a Zend Framework Bridge which let's us autoload the framework by adding the following to settings.yml:

.settings:
  zend_lib_dir:   %SF_ROOT_DIR%/plugins/sfZendPlugin/lib
  autoloading_functions:
    - [sfZendFrameworkBridge, autoload]

First we define sf_zend_lib_dir to be in our plugin's lib directory. Then we autoload the bridge framework.

After setting this up, all the Zend classes will be available and auto-loaded from elsewhere in your code.

Caching REST with sfFunctionCache

2007-04-07T00:00:00-07:00

[tags]geocoding, caching, REST, symfony, Cache_Lite, php, cache, sfFunctionCache[/tags]

For reviewsby.us we do a lot of geocoding. To facilitate we use Yahoo! Geocoding API. This helps us normalize data, obtain latitude and longitude, interpret location specific searches.

These REST queries happen a lot and will continue to happen, but this data that Yahoo! provides is fairly static. We're basically querying a database of sorts. So it makes sense that we should cache this data.

We'll demonstrate how to cache these queries using symfony's sfFunctionCache class.

I wrote a wrapper (I'll release it as a plugin if requested) for the Geocoding API, the bulk of the work (the REST call) occurs in a function called doQueryGIS:

The call to this function is always wrapped with queryGIS:

This wrapper creates a sfFunctionCache objet and calls the function and caches it for subsequent queries.

What this means is once Yahoo! teaches reviewsby.us that India is located at (25.42°, 77.830002°) and that the precision is 'country' we remember it in the future.

These features will be incorporated into future versions of reviewsby.us.

PHP double versus single quotes

2007-03-03T00:00:00-08:00

[tags]best practices, php[/tags]

If you're familiar with PHP, the difference between double and single quotes is obvious. Double quotes evaluate variables and control characters (e.g. \n or \r), whereas single quotes do not.

I've been indoctrinated with the "use single quotes whenever possible" methodology, but I never really put it to the test. Is it really worth it for me to go back and look at old code that uses double quotes and change them? Like all best practices, the answer is "maybe."

So I wrote a simple test harness (there's a lot of room for error with these, but I tried my best).

I tried different permutations of which order to run f1(), f2() and f3() they seemed to give similar results no matter which order they were run.

My results were:

Time 1: 2.898087978363
Time 2: 3.5480048656464
Time 3: 2.8503499031067

My interpretation is that quotes versus double quotes isn't as big of a deal as concatenation versus putting variables within double quotes. My guess is that PHP 5.2.0 (which is what I used for the tests) is smarter with strings than older versions.

So really if you look at the test harness there isn't any discernible differences until you hit 2 million iterations and even then nobody gets fired over 0.6 seconds of performance. Chances are it doesn't matter too much, but over time you can write enough code in the right spots or shared in the right open source projects and that few seconds will snowball. After all, I'm not so much concerned about how optimized a script is that I write, but rather how optimized is a script that everyone ends up using. But rather than do mental calculations about whether or not to optimize something, let's just assume that everything should be as optimal as we can stand to write it.

Update: 6 March 2006, I updated the test harness to reflect my intended tests. Update: 7 March 2006, I updated the results to be more clear.

How do you find good programmers?

2007-03-01T00:00:00-08:00

[tags]programmers, hiring[/tags]

I have a hard time finding good programmers. Usually I'm asked to look for one in the most inopportune times. Namely when we're on a tight deadline and it's really too late to add new people. Unfortunately, it looks like I'm not alone in learning that the programmers for hire aren't up to snuff.

Looking for a PHP developer is worse, because there is such a low barrier to entry. If you look for a Java developer or a C/C++ developer there's some reasonable assurance that they either a. learned it very well or b. was taught it formally. With PHP the barrier to entry is so low that anybody can claim to be a PHP developer. I can safely attest that if you embark on a quest to learn more about PHP or even programming in general there's a lot more out there.¹

I think the best situation is to interview for people during your downtime and keep the communication open with them. In my situation I know more positions that are open that require talented developers versus the number of talented developers I know. So even if I can't use a person when they happen to be free, I can at least pass them on good leads.

Early last month I set on a quest to learn more about PHP and learned a lot about design patterns and relating them to PHP. This subsequently highlighted how much more I don't know yet. ↩

Parsing a list of Key:Value pairs

2007-02-24T00:00:00-08:00

[tags]best practices,php,openID[/tags] I'm working on implementing openID for reviewsby.us and for use in symfony apps. One thing I was having trouble with was parsing key value pairs, which is one of the requirements to reading responses. It's a fairly easy task, but PHP offers so many ways to do this.

openID calls for the following Key-Value format:

Lines of:

some_key:some value

There MUST NOT be a space before or after the colon.

Newline characters MUST be Unix-style, just ASCII character 10 ("\n").

Newlines MUST BE at end of each line as well as between lines.

MIME type is unspecified, but text/plain is RECOMMENDED.

Character encoding MUST BE UTF-8.

So here is my attempt at parsing something like this as efficiently and error free as possible:

I wrote this function as I was writing the post... and it came out way faster than my previous implementations using strtok or a combination of explode and trim. Which is good to hear since I do use preg_ functions quite a bit in PHP and they definitely have their place.

I'm curious if people have found a faster way of parsing through a string of Key-Value pairs. I'll run it in a test harness and stand corrected if someone comes through with something faster ;).

Make Adium show your Facebook status

2007-02-03T00:00:00-08:00

By using a simple RSS feed, you can have Adium report your Facebook status.

I am a late-adopter to social networks. I participated in LiveJournal years after many of my friends started using it. I finally got into Facebook months after it was "openned up." I like this strategy because I can immediately find my friends on the network and add them and amass 100s in a few short days.

Facebook is nice for its simple and clean interface. Even the features are fairly simple. I like the status feature... for no good reason. It's a simple AJAX status update that updates your status on the site. What I don't like to do is repeat myself. I wanted Adium to repeat what Facebook said.

My first inkling was to use the Facebook API, but that seemed less appealing since it would involve authentication. I almost gave up and then discovered that in Facebook you can go to "My Profile" and under "Status" go to "See All" and there's an RSS link to your status messages. Awesome. All I needed to do is grab the first one.

I decided this looked like a task for Simple XML which can take an XML string and turn it into a PHP object.

The first step of course was to fetch the RSS feed into a string. file_get_contents made the most sense, but Facebook does a browser check even for getting RSS feeds. I'm not fond of browser sniffing, but its easy to work around if you use cURL.

Excellent... this does exactly what I want - but it's slow. Like any good script, we should use caching if it makes sense. Obviously we don't want to overload the servers at Facebook (or for that matter any place that serves up RSS feeds) so we implement PEAR's Cache_Lite package. We'll tell it to cache results for 15 minutes. The code changes are marked with //+++ at the end of each new line:

<?php

require_once('Cache/Lite.php'); //+++
    
    $url = "http://mynework.facebook.com/feeds/status.php?replace=with&your=own&feed=url";

$options = array('cacheDir' => '/tmp/', 'lifeTime' => 600); //+++

$Cache_Lite = new Cache_Lite($options); //+++

$data = null; //+++
    // attempt to load the data from cache,  //+++
    // otherwise load it anew from RSS //+++
    if (!($data = $Cache_Lite->get($id))) { //+++
    // setup curl
        $ch = curl_init();
        curl_setopt ($ch, CURLOPT_URL, $url);
        curl_setopt ($ch, CURLOPT_HEADER, 0);

//spoof Firefox
        curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.1) Gecko/20061223 Firefox/2.0.0.1");

// begin output buffering
        ob_start();
        curl_exec ($ch);
        curl_close ($ch);
        // save buffer to string
        $xmlstr = ob_get_contents();

ob_end_clean();

// convert string to xml object
        $xml = new SimpleXMLElement($xmlstr);

// status messages start with 'Dave is...' 
        // I just want everything after my name
        $data = str_replace('Dave ', null,$xml->channel->item->title); // +++

$Cache_Lite->save($data); //+++

} //+++

echo $data; //+++

That's it on the PHP side. On the Adium side you'll need to use the exec AdiumXtra. It'll allow you to set your status to something like:

/exec {/usr/local/php5/bin/php /usr/local/bin/facebook.php}

Enjoy.

Make Adium show your Facebook status

2007-02-03T00:00:00-08:00

By using a simple RSS feed, you can have Adium report your Facebook status.

I decided this looked like a task for Simple XML which can take an XML string and turn it into a PHP object.

<?php

require_once('Cache/Lite.php'); //+++
    
    $url = "http://mynework.facebook.com/feeds/status.php?replace=with&your=own&feed=url";

$options = array('cacheDir' => '/tmp/', 'lifeTime' => 600); //+++

$Cache_Lite = new Cache_Lite($options); //+++

//spoof Firefox
        curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.1) Gecko/20061223 Firefox/2.0.0.1");

// begin output buffering
        ob_start();
        curl_exec ($ch);
        curl_close ($ch);
        // save buffer to string
        $xmlstr = ob_get_contents();

ob_end_clean();

// convert string to xml object
        $xml = new SimpleXMLElement($xmlstr);

// status messages start with 'Dave is...' 
        // I just want everything after my name
        $data = str_replace('Dave ', null,$xml->channel->item->title); // +++

$Cache_Lite->save($data); //+++

} //+++

echo $data; //+++

That's it on the PHP side. On the Adium side you'll need to use the exec AdiumXtra. It'll allow you to set your status to something like:

/exec {/usr/local/php5/bin/php /usr/local/bin/facebook.php}

Enjoy.

Not taking frameworks for granted

2006-11-21T00:00:00-08:00

One of my clients approached me with a relatively easy project. She gave me a log file of PHP errors and I was supposed to fix her scripts. I fixed about 100+ different errors in a few hours. It was fairly straightforward.

Throughout the site I could understand the previous developer and the choices he or she made for better or worse. It did look like a struggle however.

Ultimately it felt that there were some fairly simple things that each script needed to do, but each task was a challenge. Validating forms, storing data across pages, decorating the site, interacting with the database. Everything seemed very kludged together.

I realized that this is exactly how I used to write code, sure... my way of course was better and more logical, yada, yada, yada... but ultimately I was there before.

My client noted that these scripts were made from another contract programmer, and then a light-bulb went on... frameworks (whether it be symfony, ROR, Django, CakePHP, etc) help iron out and standardize these tasks.

Since I know symfony best, I'll cover what I think could have helped in this last project. I'm sure other major frameworks have their equivalents.

Form validation: symfony lets you define form validation in a very simple manner. The validation logic is also separate from the rest of the code.
Storing Data: Without a framework, you generally have to rely on the $_SESSION array in PHP. While very useful and easy to use, storing parameters and attributes to a user object is done a lot more cleanly.
decorating the site: My biggest problem was with each page I had,, I used to have to call headers, sidebars, etc, etc. symfony's layout system was a boon. I had a common layout for all pages (maybe a few alternates) and hooks inside them if they needed to be adjusted. Then the various actions had their own seperate templates that were injected into the common layout. It made adding new pages easy, since I didn't need to remember header() and footer() functions for each and every page.
Interacting with the database: I've covered before the benefits of ORM.

Not only do the bulk of these problems disappear with a framework, a lot of the difficulties of switching developers melt away. If you tell me, a developer, that I'm walking into a project made with a framework, I can learn about the framework and be able to understand its ins and outs.

If you just tell me it's written in PHP, chances are I'm going to want to do things my own way. It's hard to understand the logic that another programmer was using so we fall back to standards whether they are your own or borrowed.

When we use a framework, we can find some mutually agreed upon standards and usually people who specialize in that framework and are willing to help. So my advice: stick to frameworks. The coding style will be no worse than the whims of a programmer, but at best it'll be something that anyone can pick up. The general case is that even a bad coder can only do so much damage within a framework. The bullet points I covered above will cut down on development time tremendously.

How to make Prototype Window Class as easy as Lightbox Gone Wild

2006-11-13T00:00:00-08:00

I like the way that Lightbox Gone Wild will automatically pickup any links with the class=lbOn, but I wanted to use (at some point) Prototype Window Class instead.

Luckily PWC is built on Prototype which means we've already loaded a helpful library.

In order to take all class=lbOn objects and run them through PWC we just write a simple loop and iterate.

So here's the low-down:

Download PWC
Copy window.js
Use the included prototype & script.aculo.us if you don't already include it in your page
copy any themes you wish to use.

In your page add this bit of JavaScript:

So, this code simply looks for all the anchor tags with class=lbOn and then creates a new mylb instance for each anchor. The end.

Cropping Images using DHTML (Prototype) and symfony

2006-09-16T00:00:00-07:00

Note: Like many of my tutorials, you don't need symfony, just PHP. However, I develop in symfony and take advantage of the MVC-support that it offers.

Years ago when I was working on a photo gallery for davedash.com I got the art of making tumbnails down fairly well. It was automated and didn't allow for specifying how the thumbnail should be made. With dozens of photos (which was a lot back then), when would I find that kind of time.

Flashback to today, for my company... we want users with avatars... but nothing too large. Maybe a nice 80x80 picture. Well the coolest UI I've seen was Apple's Address Book which let you use this slider mechanism to crop a fixed sized image from a larger image.

Here's a demo.

Overview

The front-end GUI is based on code from digg which is based on the look and feel (as near as I can tell) from Apple.

The GUI provides a clever visual way of telling the server how to chop the image. The gist is this, sliding the image around and zooming in and out change a few form values that get passed to another script which uses this data to produce the image.

Frontend: What would you like to crop?

In this tutorial, we're going to be cropping an 80x80 avatar from an uploaded image. The front-end requires the correct mix of Javascript, CSS, HTML and images. The Javascript sets up the initial placements of the image and the controls. The CSS presents some necessary styling. The images makeup some of the controls. The HTML glues everything together.

HTML

Let's work on our HTML first. Since I used symfony, I created a crop action for a userpics module. So in our cropSuccess.php template:

<div id="ava">
    <?php echo form_tag("userpics/crop") ?>
        <div id="ava_img">
            <div id="ava_overlay"></div>
            <div id="ava_drager"></div>
            <img src="<?php echo $image ?>" id="avatar" />
        </div>
        <div id="ava_slider"><div id="ava_handle"></div></div>
        <input type="hidden" id="ava_width" name="width" value="80" />
        <input type="hidden" id="ava_x" name="x" value="100" />
        <input type="hidden" id="ava_y" name="y" value="100" />
        <input type="hidden" id="ava_image" name="file" value="<?php echo $image ?>" />
        </div>
        <input type="submit" name="submit" id="ava_submit" value="Crop" style="width: auto; font-size: 105%; font-weight: bold; margin: 1em 0;" />
    </form>
</div>

Right now a lot of this doesn't quite make sense. If you attempt to render it, you will just see only the image. As we add the corresponding CSS and images it will make some more sense.

CSS and corresponding images

We'll go through each style individually and explain what purpose it serves in terms of the GUI.

#ava is our container.

#ava {
    border: 1px solid gray;
     width: 200px;
}

#ava_img is the area that contains our image. Our window for editing this image 200x200 pixels. If we drag out image out of bounds we just want the overflowing image to be clipped. We also want our position to be relative so any child elements can be positioned absolutely with respect to #ava_img.

#ava_img {
    width: 200px;
    height: 200px;
    overflow: hidden;
    position: relative;
}

#ava_overlay is a window we use to see what exactly will be our avatar. If it's in the small 80x80 window in the center of the image, then it's part of the avatar. If it's in the fuzzy region, then it's getting cropped out. This overlay of course needs to be positioned absolutely.

#ava_overlay {
    width: 200px;
    height: 200px;
    position: absolute;
    top: 0px;
    left: 0px;
    background: url('/images/overlay.png');
    z-index: 50;
}

#ava_drager is probably the least intuitive element (Heck, I'm not even sure if I've even got it right). In our demo you're not actually dragging the image, because you can drag anywhere within the #ava_img container and move the image around. You're using dragging an invisible handle. It's a 400x400 pixel square that can be dragged all over the container and thusly move the image as needed.

#ava_drager {
    width: 400px;
    height: 400px;
    position: absolute;
    z-index: 100;
    color: #fff;
    cursor: move;
}

#avatar is our image, and since it will be moving all around the window, it requires absolute positioning.

#avatar {
    position: absolute;
}

#ava_slider and #ava_handle are our slider components. They should be self-explanatory.

#ava_slider {
    width: 200px;
    height: 27px;
    background: #eee;
    position: relative;
    border-top: 1px solid gray; 
    background: url('/images/slider_back.png');
}
#ava_handle {
    width: 19px;
    height: 20px;
    background: blue;
    position: absolute;
    background: url('/images/handle.png');
}

Internet Explorer

PNG do not work so well in Internet Explorer, but there is a small trick, adding these components into a style sheet that only IE can read will make things work:

#ava_overlay {
  background: none;
  filter:progid:DXImageTransform.Microsoft.AlphaImageLoader(src='/images/ui/cropper/overlay.png', sizingMethod='crop');
}

#ava_handle {
  background: none;
  filter:progid:DXImageTransform.Microsoft.AlphaImageLoader(src='/images/ui/cropper/handle.png', sizingMethod='crop');
}

The Javascript

The Javascript is actually not as complicated as you'd expect thanks to the wonder of prototype. This framework provides so much so easily. You'll need to include prototype.js and dom-drag.js.

So let's take a look.

<script type="text/javascript" language="javascript" charset="utf-8">
// <![CDATA[
function setupAva() {
    if ($("avatar")) {
        var handle = $("ava_handle");
        var avatar = $("avatar");
        var drager = $("ava_drager");
        var slider = $("ava_slider");
        var ava_width = $("ava_width");
        var ava_x = $("ava_x");
        var ava_y = $("ava_y");
        // four numbers are minx, maxx, miny, maxy
        Drag.init(handle, null, 0, 134, 0, 0);
        Drag.init(drager, avatar, -100, 350, -100, 350);
        var start_w = avatar.width;
        var start_h = avatar.height;
        var ratio = (start_h / start_w);
        var new_h;
        var new_w;
        if (ratio > 1) {
            new_w = 80;
            new_h = (80*start_h)/start_w;
        } else {
            new_h = 80;
            new_w = (80*start_w)/start_h;
        }
        // these need to be set after we init
        avatar.style.top = '100px';
        avatar.style.left = '100px';
        avatar.style.width = new_w + 'px';
        avatar.style.height = new_h + 'px';
        avatar.style.margin = '-' + (new_h / 2) + 'px 0 0 -' + (new_w / 2) + 'px';
        handle.style.margin = '3px 0 0 20px';
        avatar.onDrag = function(x, y) {
            ava_x.value = x;
            ava_y.value = y;
        }
        handle.onDrag = function(x, y) {
            var n_width = (new_w + (x * 2));
            var n_height = (new_h + ((x * 2) * ratio));         
            avatar.style.width = n_width + 'px';
            avatar.style.height = n_height+ 'px';
            ava_width.value = n_width;  
            avatar.style.margin = '-' + (n_height / 2) + 'px 0 0 -' + (n_width / 2) + 'px';
        }
    }
}
Event.observe(window,'load',setupAva, false);
// ]]>
</script>

If this isn't exactly crystal clear, I can explain. If you're new to prototype, $() is the same as doucment.getElementByID() (at least for our purposes).

We need to initialize two draggable elements, one is our slider for zooming and the other is our avatar itself. We initialize the draggers using Drag.init(). We specify what to drag, if another element should be used as a handle and then the range of motion in xy coordinates. In the second call we use that #dragger to move around the image in this manner.

Drag.init(handle, null, 0, 134, 0, 0);
Drag.init(drager, avatar, -100, 350, -100, 350);

We want to initialize the the size and placement of the avatar. We do that using maths. First we want it in our 80x80 pixel box. So it should be roughly 80x80. I've set the math up so that the smallest side is 80 pixels (there's reasons for doing this the other way around).

    if (ratio > 1) {
        new_w = 80;
        new_h = (80*start_h)/start_w;
    } else {
        new_h = 80;
        new_w = (80*start_w)/start_h;
    }

We then place the avatar element. We initialize it to be in the center of the screen (top: 100px;left:100px) and then nudge the image using margins.

    avatar.style.top = '100px';
    avatar.style.left = '100px';
    avatar.style.width = new_w + 'px';
    avatar.style.height = new_h + 'px';
    avatar.style.margin = '-' + (new_h / 2) + 'px 0 0 -' + (new_w / 2) + 'px';

We also use margins to place the handle.

    handle.style.margin = '3px 0 0 20px';

#ava_x and #ava_y tell us where the center of the avatar is. So when the avatar is moved we need to set these again:

    avatar.onDrag = function(x, y) {
        ava_x.value = x;
        ava_y.value = y;
    }

That was easy. Slighly more complicated is the zoomer function. We are basically adjusting the width and the height proportionately based on roughly where the slider is. Note that we're still using that ratio variable that we calculated earlier. We basically take the new x-coordinate of the handle and allow our image to get just slightly larger than the #ava_image container.

    handle.onDrag = function(x, y) {
        var n_width = (new_w + (x * 2));
        var n_height = (new_h + ((x * 2) * ratio));         
        avatar.style.width = n_width + 'px';
        avatar.style.height = n_height+ 'px';
        ava_width.value = n_width;  
        avatar.style.margin = '-' + (n_height / 2) + 'px 0 0 -' + (n_width / 2) + 'px';
    }

We want to load initialize the slider right away when the page loads: Event.observe(window,'load',setupAva, false);

Not terribly hard or complicated. Once these elements are all in place you have a working functioning slider. It returns the x and y coordinates of the center of the image with respect to our 200x200 pixel #ava_image. It also tells us the new width of our image. We feed this information into a new script and out should pop a new image which matches exactly what we see in our GUI.

Processing the crop

Initially I was frustrated with the data that was being sent. I knew the center of the image in relation to this 200x200 pixel canvas and its width... but what could I do with that. Well I could just recreate what I saw in the GUI. I needed to create a 200x200 pixel image first, place my original avatar resized (and resampled) at the precise coordinates and then cut out the center most 80x80 pixels to become the final avatar image.

If you note in our template above for cropSuccess.php we submit our form back to the crop action. Let's look at the action:

public function executeCrop()
{
    if ($this->getRequestParameter('file')&&$this->getRequestParameter('width')) {          // we are saving our cropped image
        // Load the original avatar into a GD image so we can manipulate it with GD
        $o_filename = $this->getRequestParameter('file');  // we'll use this to find the file on our system
        $o_filename = sfConfig::get('sf_root_dir').'/web' . $o_filename;
        $o_im = @imagecreatetruecolor(80, 80) or die("Cannot Initialize new GD image stream");
        $o_imagetype = exif_imagetype($o_filename); // is this gif/jpeg/png

        // appropriately create the GD image
        switch ($o_imagetype) {
            case 1: // gif
                $o_im = imagecreatefromgif($o_filename);
                break;
            case 2: // jpeg
                $o_im = imagecreatefromjpeg($o_filename);   
                break;
            case 3: // png
                $o_im = imagecreatefrompng($o_filename);
                break;
        }

        // Let's create our canvas
        $im = @imagecreatetruecolor(200, 200) or die("Cannot Initialize new GD image stream");
        imagecolortransparent ( $im, 127 ); // set the transparency color to 127
        imagefilledrectangle( $im, 0, 0, 200, 200, 127 ); // fill the canvas with a transparent rectangle

        // let's get the new dimension for our image

        $new_width = $this->getRequestParameter('width');
        $o_width = imageSX($o_im);
        $o_height = imageSY($o_im);

        $new_height = $o_height/$o_width * $new_width;

        // we place the image at the xy coordinate and then shift it so that the image is now centered at the xy coordinate
        $x = $this->getRequestParameter('x') - $new_width/2;
        $y = $this->getRequestParameter('y') - $new_height/2;

        // copy the original image resized and resampled onto the canvas
        imagecopyresampled($im,$o_im,$x,$y,0,0,$new_width,$new_height,$o_width,$o_height); 
        imagedestroy($o_im);

        // $final will be our final image, we will chop $im and take out the 80x80 center
        $final = @imagecreatetruecolor(80, 80) or die("Cannot Initialize new GD image stream");
        imagecolortransparent ( $final, 127 ); // maintain transparency

        //copy the center of our original image and store it here
        imagecopyresampled ( $final, $im, 0, 0, 60, 60, 80, 80, 80, 80 );
        imagedestroy($im);

        //save our new user pic
        $p = new Userpic();
        $p->setUser($this->getUser()->getUser());
        $p->setGD2($final);
        $p->save();
        imagedestroy($final);
        $this->userpic = $p;
        return "Finished";
    }


    $this->getResponse()->addJavascript("dom-drag");
    $this->getResponse()->addJavascript('/sf/js/prototype/prototype');
    $this->getResponse()->addJavascript('/sf/js/prototype/effects');
    $this->image = '/images/userpics/originals/' . $this->getRequestParameter('file');
}

It's doing exactly what the paragraph above explains when the image dimensions are given. The code is well commented so it should be easy enough to follow.

GD image functions in PHP are fairly robust and can help you do a lot of tricks with image data. Note the code to save the image, we'll cover it in detail soon.

The Model

$p = new Userpic();
$p->setUser($this->getUser()->getUser());
$p->setGD2($final);
$p->save();

First some clarification the second line. myUser::getUser() gets the User object associated with the currently logged in user. The third line, however, is where the magic happens. Before we look at it, let's have a quick look at our model:

userpic:
 _attributes: { phpName: Userpic }
 id:
 user_id:
 image: blob
 thumb: blob
 created_at:
 updated_at:

We have an image attribute and a thumb property to our Userpic object. This is where we store PNG versions of each icon and their 16x16 thumbnails respectively. We do this in Userpic::setGD2():

public function setGD2($gd2_image)
{
    //convert to PNG
    ob_start();
    imagepng($gd2_image);
    $png = ob_get_clean();
    //save 16x16
    $gd2_tn = @imagecreatetruecolor(16, 16) or die("Cannot Initialize new GD image stream");
    imagealphablending( $gd2_tn, true );
    imagecolortransparent ( $gd2_tn, 127 );

    imagecopyresampled ( $gd2_tn, $gd2_image, 0, 0, 0, 0, 16, 16, 80, 80 );
    ob_start();
    imagepng($gd2_tn);
    $tn = ob_get_clean();

    $this->setImage($png);
    $this->setThumb($tn);
}

We capture the output of the full size PNG, then we scale it again and capture the output of the thumbnail and set them.

Conclusion

When it comes to web apps, having a relatively simple GUI for people to resize images can go a long way in terms of adoption rate of avatars and custom user pictures by non technical users.

Enjoy, and if you found this useful (or better implemented it) let me know.

Using Zend Search Lucene in a symfony app

2006-08-25T00:00:00-07:00

[tags]zend, search, lucene, zend search lucene, zsl, symfony,php[/tags]

If you're like me you've probably followed the Askeet tutorial on Search in order to create a decent search engine for your web app. It's fairly straight forward, but they hinted that when Zend Search Lucene (ZSL) is released, that might be the way to go. Well we are in luck, ZSL is available, so let's just dive right in.

If you aren't using symfony have a look at this article from the Zend Developer Zone. It covers just enough to get you started. If you are using symfony, just follow along and we'll get you where you need to go.

Obtaining Zend Search Lucene

First download the Zend Framework (ZF). The Zend Framework is supposed to be fairly "easy" in terms of installation. So let's put that to the test. Open your ZF archive. Copy Zend.php and Zend/Search to your symfony project's library folder:

cp Zend.php $SF_PROJECT/lib              
mkdir $SF_PROJECT/lib/Zend
cp -r Zend/Search $SF_PROJECT/lib/Zend
cp Zend/Exception.php $SF_PROJECT/lib/Zend                 
chmod -R a+r $SF_PROJECT/lib/Zend*

Index Something

We'll deviate slightly from food themed tutorials and do something generic. Let's try a user search where we can find a user by their name or email address. It's fairly simple to accomplish, and hardly requires the use of ZSL, but by using ZSL we can easily extend it to do a full-text search of a user's profile or any other textual data.

Each "thing" stored in the index is a "document" in ZSL, specifically a Zend_Search_Lucene_Document. Each document then consists of several "fields" (Zend_Search_Lucene_Field objects). In our example, our document will be an individual user and the fields will be relevant attributes of the user (username, first name, last name, email, the text of their profile).

We're going to write a general re-indexing tool. Something that will index all users.

In our userActions class let's add the following action:

public function executeReindex()
    {
        require_once 'Zend/Search/Lucene.php';
        $index = new Zend_Search_Lucene(sfConfig::get('app_search_user_index_file'),true);
        
        $users = UserPeer::doSelect(new Criteria());
        foreach ($users AS $user)
        {
            $doc = new Zend_Search_Lucene_Document();
            $doc->addField(Zend_Search_Lucene_Field::Keyword('id', $user->getId()));
            $doc->addField(Zend_Search_Lucene_Field::Keyword('username', $user->getUsername()));
            $doc->addField(Zend_Search_Lucene_Field::Keyword('email', $user->getEmail()));
            $doc->addField(Zend_Search_Lucene_Field::Text('firstname', $user->getFirstname()));
            $doc->addField(Zend_Search_Lucene_Field::Text('lastname', $user->getLastname()));
            $doc->addField(Zend_Search_Lucene_Field::Unstored('contents', "{$user->getEmail()} {$user->getFirstname()} {$user->getLastname()} {$user->getUsername()}"));
            $index->addDocument($doc);
        }
        
        $index->commit();
    }

The code should be fairly easy to follow. First of all we're requiring the necessary libraries for Lucene. The next line we are creating the index:

app_search_user_index_file is a symfony configuration that you define in your app.yml. It defines which file you want to use for your index. /tmp/lucene.user.index works for our purposes. The second parameter tells Lucene we are creating a new index.

We then loop through all the users and for each user create a document. For all the search relevant attributes that a user might have we add a field into the document. Note the last field:

By default search is made for the "contents" field. So in this example we want people to be able to type in someone's name, email, username without having to specify what field we're searching for.

Find those users

Finding the user's is equally as straight-forward. We make a new action called search:

If we have a query, open the ZSL index (note that we only have one parameter here). Run the find method to find our query and store it to the $hits array. Note that our query was cleaned with strtolower, since ZSL is case sensitive.

The template takes care of the rest:

Fairly simple... but it could use some cleaning up (enjoy).

What about new users?

Regularly reindexing might be nice in terms of having an optimized search index, but its lousy if you want to be able to search the network immediately when new people join on. So why not automatically re-index each user every time they are created or everytime one of their indexed components is summoned?

This should be fairly simple by adding to the User class:

We have an attribute called $reindex. When it is false we don't need to worry about indexes. When something significant changes, like an update to your name or email address, then we set $reindex to true. Then when we save:

We're calling a new function called generateZSLDocument. It might look familiar:

Now, whenever a user is updated, so is our index. Additionally we can modify our reindex action:

That's a lot easier to deal with.

...and beyond

Hope this article helps some of you jumpstart your symfony apps. Really cool, easy to implement search is here. We no longer have to stick with shoddy solutions like HT://Dig or spend time rolling our own full text search, as the symfony team diligently showed us we could. But there is a lot more ground to cover. Including optimization techniques and best practices.

Let me know what you think, and if you use this in any of your apps.

Digg-style AJAX comment editing in PHP/symfony

2006-08-13T00:00:00-07:00

"Digg"-style anything can be pretty slick. The AJAX-interactions on that site make it very fun to use. It's styles have been copied everywhere, and are definitely worth copying. The latest feature that had caught my eye was the ability to edit your comments for a set time after posting them. Of course, it wasn't just the ability to edit comments, it was AJAX too and it has a timer.

This is totally something I could use on a restaurant review site. So I started on this project. It's pretty straight forward. For all of your posted comments you check if the owner of them is viewing them within 3 minutes of posting the commen. 3 minutes is usually enough time to notice you made a typo, but if you disagree I'll leave it to you to figure out how to adjust the code.

For example, I make a comment, realize I spelled something wrong and then I can click on my comment to edit it. Of course using AJAX means this all happens without having to reload the web page. Therefore the edits are seemingly quick. So let's add it to any web site.

In Place Forms

First and foremost, the ability to edit a comment means you have a form that you can use to edit and submit your changes. But rather than deal with creating a boring unAJAXy form, we'll enlist the help of script.aculo.us.

First, each comment is rendered using the following HTML and PHP:

<div class="review_block" id="comment_<?php echo $comment->getId() ?>">  
    <p class="author"><?php echo link_to_user($comment->getUser()) ?> - <?php echo $comment->getCreatedAt('%d %B %Y') ?></p>
    <div class="review_text" id="review_text_<?php echo $comment->getId()?>"><?php echo $comment->getHtmlNote() ?></div>
</div>

Note that this div and it's child div have unique ids that we can refer back to (comment_n and review_text_n where n is the id of the comment). We can use this to interact with the DOM via JavaScript. What we do is for each comment, we check if it is owned by the current visitor and if it's within our prescribed 3 minute window. We can do that with some simple PHP:

<?php if ($comment->getUser() && $comment->getUserId() == $sf_user->getId() && time() < 181 + $comment->getCreatedAt(null) ): ?>
    <script type="text/javascript">
    //<![CDATA[
        makeEditable('<?php echo $comment->getId() ?>', "<?php echo url_for($module . '/save?id=' . $comment->getId()) ?>", "<?php echo url_for('restaurantnote/show?id=' . $comment->getId() . '&mode=raw') ?>", <?php echo 181-(time() - $comment->getCreatedAt(null)) ?>);
    //]]></script>
<?php endif ?>

As you can see we run the makeEditable() function for each applicable comment. As you can guess, makeEditable() makes a comment editable. For parameters it takes the comment's id (so it can refer to it in the DOM and other background scripts). It also takes as an argument the "save" URL as well as a URL from which it can load the raw comment. The last argument is for the timer.

Here is our function:

var editor;
var pe;
makeEditable = function(id, url, textUrl, time) {
    var div = $("review_text_" + id);

    pe = new PeriodicalExecuter(function() { updateTime(id); }, 1);

    Element.addClassName($('comment_' + id), 'editable');
    new Insertion.Bottom(div, '<div class="edit_control" id="edit_control_'+id+'">Edit Comment (<span id="time_'+id+'">'+time+' seconds</span>)</div>');

    editor = new Ajax.InPlaceEditor(div, url, { externalControl: 'edit_control_'+id, rows:6, okText: 'Save', cancelText: 'Cancel', 
    loadTextURL: textUrl, onComplete: function() { makeUneditable(id) } });
}

It does a couple things. It runs a PeriodicalExecuter to run the updateTime function which updates our countdown timer. It adds a CSS class to our comment div. It adds a control button to edit a comment. Lastly it uses the script.aculo.us Ajax.InPlaceEditor to do most of the magic. The hard part is done.

Periodic Execution Timer

So the updateTime function is reasonably simple. It finds the time written out in the DOM and decrements it by 1 second each second. Once it hits zero seconds it disables itself and the ability to edit the block. Let's take a look:

updateTime = function(id) {
  var div = $("time_"+id);
  if (div) {
    var time =  parseInt(div.innerHTML) - 1;
    div.innerHTML = time;
  }
  if (time < 1) {
    pe.stop();
    var editLink = $('edit_control_'+id);
    if (Element.visible(editLink)) {
      makeUneditable(id);
      editLink.parentNode.removeChild(editLink);
    }
  }
}

Call backs

We'll need a few call backs for the editor to work properly. Since many content pieces are converted from something else to HTML and not directly written in HTML we'll need a callback that will load our text. We'll also need a callback which will save our text (and then display it).

Load Text

The first call back we can see is referenced in the makeEditable() function. In our example it's:

url_for('restaurantnote/show?id=' . $comment->getId() . '&mode=raw');

Which is a symfony route to the restaurantnote module and the show action with an argument mode=raw. Let's take a look at this action:

public function executeShow ()
{
    $this->restaurant_note = RestaurantNotePeer::retrieveByPk($this->getRequestParameter('id'));
    $this->forward404Unless($this->restaurant_note instanceof RestaurantNote);
}

All this does is load the text (in our case the [markdown] formatting) into a template.

Save Text

The save text url in our example is:

url_for('restaurantnote/save?id=' . $comment->getId());

Using the Ajax.InPlaceEditor the value of the text-area is saved to the value POST variable. We consume it in our action like so:

public function executeSave() 
{
    $note = RestaurantNotePeer::retrieveByPk($this->getRequestParameter('id'));
    $this->forward404Unless($note instanceof RestaurantNote);
    if ($note->getUserId() == $this->getUser()->getId()) {
        $note->setNote($this->getRequestParameter('value'));
        $note->save();
    }
    $this->note = $note;
}

The note is also sent to a template that renders it, so when the save takes place, the edit form will be replaced with the new text.

Conclusion

As you can see with some script.aculo.us and symfony, it's fairly easy to mimic "Digg-style" in-place comment editing. You can test out a real example by visiting reviewsby.us.

Syncing with symfony and clearing your cache in one shot

2006-07-17T00:00:00-07:00

If you're using symfony's sync command to synchronize files across environments (e.g. moving your development files to a staging server), it helps usually to clear the cache of the receiving server.

The following line will help:

symfony sync production go ; ssh user@production "cd /var/www; symfony cc"

Assuming you have SSH keys defined and that you change user@production to your username and server host as well as /var/www switched to your website path. Also the symfony command needs to work on your "production" server (or whatever environment).

There may be a cleaner way to take care of this by changing the symfony command, but this works sufficiently well.

One of the more frustrating elements of web development is synchronizing multiple sites: usually a development site, a staging site and a production site. SVN helps with keeping your code versioned, but usually you don't want to check out a copy of your web site onto your live server.

Usually we use SFTP or rsync. The former has lots of problems, because a lot of manual work is usually involved to make sure you don't over-write important files.

rsync, however, is a champ and symfony takes advantage of this. The key files you'll have to deal with are $PROJECT_HOME/config/properties.ini and $PROJECT_HOME/config/rsync_exclude.txt.

Your properties.ini should look roughly like:

[symfony]
  name=reviewsby.us
[staging]
  host=staging.reviewsby.us
  port=22
  user=root
  dir=/var/www/staging/
[staging2]
  host=staging2.reviewsby.us
  port=22
  user=root
  dir=/var/www/staging

Each heading other than "[symfony]" is a different environment. In our example, we have two staging environments. The values under each heading should be self-explanatory. We can now run the following commands:

symfony sync staging
symfony sync staging go
symfony sync staging2
symfony sync staging2 go

The commands that lack go are "dry-runs" which just show you what files will be transfered. The other commands will run rsync and transfer all the files not specified in the exclude file, rsync_exclude.txt.

See askeet or the symfony documentation for more details.

Dynamic Linking to Syndication Feeds with symfony

2006-07-04T00:00:00-07:00

Adding a statically-linked syndication feed, a feed that is the same no matter where on the site you are, is a cinch with symfony, but what about dynamically linked syndication feeds? Let's say we're building the latest and greatest Web 2.0 app, there's going to be hundreds of RSS feeds, not just the most recent items. We'll want the latest comments to a post, the favorite things of a website member and it all has to be feed enabled. Sure, we can slap a link to the RSS feed and call it a day, but let's go a step further and stick it in the <head/> area as well. That way when someone clicks on the RSS icon in their browser, or adds a web page to Bloglines those extra feeds can be found.

Expanding your head

A typical layout.php for a symfony app will have a <head/> section like this:

<head>
    <?php echo include_http_metas() ?>
    <?php echo include_metas() ?>
    <?php echo include_title() ?>
    <?php echo auto_discovery_link_tag('rss', 'feed/latest')?>  
    <?php echo auto_discovery_link_tag('rss', '@feed_latest_georss', 
    array('title' => 'Latest Restaurants\' Locations (GeoRSS)' ))?>     
    <?php echo include_feeds() ?><!-- this is the custom feed includer -->
    <link rel="shortcut icon" href="/favicon.ico" />
</head>

Since this is in the reviewsby.us layout.php, the latest feed and the latest GeoRSS feed (which we developed in this article) will show up on every page. So for example, if you use FireFox, you can subscribe to either link when you click on the orange feed icon () in the URL bar no matter where you are in the web-application.

To expand this to allow for multiple feeds, we need to include <?php echo include_feeds() ?> (before or after the auto_discovery_link_tag calls makes the most sense).

Making the Feed Helper

Let's created a FeedHelper.php to put the include_feeds() function (don't forget to add use_helper('Feed') to your layout.php).

The function looks like this:

function include_feeds()
{
    $type = 'rss';
    $already_seen = array();
    foreach (sfContext::getInstance()->getRequest()->getAttributeHolder()->getAll('helper/asset/auto/feed') as $files)
    {
        if (!is_array($files))
        {
            $files = array($files);
        }
        foreach ($files as $file)
        {
            if (isset($already_seen[$file])) continue;
            $already_seen[$file] = 1;
            echo tag('link', array('rel' => 'alternate', 'type' => 'application/'.$type.'+xml', 'title' => ucfirst($type), 'href' => url_for($file, true)));
        }
    }
}

The function is doing what the deprecated include_javascripts and include_stylesheets functions did, just with syndication feeds. Also note, I stuck to just using RSS feeds. This function can no doubt be extended to Atom or other feed types, but for my purposes it was unnecessary¹.

Dynamically setting the feeds

In the reviewsby.us site, the menu items are tagged. There's tags for chicken, indian and bread for example. Each of them are to have an associated GeoRSS feed as described in a previous tutorial. I built our tagging system similar to Askeet. So in our tag module I created a function in the corresponding actions.class.php:

public function addFeed($feed)
{
    $this->getRequest()->setAttribute($feed, $feed, 'helper/asset/auto/feed');
}

This sets the attribute that include_feeds() pulls from. Here $feed is simply the route to our feed. So in our executeShow() I just make a call to $this->addFeed('@feed_tag_georss?tag=' . $tag). We're done.

We can now go to any of our tagged pages. Let's try chicken and see that we can subscribe to a GeoRSS feed of restaurants serving dishes tagged as chicken.

Slight problem. The title attribute of the generated link tags are always Rss. That can be mildly unusable.

Throwing feed titles into the mix

Let's change our addFeed() to allow for a second parameter, a title and have it store both the route and the title in the request attribute:

public function addFeed($feed, $title = null)
{
    $feedArray = array('url' => $feed, 'title' => $title);
    $this->getRequest()->setAttribute($feed, $feedArray, 'helper/asset/auto/feed');
}

We'll also need to adapt the include_feeds to appropriately accommodate associative arrays:

function include_feeds()
{
    $type = 'rss';
    $already_seen = array();
    foreach (sfContext::getInstance()->getRequest()->getAttributeHolder()->getAll('helper/asset/auto/feed') as $feeds)
    {
        if (!is_array($feeds) || is_associative($feeds))
        {
            $feeds = array($feeds);
        }

        foreach ($feeds as $feed)
        {
            if (is_array($feed)) {
                $file = $feed['url'];
                $title = empty($feed['title']) ? $type : $feed['title'];
            } else {
                $file = $feed;
                $title = $type;
            }

            if (isset($already_seen[$file])) continue;

            $already_seen[$file] = 1;
            echo tag('link', array('rel' => 'alternate', 'type' => 'application/'.$type.'+xml', 'title' => $title, 'href' => url_for($file, true)));
        }
    }
}

Note, there's a function is_associative(). It's a custom function that we can place in another helper:

function is_associative($array)
{
  if (!is_array($array) || empty($array)) return false;
  $keys = array_keys($array);
  return array_keys($keys) !== $keys;
}

It's a clever way of determining if a function is an associative array or not.

Conclusion

It looks like our GeoRSS feeds are on all our tag pages. Now we can take our favorite items labeled as Indian food and easily add the URL to a service like Bloglines and have it keep us up to date on new Indian dishes. This was simple, especially when much of the work was taken care of by the framework.

Most clients support RSS so unless there is a compelling need to use Atom or another format, then keeping it down to one choice is always your best bet. ↩

ReviewsBy.Us bugfixes

2006-05-30T00:00:00-07:00

A lot of bugs have popped up recently.

Logins

The logins weren't redirecting people to the correct place. Unfortunately the login system still needs a lot of work. I am probably going to rewrite it completely. It doesn't consistantly remember where you are coming from or where you intend to go after logging in. I'll be jotting down a clean system to log people in propperly.

Latitude and Longitude even more precise.

A small error in the database definitions resulted in lat/longitudes of restaurants that were greater than 100 (or rather abs(x) > 100 where x is latitude or longitude were being truncated to 100 or -100. This was easily fixed so things should look just right on the maps.

Quicksilver + TextMate = craZy delicious development environment

2006-05-28T00:00:00-07:00

UPDATE:Cmd-T allows you to search for the files in your currently opened TextMate project. I learned this shortly after writing this post, but forgot to mention it. Thanks again to Tyson Tune for pointing that out.

There's a number of tools for the OS X that help me with my productivity (and unfortunately have no equivalents on other platforms). Quicksilver, a launcher, and TextMate, a text editor work wonders and together work fairly well.

Quicksilver is a the GUI equivalent to the command line. You can launch applications or files or perform any number of operations on those files or applications. With its powerful collection of plugins you can have it do much more, for example you can take a music file and play it in iTunes within the iTunes party shuffle. Or take an image file and have it submit to flickr with a few simple keystrokes. Initially, I couldn't get an idea of the application, other than a lot of people loved it. Now, I'm barely using it to its potential and I love it. Using a computer without it is quite a drag.

TextMate is similarly feature rich and elegant. Just using a small number of its features makes it worth its cost. All my symfony projects are written using TextMate as are my articles for this web site. It's strength for me is its automation. Together Quicksilver and Textmate make a winning combination.

Projects in TextMate

I like the concept of "projects" in TextMate (its common to a lot of text editors). You can drag files or folders into TextMate and group them as you see fit.

Many of my projects are written using symfony, so I'll try to keep the entire project folder in my TextMate project. Additionally I'll keep the symfony libraries if not the entire PHP libraries referenced in the project as well. Now I have access to all of my files with relative ease. I generally create a file group in TextMate of frequently accessed files to bypass the pain going through hierarchies of folders.

If I have a TextMate project open, anytime I open a file that belongs in that project, it will open in that project window. That means if I use the mate command line utility, Finder or even Quicksilver, it'll still open in the project window. This is useful.¹

Catalogues in Quicksilver

When you open up Quicksilver and type in some letters, it searches some catalogues by default (e.g. Applications, Documents, Desktop, etc) in an attempt to figure out what the "subject" of your action to be. These catalogues are fully customizable, so it's trivial to add the directories of your project and your libraries into Quicksilver.

New Workflow

Now that you've setup your project in TextMate and added the same directories to Quicksilver, you'll have a much improved workflow. If you save your TextMate project in your Documents directory, you need only open Quicksilver (I use Command+Space as a shortcut) and type a few letters of the project (for example, I named my project file reviewsby.us for my restaurant review site).

When that's open, I can now open any file of that project in anyway I feel necessary. Let's say I need to open a library file, like sfFeed.class.php. I need only type in a few letters and it opens inside my project.

This process now saves me a ton of time in digging through hierarchies of folders upon folders. It's many times quicker than Spotlight. Give it a try, there's thousands of uses for it, this is just one way I use it.

While debugging errors in a project, if PHP tells you a certain file gave you a certain error, you can highlight the filename in Safari (or any other Cocoa browser), hit Command Escape (or whatever custome keystroke you setup for sending selection to Quicksilver), hit Enter and have it show up in TextMate. ↩

Google Analytics

2006-05-26T00:00:00-07:00

I've also jumped on the Google Analytics bandwagon and added a lot of the sites in "family." If you're familiar with Urchin, it has a similar feel to it. It almost feels a bit lighter feature-wise. There's a few issues I will take with it:

When they list links to your site (e.g. /login) they don't hyperlink it.
I'd like to drill down to raw data. I like to make my own correlations. Heck, I want to drill down to see if I should filter someone from the list.

Other than that, I like the potential that this will offer me. This combined with Google Sitemaps will provide powerful analysis of the site. As it stands, Katie and I should be eating at the Cheesecake Factory and non-chain restaurants as we seem to do well in the ranks and I need to write more about maps.

AJAX star rater for symfony

2006-05-23T00:00:00-07:00

Francois from the symfony project beat me to the punch. I was going to post a detailed how-to on adding a star-rater to your web site (similar to the one's I created for reviewsby.us), but for most of you this should do the trick. Unless people request it sooner, I'll hold off on publishing the details on my star-rater for a while. It only offers a few minor differences (IMO advantages) to this snippet.

Migrating from Drupal (4.7) to Wordpress

2006-05-19T00:00:00-07:00

I helped Katie setup her new blog this weekend and decided that WordPress offers much of what I want out of this blog for a lot less effort than drupal¹. I decided it might be worth my time to now while this blog is in it's infancy to try converting from drupal to WordPress.

The way I start most of my projects is with a plan:

Discovery: Find any articles about converting and get a gist of what to do.
Implementation
- Create a temporary subdomain to install WordPress
- Install WordPress
- Copy all content
- Copy all static pages
- Configure URLs to match the existing site ²
- adjust templates
- split entries correctly (i.e. change  to )
Make the switch when I'm satisfied

I'm really confident that this will be easy. I don't even have to worry about comments or anything, since this blog is pretty new, but I can demonstrate how to take care of the.

Discovery

This post details a migration path from drupal to wordpress. Some considerations had to be made since I'm using drupal 4.7.

Implementation

Copy content

I followed most of the instructions, with some alterations from vrypan.net.

I installed WordPress and in mysql ran the following commands:

use wordpress;
delete from wp_categories;
delete from wp_posts;     
delete from wp_post2cat;
delete from wp_comments

I run my drupal site in the same database server, so the data copying was a snap. If you aren't so fortunate, just copy the relevant drupal tables temporarily your wordpress database.

First we get the drupal categories into WordPress:

USE wordpress;

INSERT INTO 
    wp_categories (cat_ID, cat_name, category_nicename, category_description, category_parent)
SELECT term_data.tid, name, name, description, parent 
FROM drupal.term_data, drupal.term_hierarchy 
WHERE term_data.tid=term_hierarchy.tid;

Again with the posts:

INSERT INTO 
    wp_posts (id, post_date, post_content, post_title, 
    post_excerpt, post_name, post_modified)
SELECT DISTINCT
    n.nid, FROM_UNIXTIME(created), body, n.title, 
    teaser, 
    REPLACE(REPLACE(REPLACE(REPLACE(LOWER(n.title),' ', '_'),'.', '_'),',', '_'),'+', '_'),
    FROM_UNIXTIME(changed) 
FROM drupal.node n, drupal.node_revisions r
WHERE n.vid = r.vid
    AND type='story' OR type='page' ;

And the relation between posts and categories:

INSERT INTO wp_post2cat (post_id,category_id) SELECT nid,tid FROM drupal.term_node ;

And finally comments:

INSERT INTO 
    wp_comments 
    (comment_post_ID, comment_date, comment_content, comment_parent)
SELECT 
    nid, FROM_UNIXTIME(timestamp), 
    concat('',subject, '<br />', comment), thread 
FROM drupal.comments ;

I ended up moving the one static page I had into WordPress's "pages" section manually.

Since my pages are written in Markdown, I enabled the Markdown for WordPress plugin.

URLs

Now for the real test. I needed to go through each page on my site and see if I could get to it using the same URLs. Since I had only 14 posts, I did this somewhat manually. I used drupal's built in admin to do this from most popular to least popular. Most URLs worked fine. There were a small number that didn't for various reasons, I used custom mod_rewrite rules to handle them.

Adjusting Templates

My drupal template was fairly clean and simple. So I adjusted the CSS for the default theme in WordPress until I got what I liked. Very minimal changes had to be made to the actual "HTML."

Make the switch

Well, time to make the switch. In the WordPress administration, I just had to tell it that it's now going to be located at spindrop.us. Then I moved my WordPress installation to the spindrop.us web root. It was a snap. Let me know if you have any troubles.

Taxonomy, legible URLs and trackback support all seemed quite difficult to master in Drupal. In Wordpress they appeared to be available standard or with a minor change in the administration panels. ↩
This is a prime reason why URLs should be clean and make sense to the end user, not the programmer of the publishing software. ↩

Easy Yahoo! Maps and GeoRSS with symfony

2006-04-26T00:00:00-07:00

GeoRSS is an extension of RSS that incorporates geographic data (i.e. latitude/longitude coordinates). This is useful for plotting any data that might need to be placed on a map.

While building out the reviewsby.us map, I decided to use the Yahoo! Maps API versus the Google Maps API because I wanted to gain some familiarity with another API.

It was worth trying Yahoo!'s API. First of all, reviewsby.us has addresses for restaurants and Yahoo! provides a simple Geocoding REST service. This made it easy for me to convert street addresses to latitude and longitude pairs (even though this wasn't required as we'll soon see).¹ The real selling point of Yahoo! was the GeoRSS functionality. I can extend an RSS feed (which symfony generates quite easily) to add latitude or longitude points (or even the street address), direct my Yahoo! map to the feed and voila, all the locations in that feed are now on the map, and when I click on them, the RSS item is displayed. That cut down on a lot of development time.

Yahoo's GeoRSS

The GeoRSS format that Yahoo uses is fairly simple to grasp if you know what an RSS feed looks like. Here's a typical RSS feed:

<?xml version="1.0" encoding="UTF-8" ?>
<rss version="2.0">
    <channel>
        <title>Latest restaurants</title>
        <link>http://reviewsby.us/</link>
        <description>A list of the latest restaurants posted to my reviewsby.us</description>
        <language>en</language>
        <item>
            <title>Bubba Gump Shrimp Co. Restaurant and Market</title>
            <description>A *Forest Gump* themed restaurant.  Featuring a large selection of seafood items.</description>
            <link>http://reviewsby.us/restaurant/bubba-gump-shrimp-co-restaurant-and-market</link>
            <guid>25</guid>
            <pubDate>Sun, 23 Apr 2006 08:04:00 -0700</pubDate>
        </item>
        <item>
            <title>Famous Dave's Barbeque</title>
            <link>http://reviewsby.us/restaurant/famous-daves-barbeque</link>
            <guid>24</guid>
            <pubDate>Wed, 19 Apr 2006 19:58:08 -0700</pubDate>
        </item>
    </channel>
</rss>

The <item /> in this example is a restaurant. To turn this into a GeoRSS feed, we only need to change a few things:

<?xml version="1.0" encoding="UTF-8" ?>
<rss version="2.0" xmlns:geo="http://www.w3.org/2003/01/geo/wgs84_pos#">
    <channel>
        <title>Latest restaurants' locations</title>
        <link>http://reviewsby.us/</link>
        <description>A geocoded list of the latest restaurants' locations posted to my reviewsby.us</description>
        <language>en</language>
        <item>
            <title>Bubba Gump Shrimp Co. Restaurant and Market (Mall of America (Bloomington, MN))</title>
            <link>http://reviewsby.us/restaurant/bubba-gump-shrimp-co-restaurant-and-market/location/mall-of-america</link>
            <guid>18</guid>
            <pubDate>Sun, 23 Apr 2006 08:08:19 -0700</pubDate>
            <geo:lat>44.85380173</geo:lat>
            <geo:long>-93.24040222</geo:long>
        </item>
    </channel>
</rss>

We just added xmlns:geo="http://www.w3.org/2003/01/geo/wgs84_pos#" as an attribute to the <rss/> tag and added <geo:lat /> and <geo:long /> tags to any given item. If you'd rather use address, city, state and zip fields instead of latitude and longitude coordinates, the Yahoo! Georss page will tell you how.

symfony and Geodata

I knew full well that Yahoo! Maps did not require me to have everything in latitude longitude coordinates, but I felt that from an efficiency standpoint, it made more sense for me to convert them once using a geocoder and then Yahoo! wouldn't have to translate them later. Also, I'm trying to think ahead when more than just Minneapolis restaurants become a part of the reviewsby.us site, I now have an easy way of determining distance between a user's home and a given restaurant. Also, if Yahoo! Maps doesn't work out, I can use these coordinates in other mapping systems.

Extending the model

Models in symfony lend themselves to being easily extended. So we can easily take a model for a location:

<table name="location" phpName="Location">
    <column name="id" type="integer" primaryKey="true" autoIncrement="true"/>
    <column name="restaurant_id" type="integer" />
    <foreign-key foreignTable="restaurant">
        <reference local="restaurant_id" foreign="id"/>
    </foreign-key>
    <column name="stripped_title" type="varchar" size="255" />
    <column name="name" type="varchar" size="255" />
    <column name="address" type="varchar" size="255" />
    <column name="city" type="varchar" size="128" />
    <column name="state" type="varchar" size="16" />
    <column name="zip" type="varchar" size="9" />
    <column name="phone" type="varchar" size="16" />
    <column name="approved" type="boolean" />
    <column name="updated_at" type="TIMESTAMP" />
    <column name="created_at" type="TIMESTAMP" />
</table>

and simply add two columns inside the <table/>:

<column name="latitude" type="float" size="10" scale="8"/>
<column name="longitude" type="float" size="10" scale="8"/>

Easy! However, we don't want to have to set the latitude and longitude by hand each time we update a Location. So first we write a function that takes an address and converts it to latitude/longitude. I placed mine in a myTools.class.php in my lib folder:

class myTools
{
    public static function getLatLng($address, $city = null, $state = null, $zip = null)
    {
        $url = sfConfig::get('app_yahoo_geocode');
        $query['appid'] = sfConfig::get('app_yahoo_app_id');
        $query['street'] = $address;
        $query['city'] = $city;
        $query['state'] = $state;
        $query['zip'] = $zip;
        $query['output'] = 'php'; 
        $url .= '?' . http_build_query($query); 
        $response = @file_get_contents($url);
        if ($response) {
            $response = unserialize($response);
            return $response['ResultSet']['Result'];
        }
        return null;
    }
}

Defined in my app.yml is app_yahoo_geocode and my app_yahoo_app_id. myTools::getLatLng() queries the Yahoo! REST service and returns the coordinates that Yahoo! delivers. Note that the generated query string includes output=php. Yahoo! supports serializing output as PHP instead of XML. This can save a bundle of time over decoding XML.

So now let's look at our Location.php and override its inherited save function:

public function save($con = null)
{
    // save latitude and longitude
    $locdata = myTools::getLatLng($this->getAddress(), $this->getCity(), $this->getState(), $this->getZip());
    if ($locdata) 
    {
        $this->setLatitude($locdata['Latitude']);
        $this->setLongitude($locdata['Longitude']);
    }
    parent::save($con);
}

If you stop here, you'll at least now know how to add latitude and longitude coordinates to an object automagically.

Producing a GeoRSS feed

symfony very easily will allow you to generate an RSS feed. How do we create a GeoRSS feed? Just extend the sfFeed class. Rather than instantiating a feed like this:

$feed = sfFeed::newInstance('rss201rev2');

We do this:

$feed = sfFeed::newInstance('geoRSS');

And then create an sfGeoRssFeed.class.php and we're done. We've created a GeoRSS feed fairly easily. Comb through sfGeoRssFeed.class.php and compare it to the sfRss201rev2Feed.class.php, you'll notice it's not that different and that it's fairly easy to extend the sfFeed plugin for symfony.

Adding your feed to a Yahoo! Map.

Adding a GeoRSS feed to Yahoo! maps is simple. Before I embedded the RSS feed into my Yahoo! Map I was prepared to write an algorithm to cluster only on the points in my RSS feed, lucky for me (and you), Yahoo! Maps does this automatically. One pitfall you might reach during development is that Yahoo! Maps must be able to reach your GeoRSS feed. My development machine is my personal laptop, so this didn't work so well until I uploaded to a publically accessible staging server. The maps worked like a charm as you can see.

Conclusion

Yahoo! Maps are very powerful, and symfony is up to the task. I hope you found this tutorial useful. If you have any trouble, let me know. I hope your next meal is a good one.

Using Yahoo! Maps wasn't a requisite of using the Yahoo! Geocoder. The datasets that were returned could have been used to populate a Google Map just as easily. ↩

Dave Dash

tunneling and mysql

Why code rewrites are often coupled with redesigns

postgresql and python in osx

Optimization heuristics

The micro-caching heuristic

How to code... or do anything.

How to code... or do anything.

reusability

Templating

YUI Autocomplete the easy way

Data Source

YUI

Javascript

HTML

The End

python: relative paths

GeoDjango

django experiment: Day 1 and Model Inheritence

Model Inheritence

Where I'm still shaky

python: it begins

python: it begins

Python, Named arguments: Pure Genius

Python, Named arguments: Pure Genius

Better than ORM Object Persistence

jQuery shortcut functions and jQuery plugins

Symfony Camp: Ajax and Zend, what would you like to know?

Using sfDoctrine to match allowed email domains

Boosting terms in Zend Search Lucene

Boosting terms — some fields are better than others

Finding things using Zend Search Lucene in symfony

Equal height columns with jQuery

Debugging yaml configuration with the symfony web debugger

The Lucene Search Index and symfony

Index Something

Tips for symfony and Subversion

Ignoring files in cache/ and log/

Ignore other auto-generated files

Linking the symfony Library

Linking to symfony Plugins

sfZendPlugin

Caching REST with sfFunctionCache

PHP double versus single quotes

How do you find good programmers?

Parsing a list of Key:Value pairs

Make Adium show your Facebook status

Make Adium show your Facebook status

Not taking frameworks for granted

How to make Prototype Window Class as easy as Lightbox Gone Wild

Cropping Images using DHTML (Prototype) and symfony

Overview

Frontend: What would you like to crop?

HTML

CSS and corresponding images

Internet Explorer

The Javascript

Processing the crop

The Model

Conclusion

Using Zend Search Lucene in a symfony app

Obtaining Zend Search Lucene

Index Something

Find those users

What about new users?

...and beyond

Digg-style AJAX comment editing in PHP/symfony

In Place Forms

Periodic Execution Timer

Call backs

Load Text

Save Text

Conclusion

Syncing with symfony and clearing your cache in one shot

Dynamic Linking to Syndication Feeds with symfony

Expanding your head

Making the Feed Helper

Dynamically setting the feeds

Throwing feed titles into the mix

Conclusion

Ignoring files in `cache/` and `log/`