What a named scope does in Rails, in Django terms, is allow you to name a given query and use it as a base, or “scope,” for further queries. In other words, your narrowing down the results upon which additional queries that you run will be executed.

It’s pretty much the same idea behind modifying initial Manager QuerySets as outlined in the Django documentation, with one major difference: Scopes can be chained. Chaining is the feature I had long coveted, and I had been banging my head around for a solution for the last few months.

The other day while I was poking around the internals a bit though, a decent idea came to me. I realized it would be quite simple for me to extend the default Manager class with an attribute where I could store QuerySet methods I wanted to execute as part of a scope, and then overwrite get_query_set() with a hook that ran these methods before executing the rest of the query.

This turned out to be a nice strategy, and with the addition of some decorators the interface itself turned out to be extraordinarily simple.

Anyhow, here’s a walkthrough of how my alpha “scopable” module would work, with example code.

Let’s start with a simplistic blog post model:

class Post(models.Model):
    title           = models.CharField('title', max_length=200)
    author          = models.ForeignKey(User)
    body            = models.TextField('author')
    publish_at      = models.DateTimeField('published at', default=datetime.now)
    is_draft        = models.BooleanField('is draft', default=False)
    is_deleted      = models.BooleanField('deleted', default=False)
    objects         = PostManager()

That’s enough to provide for some good example scopes. Now, here’s what PostManager might look like, with explanation to follow:

from scopable import ScopableManager, scope

class PostManager(ScopableManager):
    @property
    @scope
    def active(self):
       return self.filter(is_deleted=False, user__is_active=True)

    @property
    @scope
    def published(self):
       return self.filter(is_draft=False)

    @property
    @scope
    def public(self):
       return self.active.published

So as you can see, the ScopableManager class is very easy to use; you just add the @scope decorator to any method that you want to turn into a scope. (Note that I added the @property decorator as a stylistic preference; it’s not required.)

To use the code, you just have to be sure to call a QuerySet method after it, e.g.:

Post.objects.active.all()
Post.objects.published.all()
Post.objects.active.published.filter(publish_at__lte=datetime.now())
Post.objects.public.all()

The only difference between a scoped and unscoped method is that execution of any QuerySet methods defined in a scoped method are deferred until a QuerySet method is called (e.g. Post.objects.public.all()), and scoped methods return an instance of your manager back, so they can be chained. As you can see from the public method, scoped methods can be nested as well.

Why is this awesome? Well the main advantage from my perspective is that it allows your business rules to be defined in one and only one discrete place.

So if, for example, I added a new business rule that blog posts shouldn’t be considered published unless they were not drafts and the publish date had already passed, I could just add that rule to the published method:

    @property
    @scope
    def published(self):
       return self.filter(publish_at__lte=datetime.now(), is_draft=False)

This means that any other queries that were built on published, such as public, are now updated with the new business logic we added to published.

The implementation of all of this is pretty straightforward. The main complication comes from implementing scope as a decorator and trying to elegantly capture all of the QuerySet methods used in a scoped Manager method.

This is still alpha code, and as of now it doesn’t handle custom Manager methods that are not scoped. I mostly just wanted to get this idea out there before I do any more work:

"""
scopable.py - A module for providing Rails-like named scopes to Django model Managers
Verson 0.1
"""

from functools import wraps
from django.db import models

class DummyManager(object):
    def __init__(self, actual_mgr, calls=[]):
        super(DummyManager, self).__init__()
        self.actual_mgr = actual_mgr
        self.calls = calls

    def __call__(self, *args, **kwargs):
        last_call = self.calls[-1]
        last_call["is_property"] = False
        last_call["args"] = args
        last_call["kwargs"] = kwargs
        clone = DummyManager(self.actual_mgr, self.calls)
        self.calls = []
        return clone

    def __getattribute__(self, name):
        if name in ["calls", "actual_mgr"]:
            return super(DummyManager, self).__getattribute__(name)

        # attr_type, args, and kwargs are set here by default in case the attr never gets called
        call = { "attr": name, "is_property": True, "args": [], "kwargs": {} }

        # if this is a default manager method
        if hasattr(models.Manager, name):
            call["definition"] = "default"

        # else if this is a custom manager method
        elif hasattr(self.actual_mgr.__class__, name):
            call["definition"] = "custom"

        # if it can't be found in those two classes, throw an error
        else:
            raise AttributeError("'%s' object has no attribute '%s'" % (self.actual_mgr.__class__.__name__, name))

        self.calls.append(call)
        clone = DummyManager(self.actual_mgr, self.calls)
        self.calls = []
        return clone

def scope(fn):
    @wraps(fn)
    def scopified(*args, **kwargs):
        args_list = list(args)
        actual_mgr = args_list.pop(0)
        dummy = DummyManager(actual_mgr, [])
        args_list.insert(0, dummy)
        deferred_calls = get_deferred_calls(actual_mgr, fn, args_list, kwargs)
        a = actual_mgr.__class__(deferred_calls=deferred_calls)
        a.contribute_to_class(actual_mgr.model, actual_mgr.model.__name__)
        return a
    return scopified

def get_deferred_calls(mgr, fn, args, kwargs):
    deferred_calls = []
    result = fn(*args, **kwargs)
    if result.calls:
        for call in result.calls:
            if call["definition"] == "default":
                deferred_calls.append(call)
            elif call["definition"] == "custom":
                next_result = getattr(mgr, call["attr"])
                if not call["is_property"]:
                    next_result = next_result(*call["args"], **call["kwargs"])
                deferred_calls = deferred_calls + next_result.deferred_calls
            else:
                raise AssertionError("Something went wrong.")
    return deferred_calls

class ScopableManager(models.Manager):

    def __init__(self, deferred_calls=[]):
        super(ScopableManager, self).__init__()
        self.scopes = []
        self.deferred_calls = deferred_calls

    def get_query_set(self):
        qs = super(ScopableManager, self).get_query_set()
        if self.deferred_calls:
            for deferred_call in self.deferred_calls:
                next_qs = getattr(qs, deferred_call["attr"])
                if not deferred_call["is_property"]:
                    next_qs = next_qs(*deferred_call["args"], **deferred_call["kwargs"])
                qs = next_qs
        return qs

Please let me know in the comments if you have feedback, suggestions, bugs, grave warnings, etc.

Well, I hit some snags as I got underway. Partly this had to do with me being something of a novice when it comes to testing, and partly it has to do with the state of testing in Django at present.

My mini-gripe with testing in Django is that it’s just kind of loosey-goosey. You can use doctest, or you can use unittest. You can put your tests in your functions, or in your models, or in tests.py, or in a tests/ subfolder.

For me at least, I want my framework to be a bit more opinionated. I want a bunch of guys who are all smarter than me to have sat around and thought about (or better yet, generalized from a working production code) things like the best way to configure testing for a Python web framework, and then to tell me, in no uncertain terms, the exact way to do it.

I’m obviously exaggerating a bit, and I don’t really mean at all to rant about a phenomally awesome, not to mention open source, framework like Django. What all of the above amounts to is that I had to ponder deeply on the problem space myself and come up with a way of structuring my test code that would work for me.

So that’s what I did, and here is the organizational structure I intend to use as I proceed.

An Organizational Structure for Test Code in Django

This is how Wikipedia defines unit testing:

In computer programming, unit testing is a software design and development method where the programmer verifies that individual units of source code are working properly. A unit is the smallest testable part of an application.

One of the things that kept frustrating me about writing tests was that my tests were all methods of a test case class that referred to a model class or a view module. Most of the examples you see are written in that form, such as the following from the Django testing documentation:

class AnimalTestCase(unittest.TestCase):
    def setUp(self):
        self.lion = Animal.objects.create(name="lion", sound="roar")
        self.cat = Animal.objects.create(name="cat", sound="meow")

    def testSpeaking(self):
        self.assertEquals(self.lion.speak(), 'The lion says "roar"')
        self.assertEquals(self.cat.speak(), 'The cat says "meow"')

What frustated me was that you’d end up with a big long catalog of test methods for all the various methods of Animal. It would also be hard to determine what the test was referring to by looking at the test method name. Sometimes a method might be testSpeaking(); other times it might be testShouldSpeak(). In either case, I didn’t feel like the methods were serving very well as documentation, which is one of the supposed benefits of user testing.

Then as I re-read the definition of a unit test to myself for the hundredth time, it dawned on me. Unit tests are supposed to focus on *the smallest testable part of an application.* But all the examples I was seeing had tests that were organized around the class, which is certainly not the smallest testable part. That distinction belongs to methods and class attributes.

I stumbled upon my first rule: Organize test methods around the individual methods and attributes of a class, instead of the class itself. How so? The easiest is to create a separate test case subclass for *each* method and attribute. Each test method of that subclass is then a test of a behavior of the method or attribute under test.

This mini-breakthrough also helped me flesh out a rule for writing better test method names: Start every test method name with “test_should_”. This borrows from a page in behavior-driven development, but basically “should” is the perfect word to use to express a desired behavior.

In Practice

Here’s what all this looks like in practice, taking the Animal class as an example again from Django’s testing documentation:

# models.py
from django.db import models

class Animal(models.Model):
    name = models.CharField(max_length=20)
    sound = models.CharField(max_length=20)

    def speak(self):
        return 'The %s says "%s"' % (self.name, self.sound)

And here’s how I would propose that test code for this class be structured (with a focus on the organization, not thoroughness of testing or appropriateness of implementation):

from django.test import TestCase
from myapp.models import Animal

class AnimalTestCase(TestCase):
    def setUp(self):
        self.lion = Animal.objects.create(name="lion", sound="roar")

class Class(AnimalTestCase):
    # obviously a silly, trivial test
    def test_should_instantiate_object_of_its_own_type(self):
        self.failUnlessEqual(type(Animal()), Animal)

class NameAttribute(AnimalTestCase):
    def test_should_be_defined(self):
        self.failUnless(hasattr(self.lion, 'name'))

    def test_should_accept_string_value(self):
        a = Animal()
        a.name = "bear"
        a.save()
        self.failUnless(a.id) # object was saved successfully

    def test_should_allow_no_more_than_20_chars(self):
        self.lion.name = "a"*21
        self.failUnlessRaises(Warning, self.lion.save)

class SoundAttribute(AnimalTestCase):
    def test_should_be_defined(self):
        self.failUnless(hasattr(self.lion, 'sound'))

    # similar test_shoulds as with 'name' here

class SpeakMethod(AnimalTestCase):
    def test_should_be_defined(self):
        self.failUnless(hasattr(self.new_pp, 'speak'))

    def test_should_return_appropriate_string(self):
        self.failUnlessEquals(self.lion.speak(), 'The lion says "roar"')

(I have not executed the above code, so please don’t be surprised if it has errors.)

So, it’s pretty simple really. Each attribute or method being tested is given an explicitly named test class, which is a subclass of a single test case class for the class under test. Each specified behavior of that attribute or method is assigned a test method beneath the appropriate test class.

Here’s what I like about this approach:

• You don’t have to “think” while writing tests. If you’ve decided to define a method, you just list off to yourself all of the things it should do and/or require.
• The tests actually read like documentation. Not that you’ll ever see them expressed anywhere in this format, but I like to read them as “NamedAttribute.should_accept_test_value” or “SpeakMethod.should_return_appropriate_string”. It’s just so easy to make sense out of exactly what each part of your model is supposed to do.
• The class names provide some grouping structure to the test methods. Ten groups of ten methods, for example, are a lot easier to make sense of and keep organized than 100 test methods without any other structure.

Getting It To Work

It’s actually quite easy to get this all to work. The only trick involved is that you have to specify which suites to run to the django test runner. (This may not in fact be a requirement, but I wanted adding new test case subclasses to be quick and easy; I didn’t want to spend my life configuring which tests to run in a bunch of places.)

Here’s a quick function I wrote to handle this:

def build_test_suite_from(test_cases):
    """
    Returns a single or group of unittest test suite(s) that's ready to be
    run. The function expects a list of classes that are subclasses of
    TestCase.

    The function will search the module where each class resides and
    build a test suite from that class and all subclasses of it.
    """
    test_suites = []
    for test_case in test_cases:
        mod = __import__(test_case.__module__)
        components = test_case.__module__.split('.')
        for comp in components[1:]:
            mod = getattr(mod, comp)
        tests = []
        for item in mod.__dict__.values():
            if type(item) is type and issubclass(item, test_case):
                tests.append(item)
        test_suites.append(unittest.TestSuite(map(unittest.TestLoader().loadTestsFromTestCase, tests)))
    return unittest.TestSuite(test_suites)

Django allows you to define a suite() function in your test module, which is then automatically detected by the test runner when tests are executed. So we’ll just embed the above function in that, like this, for example:

from myapp.test_utils import build_test_suite_from
from myapp.tests.models.some_model import SomeModelTestCase
from myapp.tests.models.another_model import AnotherModelTestCase

test_cases = [
    SomeModelTestCase,
    AnotherModelTestCase
]

def suite():
    return build_test_suite_from(test_cases)

Here’s what happens: build_test_suite_from() runs through each TestCase you pass to it, and searches the module where that TestCase is found for subclasses of that TestCase (including the TestCase itself). It then builds a test suite containing all of the test methods out of each of these, and then combines all of them together into one giant test suite which Django will run automatically when you do ./manage.py test.

The idea again is to keep configuration to a minimum. For each model you create test cases for you have to import the model’s TestCase and add it to the test_cases list, but no additional configuration is required for subclasses you create of that TestCase.

I expect that this structure will evolve as I actually start using it more. In particular, tests for forms and views may or may not work as cleanly within this structure as do those for models.

If you have any comments or feedback (good or bad), I’d love to hear about it below.

It started off pretty easy — just use “an” when the first letter of the phrase is a vowel and otherwise use “a.” And then I remembered about abbreviations (“an HR department”), silent H words (“an honorable man”), and exceptions among a few O and U words (“a one-time offer” or “a union representative”), and suddenly it wasn’t so easy.

Fortunately I managed to find an excellent database of English words that includes pronunciations, and in no time at all I was able to generate a list of exceptions.

Here’s the final result, implemented as a custom django template tag (in my use case, I needed to able to wrap the original phrase in an HTML tag, so I couldn’t use a filter tag). This could also easily be a standalone Python function as well (literally by removing the @register.simple_tag decorator).

import re

@register.simple_tag
def with_indefinite_article(phrase, before_phrase="", after_phrase=""):
    silent_h_words = ["homage", "hour", "herb", "heir", "honest", "honor"]
    letters_with_initial_vowel_sound = ["a", "e", "f", "h", "i", "l", "m", "n", "o", "r", "s", "x"]
    vowels = ["a", "e", "i", "o", "u"]
    o_exceptions = ["once","one","oneness","ones","oneself","onetime"]
    u_exceptions = ["ubiquitous","uganda","ugandan","ukraine","ukrainian","ukulele","ukuleles","unanimous","unanimously","unicellular","unicorn","unicorns","unicycle","unicycles","unification","unified","uniform","uniformed","uniformly","uniforms","unify","unifying","unilateral","unilateralism","unilaterally","union","unionist","unionists","unionization","unionized","unionizing","unions","unique","uniquely","uniqueness","unisex","unison","unisons","unit","unitarian","unitary","unite","united","uniting","units","unity","universal","universally","universe","universes","universities","university","universitys","unix","uranium","urinalysis","urinary","urinate","urinating","urine","urologist","urologists","urology","uruguay","uruguayan","uruguays","usable","usage","usages","use","used","useful","usefully","usefulness","useless","usenet","user","users","uses","usual","usually","usurp","usurpation","usurped","usurper","usurping","usury","utah","utahn","utahns","utahs","utensil","utensils","uterine","utero","uterus","utilitarian","utilities","utility","utilitys","utilization","utilize","utilized","utilizes","utilizing","utopia","utopian","utopians","utopias"]

    # get rid of everything but letters and numbers and a few characters
    first_word = re.sub(r"[^A-Za-z0-9\-]", "", phrase.split()[0])
    first_letter = first_word[0].lower()

    # default to 'a'
    article = "a"

    if first_word.isupper() or (len(first_word) == 1):
        # if all caps or a single letter, we're going to assume it's an abbreviation
        if first_letter in letters_with_initial_vowel_sound:
            article = "an"
    elif first_letter == "h" and any( [re.match(r'(?i)%s' % w, first_word) for w in silent_h_words] ):
        article = "an"
    elif first_letter in vowels:
        if first_letter == 'e':
            article = "a" if first_word.lower().split('-')[0] == 'ewe' else "an"
        elif first_letter == 'o':
            article = "a" if first_word.lower().split('-')[0] in o_exceptions else "an"
        elif first_letter == 'u':
            article = "a" if first_word.lower().split('-')[0] in u_exceptions else "an"
        else:
            article = "an"

    return "%s %s%s%s" % (article, before_phrase, phrase, after_phrase)

Hopefully this will save someone who stumbles upon this some extra work down the road. If you find any bugs, please let me know in the comments.

Surprisingly though, jQuery (as of 1.3, to my knowledge anyway) does not make any functions available to allow you to securely parse a simple text string into JSON. Not sure why this is so, especially since they must do it behind the scenes for the getJSON() method, but so be it.

I needed this functionality, though, so I could pass some data from one of my templates to a jQuery function I’m working on.

Fortunately, even though jQuery doesn’t have built-in functionality for this, it was relatively trivial to wrap the open source JSON parsing and stringifying library published by JSON.org in an easy-to-use jQuery plugin.

So that’s what I did. I changed nothing of the JSON parsing library implementation published by JSON.org; I just wrapped two jQuery methods around it, as follows:

;(function($) {
    if (!this.JSON) {
        var JSON = {};
    }
    /* ... implementation of parse() and stringify() here ... */
    $.toJSON = function(text, reviver) {
        if (typeof reviver == "undefined") {
            reviver = null;
        }
        return JSON.parse(text, reviver);
    };
    $.jSONToString = function(value, replacer, space) {
        if (typeof replacer == "undefined") {
            replacer = null;
        }
        if (typeof space == "undefined") {
            space = null;
        }
        return JSON.stringify(value, replacer, space);
    };

})(jQuery);

Pretty convenient. Now I can securely parse and decode JSON to my heart’s content.

Here’s the source code of the full jQuery JSON plugin, if you’re interested. Note that I have not extensively tested the plugin, but since I did not actually touch any part of the JSON parsing library it’s based on (save for creating a local scope for the JSON variable), I wouldn’t foresee any issues.

Here’s a bit more information about the library the plugin is based on.

I was just forwarding it to someone who didn’t know any HTML, and I figured it makes sense to post it here for the benefit of anyone else who stumbles upon it.

Why You Should Care About HTML

For writers and editors, HTML probably seems at best like a minor nuisance. The fact that it was written by (and to some extent conceived of for) programmers is probably a big reason why.

In fact, HTML is pretty easy to learn conceptually. And these days, HTML has mostly been stripped of what once made it a “programming” or design language. Properly used, HTML has now become a way for content authors to define the meaning of various aspects of a web document. In fact, content authors and editors are really the only people who can properly mark up a document and ensure it’s being done correctly.

Here are just a few of the reasons why you should care (passionately) about HTML:

• Both people and computers need to read your documents. In general, we write and edit for people. But on the web, people aren’t the only ones who read your document — machines read it as well. Proper HTML is like good grammar for machines.
• Good HTML makes it easier for designers and programmers to do their jobs. Designers are charged with making your documents look right and programmers are charged with making them work right. When you play by the HTML rules, you give them the freedom to focus on what they do best.
• Your documents will last forever. The web is constantly evolving. Programmers come up with new tricks for working with web documents every day, and designers constantly have new design tools added to their arsenal. Good HTML is forward compatible, meaning your documents themselves can evolve as well, no matter which way technology goes in the future.
• You can add a whole new level of meaning to your documents for readers. Used properly, HTML allows you to give readers more information about what various parts of your documents mean. For example, you can seamlessly define abbreviations, cite quotations, or identify people and places without intefering with the readability of your documents.

What HTML Is and Isn’t

In simple terms, HTML is a markup langauge that allows content authors and editors to explicitly define the semantic meaning of various aspects of a web document. That’s a simple and true definition, but it conflicts with what most people think HTML is and how most people use it. Unless you grasp this definition of HTML, you’re doomed to use it improperly.

Here’s an easy way to think about it:

HTML is not a way of making a document look or function in a certain way. Instead, HTML is a way of describing what a document is.

Here’s a quick little example that illustrates where most people go astray in their thinking.

Our CEO was featured in Time magazine today.

When preparing to mark this sentence up in HTML, most writers or editors would think: “Okay, I just need to put the title of the publication ‘Time’ in italics.” This is a natural reaction, since you’ve been trained to put publications in italics throughout your life, and most of us are accustomed to pressing the little “I” button in Microsoft Word when we want something to appear in italics.

While that line of thinking isn’t entirely wrong, it isn’t entirely right either — at least from a sematnics perspective. The problem is that it conflates two related but separate concepts: presentation (in this case, the italics) and meaning (in this case, emphasis, which is what the italics are actually conveying).

We can see the difference in practice. One way to mark up the document (in an older version of HTML) was to use the italics tag, <i>:

Our CEO was featured in <i>Time</i> magazine today.

The newer version of HTML, however, insists upon defining this text not by how it should appear, but by what it means, in this case using the <cite> (citation) tag:

Our CEO was featured in <cite>Time</cite> magazine today.

Citations, by default, are rendered as italics in most browsers, so for readers, the two ways of coding this document appear the same. If that makes the distinction seem merely academic, it’s not. There are some real ramifications to the two different approaches. To name a few:

• A content author might decide to provide a link to the home page of the publication being cited.
• A designer might choose to render citations with a different look — using the color green or a small graphic rather than just italics.
• A programmer might build a function that provides a complete list of references cited in an article at the bottom of the article, which can be generated on the fly.

There are many examples similar to this one — some with big implications, and others that are largely academic.

Nevertheless, good HTML is like good grammar. Although at the end of the day, you can very like ignore the rules of grammar and still get your point across, consistent use of good grammar makes your documents more readable and your image more professional. Good HTML has plenty of its own benefits — both obvious and hidden.

HTML Basics

Now that you understand the proper way to conceive of HTML, and are as convinced as I am of the importance of separating meaning from presentation and functionality, it’s time to get down to brass tacks and describe how to actually use HTML.

HTML is a markup language; i.e. it’s used to “mark up” a document. Think of it like a highlighter pen. When you mark up a word or phrase with HTML, it’s like drawing a yellow line through it. The only difference is, you add a bit of description when using HTML to explain what the yellow line actually means.

Fortunately, HTML is quite simple. Like with any foreign language, there are some vocabulary words you’ll eventually need to learn, but conceptually HTML boils down to three concepts:

1. Tags
2. Elements
3. Attributes and values

Let’s briefly put these terms in context, and then tackle them one by one. Following is a phrase marked up with generic HTML:

<element attribute="value">The cat is black.</element>

In the context of the example above:

• A tag is everything that comes between, and inclusive of, the angle brackets (< and >). HTML phrases always start with an opening tag (e.g. <element>) and end with a closing tag (e.g. </element>). To use our metaphor from above, they define where you start putting your highlighter marker on the page and where you stop. (That’s why you must close all tags. If you don’t, a browser won’t be able to know where you wanted to stop your highlighter.)
• An element is a word that describes what the phrase your marking up is. Common elements include a paragraph (<p>), a list item (<li>), a hyperlink (<a>), an abberviation (<abbr>), or an image (<img>). There are approximately 40 or so elements and only about 15 or 20 that are commonly used.
• An attribute/value pair provides more information about the phrase being marked up and is not always required. Attribue/value pairs are used, for example, to define the destination of a hyperlink (e.g. <a href="http://www.yahoo.com">), the source of an image file (e.g. <img src="http://www.photo.com/my_picture.jpg">, or the meaning of an abbreviation (e.g. <abbr title="United Nations">U.N.</abbr>).

That’s really all there is to it. HTML is not more fancy than that. There are just a few syntax rules you need to learn and abide by in order to ensure your HTML is correct and a few element definitions you need to know to start using HTML.

Let’s start with syntax.

HTML Syntax

The example from above…

<element attribute="value">The cat is black.</element>

…is properly formatted HTML. The rules are pretty straightforward and easy to follow.

1. HTML is case sensitive. Always use lowercase for elements and attribute names.
2. A tag always starts with < and ends with >. If you forget to put a > at the end of your tag, you’re going to confuse the computer, it won’t know where your tag ends!
3. An HTML phrase always opens with a start tag (<element>) and closes with an end tag (</element>). The forward slash / is what makes a tag an end tag. Exception: There are a few tags called “empty tags” that don’t a phrase between them. An example is the img tag, which generally looks like this: <img src="[image URL]">. Empty tags still must be closed. They are closed by adding a space and a forward slash at the end, as follows in our example: <img src="[image URL]" />.
4. The value of an attribute must always be placed in quotes. For example <a href="http://www.yahoo.com"> and not <a href=http://www.yahoo.com>. Don’t forget to close your quotation marks! The incorrect <a href="http://www.yahoo.com> will cause you nightmares.
5. Multiple attribute/value pairs should be separated by a space. For example, <a href="http://www.yahoo.com" title="Yahoo! home page" class="external_link">.
6. HTML phrases must be properly nested. This is best demonstrated by example. Correct: <p>That statement is <em>ridiculous.</em></p> Incorrect: <p>That statement is <em>ridiculous.</p></em>

Those are the ground rules — the grammar of HTML, if you will — and that’s all you really need to know. Everything else is just a matter of definitions and specifics. A good reference guide, like the guide at htmlhelp.com will give you easy definitions for every element and attribute. (Be forwarned, however, that most guides are written for an earlier version of HTML and might imply that you can ignore the above syntax rules, which you cannot).

There are plenty of tools out there that will help you code HTML more quickly than you can by hand. But if you take the time to learn the actual mechanics of HTML, and if you internalize the concepts behind it, you’ll find yourself with much more control over the meaning of your documents and in a position to provide much more value to your readers.

This was easy enough to accomplish via jQuery, but I quickly hit a snag. I had assigned an event listener to the “change” event, but it turns out that the change event only gets fired when a user actively focuses on a field, inputs text, and then blurs. If a script — such as Google AutoFill — updates a value in a field on its own, the change event is unaware of it.

I poked around on Google a bit, and as far as I can tell, there doesn’t really seem to be an easy answer for this. So to make it work, I decided to write a quick jQuery plugin to handle the job.

I realized fortunately that rather than reinvent the wheel, all I need to do was find a way to listen for changes to input fields and then fire the change event manually if they were altered outside of the usual focus/blur routine (i.e. by a script).

Here’s the plugin code I came up with:

(function($) {
    $.fn.listenForChange = function(options) {
        settings = $.extend({
            interval: 200 // in microseconds
        }, options);

        var jquery_object = this;
        var current_focus = null;

        jquery_object.filter(":input").add(":input", jquery_object).focus( function() {
            current_focus = this;
        }).blur( function() {
            current_focus = null;
        });

        setInterval(function() {
            // allow
            jquery_object.filter(":input").add(":input", jquery_object).each(function() {
                // set data cache on element to input value if not yet set
                if ($(this).data('change_listener') == undefined) {
                    $(this).data('change_listener', $(this).val());
                    return;
                }
                // return if the value matches the cache
                if ($(this).data('change_listener') == $(this).val()) {
                    return;
                }
                // ignore if element is in focus (since change event will fire on blur)
                if (this == current_focus) {
                    return;
                }
                // if we make it here, manually fire the change event and set the new value
                $(this).trigger('change');
                $(this).data('change_listener', $(this).val());
            });
        }, settings.interval);
        return this;
    };
})(jQuery);

The plugin makes use of the trigger() and data() functions. In a nutshell, we loop over the input element or set of children input elements, storing their initial value in the data cache provided by the data() function. We then check to see if the stored value matches the value of the input during the current iteration. If so, we do nothing, if not, we manually fire the change event via trigger().

There’s also a bit of logic in there to ignore the element that has focus. We don’t need to worry about this element, since if the value is changed while the user has focus, the change event will be fired as normal when the element is blurred.

That’s all there is to it. To use the plugin, you simply call listenForChange() on the form or input you want to listen on, e.g.:

$("form").listenForChange();

You can then assign whatever functions you want triggered to the change event as normal, and they’ll be fired regardless of whether the user types them or they are updated via a script.

I’ve tested this on Firefox with Google AutoFill and Safari’s built-in AutoFill, and it seems to work on both. I haven’t tested it on fields injected after page load, but presumably it would work for these as well.

I’m not sure Furry Brains can take much credit for having a role in their winning this award, since it’s really the outstanding writing day in and day out that makes The Glass Hammer such a great publication. If anything, the design and implementation of the website step out of the way for the content itself to shine through.

But I’m pleased and proud if my contribution at least in some small way helped them get there. Way to go, TGH!

The API is really simple to use. It took me only a few hours to put together a quick little utility called Address on the Spot, which allows you to double click anywhere on a map to find an address for a location.

So here’s a quick how to for using the Reverse Geocoding API.

First, create a GLatLng object with a set of geocoordinates:

point = new GLatLng(45.523875, -122.670399);

Next, create a GClientGeocoder object and pass the point object to it, along with a callback function. The callback function display_address() will be executed when the geocoder returns a response.

geocoder = new GClientGeocoder();
geocoder.getLocations(point, display_address);

Here’s what display_address() might look like:

function display_address(response) {
    var address;
    if (!response || response.Status.code != 200) {
        address = "No location data found."
    } else {
        address = response.Placemark[0].address;
    }
    alert(address);
}

Couldn’t be easier. It’s definitely a handy addition to the Google Maps API, and this utility was pretty fun and easy to build.

In today’s example, the goal was pretty straightforward. We have a model class, Topic, that has a one-to-many relationship to a ThreadedComment model class. Each ThreadedComment for each Topic, has a User associated with it; furthermore the Topic itself as a User associated with it (the creator of the topic).

The issue here is that it’s not actually a straightforward relationship between Topic and ThreadedComment, due to some complicating factors beyond the scope of this discussion. Suffice it to say what we’re looking for is a list of users who are participants in the discussion, inclusive of the topic creator. But we’re not looking for just any old list — what we need is a QuerySet, for the reason that we are making use of some specialized tags in our templates that require data to be in a QuerySet.

Here’s a quick example of what we want the interface to look like in the template:

	<ul>
	{% for participant in topic.participants %}
		<li>{{ participant.name }}</li>
	{% endfor %}
	</ul>

So let’s do this step by step.

First, let’s get a list of the ThreadedComment objects we need:

ThreadedComment.public.all_for_object(topic))

The all_for_object() method is one of the “complications” we were referring to above. All we care about is that it returns a set of ThreadedComment objects for a given topic.

Okay, but now that we have this set of objects, how are we going to return a QuerySet that yields us User objects?

My co-worker actually showed this to me, so I’ll share it here. We’ll use in keyword argument in filter(), which maps to the SQL IN clause. The Django documentation has a simple example:

Entry.objects.filter(id__in=[1, 3, 4])

is equivalent to:

SELECT ... WHERE id IN (1, 3, 4)

So that’s just what we’re going to do: pass a list of user ids as an in keyword argument to identify the users we want.

First, let’s extend our original QuerySet to get the values we need as a list:

ThreadedComment.public.all_for_object(self).values_list('user_id', flat=True)

One thing is missing — we need to include the user id of the topic creator, as I mentioned above. We’ll just evaluate the QuerySet to a list and then add the topic creator to it:

list(ThreadedComment.public.all_for_object(self).values_list('user_id', flat=True)) + [self.creator.id]

Now let’s take the list of ids and pass it to a QuerySet that we’ll return User objects. Our list goes in the pk__in keyword argument:

User.objects.filter(
    pk__in=list(
        list(ThreadedComment.public.all_for_object(self).values_list('user_id', flat=True)) + [self.creator.id]
    )
)

Now let’s add this as a method participants() to the Topic model and throw a @property decorator on it so we can call it as an attribute:

@property
def participants(self):
    return User.objects.filter(
        pk__in=list(
            list(ThreadedComment.public.all_for_object(self).values_list('user_id', flat=True)) + [self.creator.id]
        )
    )

And that’s all there is to it.

What I love about this approach is that it’s a pattern that fits in to so many places (so many that I almost feel like it deserves a higher level abstraction within the QuerySet API). There are variations that make it slightly more efficient as well; for example, if I didn’t need to append the topic creator id in my example, then I could have passed the list of ids as a query as opposed to evaluating it:

User.objects.filter(
    pk__in=ThreadedComment.public.all_for_object(self).values_list('user_id', flat=True)).query
)

The advantage here is that this is evaluated as a subselect statement, so it only requires a single database call.

Anyhow, this technique is really fantastic when you have a set of related objects that for whatever reason you aren’t able to easily obtain as a QuerySet. Enjoy.

In a nutshell, I’m going to be working with some data sets that are grouped into grid squares on a map. The grid squares are based on the integer values of latitudes and longitudes, which I have arbitrarily chosen to identify by the coordinates at their southwest corner.

So let’s put this in a picture to make it a bit easier to visualize. Basically, just try to visualize a flat map of the Earth. Longitude runs E/W along the X axis, with 0 degrees being the central horizontal point and 180/-180 degrees being the edges (which are actually the same thing, since the Earth wraps around in a sphere), while latitude runs N/S (along the Y axis), with 90 being the north pole and -90 being the south.

Note the following map isn’t really to “scale,” in the sense that the coordinate system I am using doesn’t perfectly match up in the real world with the map behind it. I’m just using the map to help visualize the grid:

I’ve divvied up the map into “squares”, each of which is 1 degree of latitude tall and 1 degree longitude wide. The points are identified by their southwest corner coordinates, so the bottom left grid square on the map would be (-180,-90). I also plotted another random grid square at (-120,40).

So this is just a long-winded introduction to the problem at hand. I am going to be creating larger regions composed of these grid squares, which will be expanding outwards from a single grid square. There’s a chance that some of these regions will expand outwards at the edges of the map, so that the longitude and latitude points will have to be able to flip around and come back to the other side of the map.

In order to accomplish this, I basically need to be able to perform some simple math operations on the longitude and latitude integers that identify the grid squares. Take the grid square at (-180,-90). If I were to move three squares to the east, I would be at (-177,-90). If I were to move three squares to the west, though, I would be at (177,-90), because the longitude flips around to the other end of the scale at that point. Latitude is slightly different. If I go “up” three squares I’m at (-180, -87), whereas if I go “down” three squares, I’m at (0,-87)…in other words, the same latitude either direction.

Now we finally get to some programming.

To make sure that latitude and longitude behaved correctly in the calculations I would perform on them, I decided to encapsulate each of them in their own class. For my purposes (since the grid squares are spaced at integer values of latitude and longitude), I decided to subclass int, since the only behaviors I really needed to override were addition and subtraction.

Here’s a look at the final classes for both Longitude and Latitude:

class Lng(int):

    def __add__(self, number):
        return self.__bound(int.__add__(self, number))

    def __sub__(self, number):
        return self.__bound(int.__sub__(self, number))

    def __bound(self, num):
        return int(num - 360*math.floor(num/360.0 + .5))

class Lat(int):

    def __add__(self, number):
        return self.__bound(int.__add__(self, number))

    def __sub__(self, number):
        return self.__bound(int.__sub__(self, number))

    def __bound(self, num):
        return int((num - 180*math.floor((num/180.0) + .5))*2*(math.floor(math.sin((math.pi*num)/180 + math.pi/1.9999)) + .5))

What I really wanted to discuss here was my process for coming up with the “bound” functions in each class. “Bound” is what I named the function which takes the result of an addition or subtraction function and ensures that it stays within the boundaries defined by the map, according to the rules of that class.

Now, the last time I took a math class was in college (calculus), so what do I know about defining an oscillating function that repeats an upward movement in a linear fashion, or in the case of latitude, reverses that direction? In short, not much.

So how did I come up with a solution? Simple. I whipped out the excellent application called Grapher, which comes pre-installed on Mac OS X. This tool allows me to devise complex mathematical functions and see them plotted immediately. And that’s what I did.

Here’s how that process worked. Let’s start with longitude.

For longitude, I knew I needed a function that would start at (-180,-180) and proceed linearly until it reached (179,179). At x=180 though, the function would have to flip back down and start back at -180, since 180 and -180 are actually the same longitudinal line.

I basically started “brainstorming” at the Grapher function input, and finally stumbled upon:

Function 1. y=x-floor(x)

A good start, but the the lines are running from 0 to 1 on the y axis; we want the middle of the y axis to be 0. This should do the trick:

Function 2. y=x-floor(x+.5)

This function cycles through from -.5 to .5. We’re almost there, now we just need it to cycle through -180 to 180. A bit more fumbling around and we stumble upon:

Function 3. y=x-360*floor(x/360 + .5)

There you go, and that’s exactly what’s coded up in the __bound() method of Lng cited above.

Latitude was a bit trickier. In fact, I’m not thrilled with the implementation, but it works, so I’ll leave it alone until someone advises me of a better solution.

So our work with longitude sets things up nicely for latitude. We’ve already got a function that we can use to get us part of the way there:

Function 4. y=x-180*floor(x/180 + .5)

The only problem is, for latitude we need every other one of these upward sloping to be downward sloping (i.e. negative), since when latitude hits 90 degrees (the north pole) it then retreats back down rather than starting from the bottom again at -90 degrees.

What we need then is a function that oscillates back and forth between 1 and -1. If we can get that function to do its flip every 180 units, and we align it right, then we can apply it as a multiplier to Function 4 above.

I’m not very bright; the only function I know that oscillates is sin(x).

Function 5. y=sin(x)

Looks great. It’s flipping positive and negative at regular intervals. Now if we can only coax it to doing it at the intervals we want. Let’s see, it appears to be flipping signs in units of 3.14… hey, this is ringing a bell:

Function 6. y=sin(pi*x)

Sure enough, when we multiply x by a factor of π we get a graph that oscillates between positive and negative every one unit. For the next step, let’s get it to truly flip from 1 to -1 and get rid of the curving line. To do that let’s apply a floor() function, which will cause the line to flip between 0 and -1. Then all we need to do is add 0.5 (so that it flips between -.5 and .5) and then multiply the whole thing by 2. Et voila:

Function 7. y=2*(floor(sin(pi*x))+.5)

Getting closer. Now, let’s see if we can’t get this to stretch across 180 units for each sign switch. Some messing around a bit gets us here:

Function 8. y=2*(floor(sin((pi*x)/180))+.5)

Almost there! Now the only problem is that we have to shift everything to the left, since right now each range starts at 0 and ends at 180 on the X axis, whereas latitude starts at -90 and ends at 90 on the X axis. To be honest this required a feat of minor inspiration, which yielded the following:

Function 9. y=2*(floor(sin((pi*x)/180) + (pi/2))+.5)

By adding pi/2 to the sin function, we shift everything over by half the range, which yields exactly the result we are looking for!* Now, all we need to do is apply our multiplier to Function 4:

Function 10. y=(x-180*floor(x/180 + .5))*2*(floor(sin((pi*x)/180) + (pi/2))+.5)

And as you can see, this is pretty much exactly what’s coded as the __bound() function for Latitude above.

So how does all of this at long last play out in our code? Well, since we’ve overridden the addition and subtraction functions for each of these and applied our __bound() function to the results of these calculations, it means we can add and subtract numbers from these objects and they will stay within the defined boundaries of our grid.

>>> import grid
>>> x = grid.Lat(10)
>>> x
10
>>> x + 60
70
>>> x + 80
90
>>> x + 85
85
>>> x + 250
-80
>>> x + 270
-80
>>> y = grid.Lat(10)
>>> y
10
>>> y + 60
70
>>> y + 80
90
>>> y + 85
85
>>> y + 250
-80
>>> y + 270
-80
>>> y - 270
80
>>> x = grid.Lng(10)
>>> x
10
>>> x + 120
130
>>> x + 169
179
>>> x + 170
-180
>>> x + 350
0
>>> x - 350
20

What’s the moral of the story here? I guess it’s that you don’t always have to know what you’re doing to figure out how to do something. My math and trigonometry skills are rusty at best, and yet with a little futzing around with Grapher I was able to whip together a pretty tidy little function that did just what I wanted. I hope this post inspires you to make use of Grapher if you ever find yourself in need of some serious math back-up.

On a final note, if anyone reading this has a better implementation of the mathematical functions I whipped up here, I’d love to hear about it.

*Update: Re-reading my statement above that sin() was the only oscillating function I knew, I chuckled to myself that I also know cos()…at which point it dawned on me that I could replace out sin(x + pi/2) with cos(x), which I have since done.