micmath blog

To my shock today I noticed that on some of my live web sites many of the HTML forms were no longer working. By "no longer working" I mean they simply did nothing when you pressed the submit button. By "simply did nothing" I mean you clicked the submit button and the browser just ignored you. There was no perceivable reaction, no error message, no form submission and no explanation why not. I am practiced at debugging but this mystery didn't provide me much to get started with.

One clue was that these same forms worked fine on another browser. The second clue was that I'd recently upgraded to Safari 5. And then I remembered I was doing something slightly interesting with these forms: I was using JavaScript on the client (and PHP on the server) to validate the form input. As part of that system I was using some HTML5 attributes before they were supported in any browsers, the new pattern attribute, for one:

<input type="text" id="email" name="email" required="required"
pattern="^[^@]+@[^@]+\..+$" hint="Email must be a valid email address." />

My scripts would check each input element for a pattern attribute and perform a validation check against the corresponding submitted value. If the value did not match the pattern then the hint would be displayed as an explanation for what correction was needed from the user. I purposely chose the name pattern for my little system because I knew it would eventually get supported by new browsers that could do HTML5 validation -- pattern is a part of the HTML5 spec.

So what was happening here? Well it turns out that Safari 5 was exactly what I was waiting for: a browser that supported HTML5 form validation natively. The problem is that Safari does its validation before my code runs, and when Safari sees a validation fail it just does nothing. Well actually it does give the failing input element focus, meaning a hazy blue halo appears around it, but this is very subtle and may as well be considered imperceptible by anyone who isn't sharp-eyed and trained to look for it.

So Safari's native validation is useless because it provides no feedback. In fact it's worse than useless because it prevents my scripts from providing any feedback either, since they never get a chance to run. I can't find any documentation on how to control Safari 5's new form validation features so for now I must change my forms so they will no longer use any HTML5 validation attributes.

Safari 5: making my web pages less HTML5 compliant.

My momma always told me to keep the visibility of my variables as small as possible. Well, okay I can confidently say those words never passed her lips, but I can easily imagine some stern-faced Code Matron wagging a finger and chiding, "Global variables make baby Jesus cry." I won't go into all the whys of limiting the visibility of your variables, the point of this post is to compare a few different hows.

We have all been told that if we want to limit the visibility of a variable in JavaScript there is only one way to do it, and that is to use var inside a function. For example, if you wanted to ensure that a variable x did not touch the larger scope of your program, you could do something like this:

(function () {

  var x;    // reference to variable x exists in here
  alert(x); // undefined

})();
alert(x); // throws ReferenceError: x does not exist here

It is worth mentioning at this point that JavaScript has a very relavant little feature known as "variable hoisting," meaning that variables are effectively always declared at the top of the function, regardless of where you write the var x bit. The example below is effectively identical to example above:

(function () {

  // reference to variable x exists in here
  alert(x); // undefined
  var x;

})();
alert(x); // throws ReferenceError: x does not exist here

So you can't limit the visibility of a variable lexically, by writing the var stuff near the end of a function. If you declare a variable anywhere in a function the effect is the same as declaring it at the top of that function. For that reason it is arguably good practice to actually declare variables at the top of functions--you may as well make it look the same as the way it works, right?

But using var isn't the only way to accomplish our goal. You can also use function parameters.

(function (x) { // reference to parameter x exists

  alert(x); // x is undefined

})();
alert(x); // throws ReferenceError

This time the x listed as a parameter of the function actually pops into existence right before the rest of the function body. So it's possible a var somewhere (anywhere) inside the function could overwrite the parameter x everywhere.

And that would appear to be the end of our options, but in fact there is yet another way to achieve this same goal, and it doesn't use functions at all. This time we'll use with:

// assume undefined === void(0);

with( {x: undefined} ) { // reference to property x exists

  alert(x); // x is undefined

}
alert(x); // throws ReferenceError

People don't use with very much (for reasons I'll discuss a bit later) but it is designed to do exactly what we are after here: alter the scope chain for just a little while.

But let's compare these two techniques in a more real world situation: here I want to fill an array with the first few integers squared; I need to create a temporary variable named i but I don't want that i to exist or have any effect whatsoever outside my limited 2 or 3 lines of code.

First the functional way:

squares = [];

(function() {
  var i = 5;
  do { squares[i] = i * i } while (i--);
})();

alert( squares.join(', ') ); // 0, 1, 4, 9, 16, 25
alert(i); // ReferenceError

or alternatively:

squares = [];

(function(i) {
  do { squares[i] = i * i } while (i--);
})(5);

alert( squares.join(', ') ); // 0, 1, 4, 9, 16, 25
alert(i); // ReferenceError

And the same using with:

squares = [];

with( {i: 5} ) {
  do { squares[i] = i * i } while (i--);
}

alert( squares.join(', ') ); // 0, 1, 4, 9, 16, 25
alert(i); // ReferenceError

You could argue that the functional techniques look a little more sugary than the with technique and thus with is the cleaner way to go, but there are several other points against using with. First is a slightly circular observation that people are likely to be unfamiliar with with and so don't often use it. I suppose we could start a campaign to get more people to use it and therefore solve that problem, but here's why I wouldn't: with, like its ginger-haired stepbrother eval, is an easy to abuse little sucker, and you don't have to use it very much before you start to consider it confusing at least. What's more, it's slower than the functional equivalents shown here. Well, mostly. In some browsers it's actually slightly faster, but this is more than offset by the fact that it is incredibly humongously slower in almost all other browsers.

Of course, in the big picture, we're still only talking small fractions of seconds difference here so if you really just love the syntax of with shown above, you could still get away with using it. I leave that decision to you.

One of JavaScript's peculiarities is the 'new' expression: when a function invocation is included in that expression that function behaves like a constructor. But the function, by itself, is not intrinsically a "constructor," it's still just a function, and as such any function intended to be called with new can just as easily be called without new. Of course if you, the function's author, are relying on the side-effects provided by new things may not work as intended when those effects are missing. There are however many built-in examples of functions which work equally well, if perhaps slightly differently, when invoked as a constructor or as an ordinary function:

Objects are created by using constructors in new expressions; for example, new String("A String") creates a new String object. Invoking a constructor without using new has consequences that depend on the constructor. For example, String("A String") produces a primitive string, not an object. -- ECMAScript Language Specification, 4.2.1 Objects

Constructors are not the only way to build objects, there is a similar pattern that involves a function, called a "factory." The difference is that a factory is not called with new and factories don't return their this value, as constructors do. You could say that String is an example of a function that can be invoked as a constructor to return a String object, or as a factory to return a string primitive. This design blurs the distinction between factories and constructors. What can we do in this blurry area?

One thing we can do is to create a wrapper for an existing constructor, where we add on our own extensions:

function ArrayPlus() {
    var self = Array.prototype.slice.call(arguments);
    self.last = function() {
        if (this.length) return this[this.length-1];
    };

    return self;
}

var a1 = new ArrayPlus(1, 2, 3); // like a constructor
var a2 = ArrayPlus(1, 2, 3);     // like a factory

// the result is the same, arrays that can last()
alert( a1.push(4), a1.last() ); // alerts 4
alert( a2.push(4), a2.last() ); // alerts 4

When we call ArrayPlus as a factory it returns an instance of an Array with an extra method added on. However the same thing happens when we call ArrayPlus as a constructor, how is that possible?

Typically when a function is invoked as a constructor it automatically returns the constructor's own this value, however in ArrayPlus we are returning our own object, named self. This is allowed under the rules of the ECMAScript Language Specification (section 13.2.2), but it is important to know what the difference is: self is an instance of an Array, not an ArrayPlus. In essense we are overriding the typical behavior of a constructor and forcing ArrayPlus to always behave like a factory, whether it is called with or without new.

I referred to the ECMAScript Language Specification a moment ago, and it does indeed allow arbitrary objects to be returned from constructors, but there is a caveat worth knowing related to this: a constructor (called with new) cannot return a non-object. For example:

function MyNumber(value) {
    var self = new Number(value); // an object
    return self;
}

var one = new MyNumber(1);

function Three(value) {
    var self = 3; // a primitive
    return self;
}

var three = new Three();

alert(one);   // alerts 1
alert(three); // alerts "[object Object]"

What happened? The rules of ECMAScript say that if a constructor returns an object, as MyNumber does, then proceed as expected. But if a constructor returns a non-object, as Three does, then ignore the return statement entirely and instead return whatever the constructor's this value is. In the case of Three the this value is an instance of Three; in any case it is not the number 3, as you might think it is.

Quick, just from examining this code (no fair testing it), what will the following alert?

var f = 1;
function f() { }

alert(f);

The fact that I consider this code intersesting (or at least instructive) may tip you to what the answer will be: it's 1. Surprised? If you are you're likely suffering from the common misconception that your JavaScript code is executed in the order you wrote it. If that were true the variable named f would first be declared, then assigned the value 1. Next the function named f would be declared, replacing that existing f and finally the alert would show us the function. But we know that's not what happens.

What really happens is explained in section 10.1.3 of the ECMAScript specification, entitled "Variable Instantiation". If you imagine there is a Variable Object and the JavaScript engine is adding properties to it based on your source code, the specification defines the following distinct steps and the order in which they must happen:

• Function declarations
• Variable declarations
• The rest

Note that when you write var f = 1; you are really doing two things together: declaring a variable named f and then assigning the value of 1 to it. So, putting all that together, effectively the code in our previous example is the same as this:

function f() { } // function declarations first
var f;           // variable declarations next
f = 1;           // the rest
alert(f);

Web-based advertising has always been the unloved stepson of web development. Well I say unloved but actually there are lots of people who feel a sort of passion about things like "branding", "click-through rates" and "return on investment". The fact that these things personally bore me doesn't change the fact that there has always been an understanding in this business that if we want to have free-to-view web pages, we must then have a load of ad banners all over those pages. I'm going to concede that point, at least for the sake of my bigger argument.

And I'm conceding the point even though I have no personal experience to base it on: I myself have been using the World Wide Web from it's first days and can safely say I have never, not once ever, clicked on a single web ad. Not even one. Am I living this lifestyle to make some socio-political point? Nope, I'm just still waiting to see a web ad that is interesting enough to click on.

The ad mongers obviously know about uncooperative potential-customers like me, but when they analyze the problem of "he's still waiting to see a web ad that is interesting enough to click on" they come to the wrong conclusion that the solution is to make their uninteresting ads easier to see. If the blinking banners aren't appealing, they reason, then waving a popup window about in front of the content will surely make him want to click; it's worth a try anyway, right? I mean what's the harm?

So here's the harm: Since 2004 every major web browser has had an option to "block popup windows" built-in. And if that's not enough, users can choose from a wide variety of addons and plugins to do the same. If you, wayward web developer, have ever put an uninteresting ad in an popup window, you have helped to practically remove the popup window feature from every modern web browser. Well done, I hope you're proud of yourself.

But in the arms race for greater "CPMs" and "ROIs" there can be no surrender. If the user has developed an immunity to popup windows then there are other fronts to attack.

Now, based on the title of this blog entry, you might guess that I'm suggesting that Lightbox style ads are the latest front, and you'd be right. But I don't intend to criticize the technique used by JavaScript Lightbox implementations, they are no more evil than popup windows were ten years ago. Lightboxes don't annoy people, but annoying Lightboxes do. And here we are standing at the start of yet another very predictable path: how long before we see advice on how to disable JavaScript Lightboxes?

The problem is that a JavaScript Lightbox is not as distinct and self-contained as a popup window. Probably the most practical way to disable all JavaScript Lightboxes will be to disable JavaScript. This is a race back to the 1990's in terms of web development and it should be a concern to all of us.

But it's hard to know who to address this point to: I don't have the illusion that aggressive web advertisers, working for financially-desperate companies are going to suddenly develop a long-view ethos about this issue based on my warnings. In-your-face, hard-sell, intrusive web advertising will always be around but maybe it's up to all of us to enforce a new social contract, if these offenses are offered up by people who speak in terms of dollars and "clicks", then we can give our answer: don't click on those damned things. In fact add those products to your own blacklist of bad members of our web community. Avoid the hosting web sites altogether if possible. This is a campaign for us, if you're a web developer you may be helping yourself in the long run.

I've tried to clarify for myself something about how values are passed to functions in JavaScript, and it goes something like this:

• Variabless are always passed by value.
• Properties of objects act like references.
• Passing an object to a function passes the value of the object.
• The value of a reference acts like a reference to the same place in memory.

Okay, so how does this help us simulate pass-by-reference in JavaScript? Simple, if you want to pass a reference to a variable, you just need to make the variable a property of an object. Pass that object to teh function and you indirectly pass a reference to the variable.

This is especially useful when you want to write a function that works as a mutator -- a function that directly changes the variable that is passed to it.

Typically the pattern often used in JavaScript is that you pass the value of the variable, and the function returns a brand new value. This is really the only option if you are limited to pass-by-value. But, if you want to modify the variable passed in directly you must have pass-by-reference, and that's where this post is all leading up to.

var name = {ref: "Bob"}; // create a reference

function mutate(n) {
    if (n.ref) n.ref += "-O!";

    // no need to return anything, we changed the name directly
}

mutate(name);
alert(name.ref); // dereference

I was surprised today by an unexpected behavior of JavaScript. In most computing languages the idea of references is often confusing to new programmers, but for some reason this is not emphasized as much in JavaScript. However it is not a topic that can be safely ignored.

var arr = [1, 2, 3];

function foo(arr) {
    arr.shift();   // modifies the array "arr" refers to
    arr.push("a");
    arr[3] = "b";
}
foo(arr);

console.log(arr); // [2, 3, "a", "b"]

I was tripped up momentarily today, swept away by the flexibility of JavaScript, where you can add and modify properties of anything whenever you feel the whim to do so. Well, it turns out that there are a few exceptions to that rule: you can't add properties to primitive data types. JavaScript has the following primitives:

• Number
• String
• Boolean
• Undefined
• Null

And the example that shows a primitive resisting my attempt to stick a property onto it:

var s1 = "quick brown fox";
s1.lang = "en";
alert(s1.lang); // undefined

But there is a solution if you want to do this sort of thing: you must wrap your primitive data up in an object-instance cloak.

var s2 = new String("quick brown fox");
s2.lang = "en";
alert(s2.lang); // en

A few more examples of primitive wrappers (you cannot construct a new Undefined or Null):

• var value = new Number(3.14);
• var value = new String("hello");
• var value = new Boolean(false);

Interestingly these wrappers affect the result returned by typeof, causing them to all become "objects" whose constructor is equal to either Number, String or Boolean. You do some coersion with string to promote it to an object and then demote it back to a primative, like so:

// start with a primitive
var s1 = "hello";
alert(typeof s1); // string
alert(s1.constructor == String); // false

// promote to an object
var s2 = new String(s1);
alert(typeof s2); // object
alert(s2.constructor == String); // true

// demote back to a primitive
var s3 = ""+s2;
alert(typeof s3); // string
alert(s3.constructor == String); // false

I've been playing around with the idea of writing lexers in JavaScript, it seems like a language that should be well suited to the task. Unfortunately rhino's regex engine isn't the most powerful on Earth.

While researching this I found an interesting link on Jon Aquino's Blog that discusses two features I wish rhino had: recursive references to captured matches and named references to captured matches, both apparently are available in PHP and other languages too.

I tried to simulate something similar in JavaScript with questionable success: this example shows a method for working outwards from the innermost parenthesized content, out to the outermost, evaluating as it goes.

var str = "(10+(6+(1+1)*(3+2)))";

while (str.indexOf("(") > -1) print(str), str = str.replace(
    /\(([^()]+)\)/g,
    function() { return eval(arguments[1]); }
);

print(str);

// (10+(6+(1+1)*(3+2)))
// (10+(6+2*5))
// (10+16)
// 26

Edit: Is Steven Levithan's XRegExp library a solution?

It can be difficult dealing with JavaScript's duck-typing when you just want to know if a given object is or isn't an array. Especially as the typeof operator will return "object" for an array -- true but not very specific.

Douglas Crockford suggests the following as a good (but not perfect) technique to determine if you have an array:

function isArray(value) {
    return value &&
        typeof value === 'object' &&
        typeof value.length === 'number' &&
        typeof value.splice === 'function' &&
        !(value.propertyIsEnumerable('length'));
};

If you want to keep a series of data together in a collection JavaScript provides two built-in choices: array or object. Douglas Crockford writes in his book JavaScript: The Good Parts:

The rule is simple: when the property names are small sequential integers, you should use an array. Otherwise, use an object.

It's not that simple. In practice this decision is going to be weighted one way or the other based on your usage of the data.

For example, if you have a collection of employee objects where the employee id numbers happen to be small sequential integers, you might decide to go with an array. But what if you find that most of the time you need to extract employees based on their name property?

function getElementsByProperty(property, value, array) {
    var found = [];
    for (var i = 0; i < array.length; i++) {
        if (array[i][property] == value) {
            found.push(array[i]);
        }
    }
    return found;
}

var susan = getElementsByProperty("name", "Susan Smith", employees)[0];

Not pretty and not fast, especially on big arrays. But if that property you are searching for is unique for any element it becomes a primary key and you can do something much more elegant by creating employees as an object:

var susan = employees["Susan Smith"];

Okay that looks a lot nicer, so maybe we should try to use objects with primary keys? There are two problems you could have. The first has to do with order: in JavaScript the order of keys in an object is not guaranteed to be the same as the order in which you added them, so if the order of your elements is important you can't use an object.

And there's a second problem: what if you frequently need to modify and read the total number of employees? Unlike arrays, there is no magical length property to objects, instead you'll have to loop over every key in the object, incrementing a counter as you go. We're back to "not pretty and not fast" again.

So your choices look more like this:

• If the order of the elements must be predictable: use an array.
• If you will need a fast, simple way to get the number of elements: use an array.
• But if you will need a fast, simple way to access elements by a string primary key: use an object.

And if you need some combination, you must resort to one of the not pretty, not fast approaches.