The Bright Lines » JavaScript

Misconceptions on URL encoding and replace() in JavaScript

Wouter Bos — Thu, 08 Apr 2010 21:26:34 +0000

Escape

Take for example the method of URL encoding a piece of string. There’s a pretty good chance that you run into a page that advises to use the escape() method for that. And although it seems to work, it will only handle ASCII-characters. You should use the newer method encodeURIComponent() for that.

The reason why escape() is still popular is probably because that method has been around longer, seems to do what you think it should do and is easier to remember. But if you use it for URL encoding, the end result will be buggy. Just Google for ‘escape JavaScript URL’ and you’ll find lots of misinformation on this subject. xkr.us has a a good web page on that subject.

Replace

Another one: replace(). That too seems to do what you think it should. Right? Well, not really. Only the first occurrence of a given string is replaced with another. Once again this is a method that pops up when you search for something like that and quickly copy-paste into your code. If you want to replace all strings you’ll have to use regular expressions. Instead of replace('foo', 'bar') you’ll have to write replace(/foo/g, 'bar'). Please pay attention to the lowercase ‘g’ after the regular expression. That’s what toggles the Global replace.

No related posts.

Document your Javascript: interview with Michael Mathews of JSDoc Toolkit

Wouter Bos — Sun, 28 Mar 2010 20:19:45 +0000

Michael Mathews

I recently discovered JSDoc Toolkit, a tool that generates documentation of your Javascript code in the form of a website. That was dearly needed because I wanted to give me and other team members a better overview of my growing JavaScript codebase.

The history of JSDoc

The JSDoc-toolkit and its predecessor JSDoc have been around for nearly 10 years now and in both cases the bulk of the code is programmed by Michael Mathews, often shortened to “micmath”. JSDoc started as a solution for the growing Javascript code at the company he worked for: “I couldn’t find any existing tool that did what I wanted, so I just naturally started building my own. I didn’t want to totally reinvent the wheel of course. I basically tried to port Sun’s JavaDoc idea over to JavaScript.” So he wrote a documentation tool in Perl. “Once it was done and we’d used it a bit, I decided to release it as open source on SourceForge. I figured if I found it useful then maybe someone else would as well.”

Although JSDoc is a very useful documentation tool and probably the first free tool as well, it wasn’t received that well back in 2001 according to Michael Mathews: “It seemed the general JavaScript community was not yet ready for JSDoc. Back then the majority of folks doing JavaScript were not in the habit of practicing what you would call ‘Software Engineering’ best practice! It was, more often than not, a field dominated scripting cowboys. The idea of automatic documentation generation, I think, was seen as something unneeded by most. According to the stats from SourceForge, the JSDoc tool was only downloaded a few hundred times in the first several years – and frankly that already seemed amazing enough to me.”

The renaissance of JavaScript

But as the web got more mature, so did the developers. Javascript gained respect in the last few years and the emerge of large libraries such as YUI, jQuery and Prototype showed that Javascript can be used in a complex but constructive way. New ways of coding were used to structure Javascript. Michael noticed the trend: “Since about 2005 JSDoc really started to take off. Since then it’s been downloaded over 30,000 times. It still seems amazing to me.” The latest version of JSDoc Toolkit is downloaded almost 5000 times since it was published in September 2009.

JSDoc in the wild

The term JSDoc by the way is a bit confusing. It refers both to the syntax and the program. The syntax has been relatively successful as well: “I know that a few of Google’s projects use JsDoc comments to extend their functionality. Closure Compiler is a good example of that, though it can be a pain when other projects start redefining the JsDoc syntax for their own purposes in ways that are incompatible with the original project. Of course there is no official standard for JsDoc comments, so I can’t blame them, but that is something I’d really like to see: an official JsDoc standard, I mean.”

“The idea of JSDoc has been picked up by many large projects over the years. However, my application was originally written in Perl and wasn’t very flexible, consequently it got rewritten by others a few times for internal use only. I don’t want to name any names because I personally think that, while perfectly legal to do that, it violates the open source spirit I intended the work to be used in.”

Document your code...

...run the tool...

...and publish the generated documentation website.

The Toolkit of JSDoc

A year after JSDoc was released, Michael resigned as an active contributor to the project. Gabriel Reid took care of JSDoc after that. He maintained and extended it. But around 2006 Michael decided to do a major upgrade for JSDoc. “Big frameworks were the norm and along with that was both the need for API documentation as well as the complexity of writing documentation for all the new and interesting things people were doing with JavaScript. I was beginning to think that, in order to get JSDoc up to speed with this new world, it would need to be rewritten from scratch. So I started a discussion about that on the users’ list. That eventually led to the current incarnation of the project: JsDoc Toolkit.”

The rewrite was written in Javascript itself. That was possible with the open source Rhino platform that parses the Javascript. That wasn’t based on any technical issue: “I just liked the idea of a program parsing itself. As a bonus the users were more likely to be able to modify the source code if it was written in a language they work in.”

Rhino runs on Java and was initially developed by Netscape. Rhino would have been a part of a larger project to port Netscape Navigator to the Java platform, but that never happened. Rhino is still being used and is maintained by the Mozilla Foundation.

How JSDoc works

JSDoc reads the code and the comments. The comments need to be formatted according to the JSDoc convention. It’s a notation that’s used in other tools like Google’s Closure as well and is similar to the JavaDoc convention.

The comments give hints to JSDoc Toolkit how the JavaScript is structured. Unfortunately not all notations are recognized by the tool. In fact: most of the really exotic notations are not recognized. Michael: “People don’t realize how difficult it can be to do static code analysis on a language as dynamic as JavaScript. Well, at least I didn’t. It is provably impossible in most cases to know what the various states of a JavaScript program will be at runtime, just from looking at the source code. I tell people, if you can get a program to know what you want have programmed, just from the plain source code, consider yourself lucky!”

“But for the rest of us JsDoc Toolkit provides a way to document your code in such a way that the tool never has to even see your code. In fact, you can even use JsDoc Toolkit in files with just comments, no code whatsoever! So in that sense there isn’t any code you can’t document with JsDoc Toolkit, as long as you don’t rely on the static code analysis features.”

The word “Toolkit” in the name “JSDoc Toolkit”, by the way refers to the template engine JSPlate. Michael: “I eventually went with the ‘Toolkit’ name mostly because I liked the sound of it, but also as a little nod to a favourite templating system of mine, Andy Wardley’s Perl Template Toolkit.”

JsDoc Toolkit Version 3

Right now Michael is in the middle of programming version 3 of JSDoc of which the first beta should be released somewhere this summer. “Basically there are a few feature requests from JsDoc Toolkit 2 that seem reasonable enough -things like documenting the same symbol more than once- which are utterly impossible due to the core architecture of the tool itself. All those feature requests are now on the table and I’ll try to include them in the new version.”

“One feature that will definitely be supported is the idea of overloaded functions. A typical example would be a function named ‘width()’ that returns the width when it is called with no arguments, but will set the width when called with a numeric argument.”

“This is fundamentally not possible to document in JsDoc Toolkit Version 2. And that really is the point of rewriting the tool for Version 3: it will include, in it’s core, all the lessons learned from the past several years of supporting JsDoc Toolkit. I believe it will provide a much more flexible foundation to build on.”

In addition, Michael will make more use of Rhino than he already did. Until now Rhino was only used to parse the server-side JavaScript. The parser that walks over the source code looking for doc comments and things like function declarations was written by Michael himself. Rhino has a JavaScript parser built in, but it would took “some serious Java hacking” to get it right.

“Fast forward a few years and some important things have changed: namely, Norris Boyd has made accessing Rhino’s internal Abstract Syntax Tree much easier. In fact, it is now as easy to walk over your source code as it is to use, say, DOM nodes to walk over HTML. I’d be crazy not to take advantage of all that free power just sitting there waiting to be used!”

“So Version 3 will have none of my own parser left in it. This means I get to shed about 40% of the project’s code, which was dedicated to just that one task, and can now focus on the more important bits: parsing the doc comments.”

“It also means that, in theory, if Rhino can understand it, you’ll be able to get JsDoc Toolkit to understand it. Of course this is all taking time to get up to speed with, but I’ve decided to use build my tool on top of the new RingoJS framework and that’s made my life much easier, and the process much faster.”

The first beta of version 3 is estimated to be released in early summer.

Continuation

One of the weak spots of the JSDoc project is that most of the time there’s only one active developer working on it. Although the tool is still alive and kicking, it naturally needs some nurturing to prevent it from getting stale. “To be honest it can feel like a real burden sometimes.” Michael admits. “It’s essentially just me these days, and I mostly can’t give the project all the time it deserves.” So developers are welcomed to join in the project: “I’m ready to sign them up! The fact is that I think of JSDoc as my baby. Not that I invented anything new or amazing, I most emphatically didn’t. But I’d lived with it long enough and I’d become associated with it, so I wanted to see it do well in life.”

And so the development continues even though there are now some other tools around like JSDoc. He even used some of them. “I used to write a lot of Perl and JavaScript at the same time and found NaturalDocs to be a great tool because it required a single format that could work in any language.”

Server side JavaScript

Although JavaScript as a program language is appreciated much more than a few years ago, it is still a quirky language. But that’s no problem for Michael: “Yeh, it’s a weird language. But I come from a background of Perl, so I like weird languages! But actually, JavaScript on the server side is a totally refreshing experience. Rhino, for example, allows you to use all the latest cutting-edge JavaScript syntax and features, and you never have to worry about running in an ancient or buggy web browser. I hope and expect that server side JavaScript will really start catching on in the next few years.”

A bit about Michael Mathews

It was hard to find a bit of information about New Yorker Michael Mathews. Although JSDoc has been around for almost 10 years and is downloaded a lot, he hasn’t been interviewed before. He has been asked many times for some assistance though. It can create funny situations: “I was doing some freelance work and the fellow sitting next to me asked if I’d ever heard anything about this ‘JSDoc’ stuff he was trying to use. I said I had heard of it and helped him sort it out. I had to eventually admit that I was the creator of the tool. He seemed a little surprised by that.”

Mathews started as a zookeeper, which he had done for a few years. But it wasn’t that much fun for him. “I basically cleaned up after lots of unusual animals.” When he wasn’t working, he played with his computer. “By the late nineties I was had my own homepage, complete with the large, requisite, ‘Under Construction’ message.” His work behind the computer payed off eventually: “in the fall of 1998 I was employed as a Web Developer for MTV Online, working in Times Square in New York City. It still seems amazing to me.

He now works since 2008 for the BBC in London where he works on the Glow JavaScript library. And yes, everything of it is documented with the help of JSDoc Toolkit.

Glow is a JavaScript library that just, like Prototype and jQuery, makes programming in JavaScript easier. The difference is that Glow supports whatever standard is currently defined by the BBC Standards and Guidelines for browsers, accessibility and usability. This could mean it has to support older browsers that other major JavaScript libraries have already dropped.

Hack: how to enable :first-child in IE6

Wouter Bos — Sun, 31 Jan 2010 00:43:14 +0000

UPDATE: The pure Javascript routine is updated. It is now as fast as jQuery.

One of the things I most dearly miss in IE6 is the lack of support for the CSS selector :first-child. Some shortcomings in browsers can be patched with Javascript. And :first-child is one of them for sure. If you use a Javascript library like jQuery or Prototype it’s just one line of Javascript:

If you don’t like Javascript libraries you can use this pure js-code:

var tags = document.getElementsByTagName("*"); var nTags = tags.length;
for (var i=0;i
Now every first child has the class firstChild that can be used in CSS selectors. You'll get something like this:
ul li:first-child,
ul li.firstChild {
    font-weight: bold;
}

The speed
Throwing around classes through the whole HTML tree is not without a price. Certainly not if you keep in mind that Javascript in IE6 is much slower than in Firefox or Safari. So I tested what would happen if the Javascript would run in the homepage of amazon.com, a page with almost a 1000 tags. Here are the results:

Adding firstChild with jQuery:

Amazon.com homepage (1000 tags): +- 215milliseconds
Five times the Amazon.com homepage (5000 tags): +- 742 milliseconds


Adding firstChild with plain Javascript:

Amazon.com homepage (1000 tags): +- 129 milliseconds
Five times the Amazon.com homepage (5000 tags): +- 798 milliseconds



As you can see there is not much performance difference. Especially when you keep in mind that every time I ran my test page I got different results. With a page of 5000 tags it could run as fast as 481 milliseconds or as slow as 931.
Code crafting
The first test I did with the pure Javascript code gave different results because of a inefficient routine. It ran about 1/3 slower in the 1000-tag page and 4 times slower in a 5000-tag page. After consulting a colleague I was able to find a more efficient routine that could match with jQuery.
Conclusion
I think a lag of about 200 milliseconds during a page load with a 1000 tags can be acceptable. Especially since a web page of 1000 tags is already a large website to my standards. Most websites I develop have about 300 tags a page at best. If you want to use this hack, I guess it's best to test the script in the largest pages of the website. The gain isn't that spectacular: just support for :first-child. But on the other side it could really make your life as a web developer just a bit easier.
You might be tempted to patch other shortcomings of IE6 with Javascript. Don't start coding, but consider the IE7.js-script. But I must say up front that I don't have good experiences with it. The script works good in small pages with small CSS files, but you'll experience serious performance issues with medium or large websites.
Putting it in perspective
Chrome 4, by the way needed just 37 milliseconds to run the Javascript in the 5000-tag page. That's 20 times faster then IE6.


Related posts:
Alternative for :nth-of-type() and :nth-child()
For the sloppy clickers
Too advanced CSS selectors
How to overcome the limits of CSS in Internet Explorer 6

jQuery 1.4 performance charts

Wouter Bos — Fri, 15 Jan 2010 11:04:47 +0000

Yesterday jQuery 1.4 was released. They celebrate it with a special website: jquery14.com. There are a lot new features, but what I definitely like is the fact that performance improves with every release. Here below are a few performance charts from that website. Since the performance is broken down by browser you can see clearly that IE is lagging behind when it comes to Javascript performance.

No related posts.

Alternative for :nth-of-type() and :nth-child()

Wouter Bos — Mon, 04 Jan 2010 21:17:14 +0000

View demo

One thing I really like about jQuery is the ease by which you can select the nth item of a list of items like this: jQuery("ul > li:eq(3)"). In CSS you would select an nth item like this: ul > li:nth-of-type(3). The problem is that :nth-of-type() is not supported at all in Internet Explorer. IE9 will support all CSS3 selectors, including :nth-of-type(), but by the time IE9 is a mature browser, Webkit and Gecko-based browsers are be able to do everything except your dishes.

So if you want to make an exception for, say, the third menu item, you have to either add a class to that menu item or select that item through Javascript. But if you don’t care for IE6, there’s a reasonable alternative:

ul > li:nth-of-type(3) equals ul > li:first-child + li + li.

Although both selectors are different, they do select the same element. But the first is not supported in IE and the second is. From IE7 and up! Sure, it’s not feasible to select the 100th item in a list, but not all lists are that long. And you can always use Javascript as a fallback.

This alternative selection method is by the way by no means equal to :nth-of-type(). With :nth-of-type() you can make a cool zebra striping without the use of Javascript.

And finally, as suggested in the title, there is also a way to handle :nth-child(3) in an almost similar fashion:

ul > li:nth-child(3) equals ul > *:first-child + * + *.

Too advanced CSS selectors

Wouter Bos — Wed, 23 Dec 2009 22:27:05 +0000

A few years ago the only reasonable way to select an element through CSS was by using classes and/or tag names. If you wanted to target a specific element, you had to select it by using its classname. Selecting elements with tag names wasn’t always practicle. For example: if you use the selector div.container div, you style all div elements inside the div.container.

Most of the time that’s not what you want. Using tag names as selectors can also make design changes troublesome if you have to add more divs and need to give it a different styling. But since IE7 we can use new selectors that help us to build much more delicate CSS selectors. Those new selectors are:

Child selector: .parent > .child
First child selector: .parent .child:first-child
Adjacent selector: .sibling + .sibling
First child selector: div:first-child
Attribute selectors: div[id]

Now you can style the following HTML with just a single class:

HTML:


	This is some title
	Lorem ipsum
	Lorem ipsum
	Lorem ipsum
	Some body text with lots of lorem ipsums in it
	Some body text with lots of lorem ipsums in it
	Lorem ipsum

CSS:

.container span {
	font-family: sans-serif;
}
.container > span:first-child {
	font-size: 1.5em;
	font-weight: bold;
	font-family: serif;
}
.container > span > span {
	font-style: italic;
}
.container > span > span > span {
	font-weight: bold;
}
.container > span + span {
	display: block;
	margin-top: 0.5em;
}
.container a:nth-child(2) {
	font-weight: bold;
}
.container a {
	display: block;
	color: #f0f;
}
.container a:nth-child(4) ~ a {
	font-size: 0.9em;
}

The advantage is that you can, as you can see, style whole blocks of HTML code with just one class. In theory you could even use no classes at all. Without ‘class’-attributes on every element, the HTML code looks just a little bit more cleaner and readable. You can also use those and other selectors in javascript libraries like jQuery to select particular HTML elements. Since most libraries are cross-browser you don’t have to worry if a selector is or isn’t supported by a browser. Consider this piece of code in jQuery:

jQuery(".container a:nth-child(4) ~ a").click( function() {
	alert('hello world');
})

That’s all nice, but should I use it?

No. I use them some times, but I try to stay away from it if possible. The problem with these selectors is that they rely so much on the structure of the HTML that it becomes harder to change the HTML structure. If you take another look at the HTML and CSS that I showed as an example in the beginning of the article, it’s clear that you can’t add a few links or spans without having to change the CSS. Add some advanced jQuery selectors in the mix and you may have locked the HTML structure forever since no-one dares to change it.

Should I discard those new selectors then?

Not necessarily. Sometimes you just have no choice if you for some reason are not able to change the HTML. My personal guideline is that styling has to be predictable. So when you edit some piece of HTML code, the styling should behave as expected. That means that you don’t style your HTML in a way that the text-color of the 4th span is red.

So what can be styled predictable? If you ask me advanced selectors shine when the structure is relatively fixed. Lists are a good example. Just take a look at these code blocks below. The styling will behave normal, regardless what you do with the HTML.

HTML:


Lorem
Ipsum
Dolor

Sit
Amet

CSS:

ul.list li + li {
	margin-top: 10px; /* adds top margin to all list items except the first */
}
ul.list > li {
	font-weight: bold; /* styles only the first level list items differently */
}

Javascript:

jQuery("ul.list li:nth-child(even)").addClass("zebra"); // adds zebra striping to a list

HTML elements you can easily apply advanced selectors

Besides ul, you can style uls, dls, tables. That doesn’t mean that other elements should only be styled with classes. There are always some exceptions to the rule. Just let your common sense be the judge.

Popups dodging popup blockers

Wouter Bos — Thu, 29 Oct 2009 21:29:41 +0000

A few days ago a collegue asked how he could create a link to another page with Javascript that would open in a new window. He reckoned that since the emergence of the popup blockers it’s impossible to create a popup of any sorts with Javascript. Not true, but understandable. Popup blockers changed the world and protected us ever since from both irritating bannering and designers that didn’t care about usability. And there are valid solutions like absolute positioned blocks.

Anyway: most popup blockers only block popups that open without any user interaction. If you use an onclick event to open a new window, it will be ignored by convention.

What’s creepy by the way is that you could open multiple popups in one click like this:

open popups

No related posts.

For the sloppy clickers

Wouter Bos — Wed, 28 Oct 2009 19:21:23 +0000

The best example I can think of is a series of blocks that each contain just a head, an intro and a link to an article, like the one you see here underneath.

A list of articles

The only linkable items here are the “Read more” texts at the bottom of the blocks. The source code looks like this:



Lorem ipsum dolor sit amet

		Suspendisse potenti. Pellentesque nibh ...
		Read more

Nothing wrong with the code, but why don’t we replace the li with an a-tag? That way the whole block is clickable. Much easier for the sloppy clicker. Especially when you use the :hover pseudo-class to add an effect like changing background to make the clickable area even more obvious to the user.


   
Lorem ipsum dolor sit amet

Suspendisse potenti. Pellentesque nibh ...
    Read more

ul.articles a {
	float: left;
	width: 100px;
	margin: 0 0 10px 10px;
	background: #ccc;
}
ul.articles a:hover {
	background: #eee;
}

If you like to have some control over what Google indexes you might not want that the whole text block will be indexed as important text, but just the heading. In that case you might take a different approach. You could put the link in the head of the article block and let Javascript take care of making the whole blocks linkable. Then you’ll have to use some code like this:



Lorem ipsum dolor sit amet

Suspendisse potenti. Pellentesque nibh ...
    Read more

jQuery("ul.articles a:odd").each( function() {
	var anchor = this
	jQuery(this).parent("li").click( function() {
		document.location.href = jQuery(anchor).attr("href")
	})
	jQuery(this).parent("li").mouseover( function() {
		jQuery(this).addClass("alternateBackgroundColor")
	})
	jQuery(this).parent("li").mouseout( function() {
		jQuery(this).removeClass("alternateBackgroundColor")
	})
	jQuery(this).parent("li").addClass("link")
})

This is just an example, but I hope that I could show you what alternatives you have when you add a link to a website.

The Bright Lines » JavaScript

Misconceptions on URL encoding and replace() in JavaScript

Escape

Replace

Document your Javascript: interview with Michael Mathews of JSDoc Toolkit

The history of JSDoc

The renaissance of JavaScript

JSDoc in the wild

The Toolkit of JSDoc

How JSDoc works

JsDoc Toolkit Version 3

Continuation

Server side JavaScript

A bit about Michael Mathews

Links

Hack: how to enable :first-child in IE6

The speed

Code crafting

Conclusion

Putting it in perspective

jQuery 1.4 performance charts

Alternative for :nth-of-type() and :nth-child()

Too advanced CSS selectors

HTML:

CSS:

That’s all nice, but should I use it?

Should I discard those new selectors then?

HTML:

CSS:

Javascript:

HTML elements you can easily apply advanced selectors

Popups dodging popup blockers

For the sloppy clickers

Lorem ipsum dolor sit amet

Lorem ipsum dolor sit amet

Lorem ipsum dolor sit amet