Decrementing vs Incrementing

A while back, I had read some things about how decrementing loops in javascript were faster then incrementing, especially in IE. (see http://www.moddular.org/log/javascript-loops) The main point being that it is faster to compare to 0 then any other number. As a result, I had implemented some decrementing loops in my own code (loop unrolling seemed too heavy handed) but haven't been too compelled to make a habit of it. Today I decided to investigate this again, since I had some lingering doubts about if any of this made a difference at all... So, here it is.

My test code consists of the following:

var limit = 1000000;
function testi(){
var n = 0;
for (var i = 0; i < n =" i;" n =" 0;" i =" limit;"> 0; i--)
n = i;
}

time1 = new Date().valueOf();
testi();
time2 = new Date().valueOf();
document.write("time incrementing (" + limit + " times) = " + (time2 - time1) + "
");

time1 = new Date().valueOf();
testd();
time2 = new Date().valueOf();
document.write("time decrementing (" + limit + " times) = " + (time2 - time1) + "
");

Fairly straight forward. I can change the limit value and check to see what the differences are for different size loops. I compared in Firefox 2 and IE 7. And I found the following.

1. At a million loops in FF and IE, decrementing is usually twice as fast as incrementing (~220ms vs ~440ms). But how important is a 200ms baseline difference when you are doing a loop a million times? Presumably, whatever you are doing inside of that loop will most likely eclipse the difference.*
2. If we decrease the loop to 100,000, the difference in IE becomes ~20ms decrementing vs ~40ms incrementing. And Firefox, surprisingly, shows the same results.
   
3. At 10,000 loops, Firefox wavers between 0 and 10 ms for either incrementing or decrementing. As does IE.
   
4. At 1,000 loops, we are in 0ms land for both IE and FF.

So, we see that at a million, decrementing saves us some time -- especially in IE. When we drop down to 100,000, the differences between the browsers disappear, and while decrementing seems to be twice as fast, the ~20 ms difference is tiny in the grand scheme of things. At 10,000, the differences incrementing and decrementing in both browsers disappears entirely.

One problem with decrementing loops I have found is that they don't behave the same as incrementing loops. If you are looping through an array, you end up going through it backwards. This is sometimes a problem, if for example you are executing a string of functions stored in an array, and you want to ensure that the first one in is executed first. A simple way to fix this, is to find the index by subtracting your decrementing index from your limit. But does putting this simple bit of math into the loop corrodes any performance gains we made by decrementing in the first place? To find out, I substituted the n = i; assignment in the decrementing loop function with a n = limit - i; assignment. I found that:

1. In Firefox (at 1 million loops), the cost of decrementing rose to ~550ms vs ~440ms for incrementing. Erasing any gains and adding cost. In IE, the time for decrementing increased beyond the time for incrementing.
2. Below 1 million loops, decrementing lost all of its advantage (at best being equal) when the compensating math was implemented.

In conclusion, it seems that decrementing loops are more efficient, but only when the loops are extremely large. This advantage pales next to whatever else you might be doing in an extremely large loop. Above all, decrementing is counter-intuitive and requires compensating math when used with loops that are order dependent (or might be order dependent), and this compensation nullifies any gains that decrementing gives in the first place. As with many optimization tricks, it seems that the cure is worse then the symptom.

*Strangely, in Internet Explorer only, if I reverse the order of the loops and execute the decrementing function first (at a million loops), decrementing takes ~210ms and incrementing takes anywhere from 900 to over 2000 ms. This is a big difference, and I haven't been able to figure out the source of the disparity.

Always Eat Alone (sometimes)

A friend of mine recently read the book Never Eat Alone, and since he's been singing the praises of networking. Yes, networking is important.

I have always seen the value in networking, but I get a little cranky when networking gets all of the attention at the expense of well... being alone. Being alone is really important.

I recall a networking buff extolling the virtues of practicing small talk at every possible opportunity, like when your in a cab. When I'm alone in a cab with a non-talkative driver, I see it as an opportunity to think and reflect. Maybe I am just a misanthrope, but I actually like, and require, time alone with my thoughts.

Since I began working in France, one of the customs that has stuck out is lunch. In the office I work in, everybody takes about a full hour for lunch, and almost always in a group. In my office in New York, it is normal to eat alone at your computer in about 10 minutes, eating socially is the exception for most.

Part of the difference is due to the fact that in France, lunch is generally the main meal of the day -- so having a sandwich for lunch would be like having a sandwich for dinner in the US. Another reason, is that in France, eating is considered a social activity and not just an unfortunate biological function. I have to say that, in general, I am much more partial to the French attitude then the American.

However, I still do miss eating alone sometimes.

Anybody who does creative work (which I count software development as), must find a balance between being alone and being social. We need time alone to think and to create, and we need time with people to communicate ideas, share our work, and get a break from being alone. Going between the two requires a shift in gears that can take time. So, taking an hour break for a social lunch, can easily add up to an hour for lunch plus an hour afterwards getting back into the mindset you were in before lunch.

So, when you eat alone, you are free to work on the problem you failed to solve before lunch, to catch up on your reading, to think about something entirely different, and you can preserve your mindset. Consider of this the next time you feel a pang of shame as you hunker down in front of your computer with a sandwich. Don't let the networking people guilt you into going out for lunch everyday. You have nothing to be ashamed of (as long as you keep the mustard off of the keyboard).

Questions for Review

As the end of the year draws near, it is review time at my company again. Here are some questions that I wish appeared on the review (oh well, they are good for my own personal reflection anyway):

1. What risks have you taken this year?
2. How much code have you eliminated?
3. What tasks have you eliminated?
4. How many people have you pissed off and why?
5. How many projects did you start up that failed? (if you have a lot of failed start ups, thats a good thing)
   
6. Who did you meet this year?
7. How many design documents did you write?
8. What new hobbies/interests that have nothing to do with your job do you have?

The Led Zeppelin Principles of Innovation

The recent Zeppelin reunion and a book on innovation (Weird Ideas That Work by Robert I. Sutton) I have been reading have given me the opportunity to reflect on the particularly innovative qualities of the band.

Whether you think Zeppelin rocked or sucked, there is no denying the innovative success of the band. No, they didn’t innovate the way jazz musicians innovate, but they were probably the most disruptive force to hit popular music in the roughly 25 years between the age of Elvis and the era or The Sugar Hill Gang and The Sex Pistols. Led Zeppelin was not only hugely commercially successful, but they created an entirely new category of music that to this day bears their mark clearly. How did they do it? Here are some lessons I think we can learn from Zeppelin about innovation:

1. Be proud of other people’s rejection.

The story goes that the name “Led Zeppelin” came from an insult thrown at Jimmy Page: that his new band would go over like a “lead balloon”. Who names their band “Led Zeppelin” (in 1968)? Who names a company “Google” or “Yahoo!”? If people think your idea is going to fail, great. That’s less competition for you. If you are doing something innovative, don’t call it something traditional to try to make it palatable to the main stream, put your differences front and center.

2. If you are doing something that seems ridiculous and makes you uncomfortable, then you might be doing something right.

Jimmy Page actually wanted Rod Stewart for the singer in his new band, but Jeff Beck got him instead. The legend is that Robert Plant’s over the top vocal and stage persona bothered Page so much at the beginning that he was ready to fire him after their first tour. What 26 year old seasoned musician wants to be on stage with an 18 year preening hippy doing a bizarre James Brown impersonation? In the end though, it is the un-self-conscious audacity of Zeppelin that made it so different from the myriad of other British bands imitating black American blues.

3. Forget about tradition and following the proper path.

Led Zeppelin basically hacked traditional blues songs and called them their own. While this practice has many ethical issues, there is a lot to be admired in their lack of timidity in appropriating what they liked, and they weren’t overly concerned with following the established traditions of appropriation (i.e. performing traditionals as “traditionals”).

Jimmy Page, Jeff Beck, and Eric Clapton all came out of the Yardbirds, a British blues/garage rock band. Beck and Clapton continued on the trajectories set in the Yardbirds, refining traditional blues motifs and conservatively synthesizing them with rock. Page violated those traditions by stealing riffs and putting them in non-traditional contexts. For traditionalists, the original context is sacrosanct. If you are going to play the blues, you have to play it like a bluesman from Mississippi, not like a Viking. So, while Eric Clapton aspires to play “Crossroads” in as close an homage to Robert Johnson as possible (with some incremental changes), Jimmy Page took a Muddy Waters riff and turned into “A Whole Lotta Love”, the first Heavy Metal anthem of all time.

*Disclaimer: everything above on Led Zeppelin was taken from memory, from my many readings of Hammer of the Gods during the 8th grade. All of you devotees out there, please excuse any minor factual inaccuracies.

Display is Data

Traditionally, the judgment has been that HTML is entirely about display, is generally throw away, and at best is for graphic designers to lovingly handcraft into skillful and brittle, baroque shapes.

Or to put it more bluntly: “HTML is the crap the server has to spit out and we don’t care what it is, just as long as it makes the browser do what the spec says.”

But what about a world where HTML is data returned from web services, display is the domain of CSS only, and javascript runs on the margins of the page and is not embedded into the HTML?

For example:

Say you need to build a page that is going to display a table of data. Let’s say it’s a table of charges on an account that includes name, amount, and date. In standard web development practice, this page would be given to a graphic designer and then to a developer who will plug data into the designer’s display, and you might end up with something like this:

<table class=”dataTable”>

<tr>

<td class=”tableHeaderLeftSelected”>Name</td>

<td class=”tableHeader”>Amount</td>

<td class=”tableHeaderRight”>Date</td>

</tr>

<tr>

<td class=”tableDataLeft”><span class=”bold”>Bob Smith</span></td>

<td class=”tableData”>57.50</td>

<td class=”tableDataRight”><span class=”highlight”>December 1^st, 2007</span></td>

</tr>

</table>

The above example was driven by the following requirements:

1. the first column had to be left aligned
2. the last column needs to be right aligned
3. all other columns need to be center aligned
4. dates need to be highlighted
5. names need to be bold

What’s so bad about this? Nothing is terrible, and in fact, this is a big improvement over a lot of markup I have seen (no inline styles and relatively clear class names), but there are a number of ways this could be improved:

1. use the HTML structure as data (hence the “Cascading” in CSS)
2. break up the classes into semantic tags
3. tag data, not styles

here is the transformed HTML:

<table class=”data accounts”>

<tr>

<td class=”header first selected”>Name</td>

<td class=”header”>Amount</td>

<td class=”header last”>Date</td>

</tr>

<tr>

<td class=”data first name”> Bob Smith</td>

<td class=”data currency dollars”>57.50</td>

<td class=”data last date”> December 1^st, 2007</td>

</tr>

</table>

This not only lets you build cleaner CSS, but also allows you to load the markup with of latent capabilities (for example, lets say months down the road, you want to make all ‘dollar’ data green, you just need to make one change in the CSS), and provides clean hooks for plugging in functionality to this HTML using javascript without actually putting any code inline. This has a lot of advantages in browser mashups where the content of a page is built on the fly and we want to defer the definition of most behviors to the controlling page. For example, if the above HTML were returned as the result of an AJAX request, a querying utility such as dojo,query could be used to pick out the headers and attach event handlers to control sorting in a way appropriate to the particular page:

var headersArray = dojo.query(“.header”);

for (var i = 0; i < headersArray.length; i++) {

//attach event handler to each header
}

Of course, HTML is not an alternative to XML or other pure data formats, but it can provide a useful middle ground in many content syndication scenarios. And overall, data is a good paradigm to start with when implementing HTML. The W3C sums this up well in the Working Draft for HTML 5:

HTML should allow separation of content and presentation. For this reason, markup that expresses structure is usually preferred to purely presentational markup. However, structural markup is a means to an end such as media independence. Profound and detailed semantic encoding is not necessary if the end can be reached otherwise. Defining reasonable default presentation for different media may be sufficient. HTML strikes a balance between semantic expressiveness and practical usefulness

smells like teen spirit

I googled "kolba mashup" to see if my blog would come up.

It didn't.

But I got this gem as a consolation prize. Wow. We didn't have pep rallies like this in my high school.


http://www.tubarati.com/video/uvYl7-aR3Kw/Mini-playback-show-in-Kolba-City-CZARNO-CZARNI--NOGI-:PP

I think my next vacation has to be to "kolba city".

end of mashup camp

Today was the last day of Mashup Camp in Dublin. Here is a quick summary of days 2 & 3 (from my perspective):

Day 2
The first half of the day was given to more presentations as part of "Mashup University". Although, many of the presentations are interesting (and some are very interesting) they start to drag a bit after a full day of presentations yesterday.

Mashup, SOA or RIA?
Martha Rotter from Microsoft kicked off the day with another go at some of her material that had been a casualty of the wireless network the previous day. We saw some more demos of mashups using Silverlight with more of an emphasis on social networking functionality. Martha also touched on codeplex.com: "Microsoft's open source project hosting web site".

As usual, everything coming out of Silverlight looked very slick and pretty, yet I found myself often wondering "is this really a mashup?". This would become a reoccurring theme for the rest of the conference.

AOL & LOL
Next, there were two presentations on APIs from AOL. The first was a very funny one, by John Herren, on AOLs XDrive storage API. (John isn't with AOL, he was just filling in). The presentation is available here.

After John, Stephen Benedict (who really is from AOL) talked about Open AIM. The goals are to allow users to build their own custom AIM clients as well as allow users to take bits of functionality from the AOL messenger and plug it into their web apps. Developers can also write their own bots and and plugins for the client and contribute them. This is cool stuff, an intd Stephen made the very good point that it allows start-up sites to automatically tap into the AOL community.

Also, Stephen demoed what AOL is calling WIMZI. Which is an embeddable IM window that the owner of a site can put on a page using a small amount of HTML. The IM window lets end users of the site message the owner directly, anonymously, and also while protecting the owner's identity. This is very similar to what Meebo has done.

The Mashups are coming...
The first day, I had noticed a number of large poster displays (like the kind you'd see in an actual conference) featuring close up shots of people who are half modern primitive / half corporate suits. The most prominent image was of some tough looking guy with full body tattoos and big honking posts in his ears wearing a button down shirt and tie (and cuff links!). On the top of the poster it reads: "The Mashups are coming. Unleash your inner developer." The whole display seemed incongruous with the spirit of Mashup Camp, and when Serena did their presentation it became clear that the guy in the suit had gone and gotten the tats to mix with the young crowd. Serena is a 20 something old mainframe company that has taken their workflow application and hooked into SOA on the backend and onto HTML on the frontend. Not that there is anything wrong with any of this. I proudly work for a company that goes all the way back to the 1800's and that is in the middle of reinventing itself right now. I just didn't see how this was a mashup. To quote Fight Club: "sticking feathers up your ass does not make you a chicken", and I would say that calling Salesforce.com, while very nice, does not make you a mashup.

Kegerator
The last talk (that I can remember at least) was by Chad Dickerson, who is the director of the Yahoo Developer Network. He of course talked about the developer network and the great work they have been doing. Not just with APIs, Pipes, and YUI (as if that weren't enough), but with design patterns documentation and performance tunning recommendations and the YSlow plugin. If you are a web developer and haven't looked at Yahoo's Design Patterns and rules for Exception Performance, I strongly recommend you do. However, the biggest highlight of Chad's talk was his revelation that he is best known on the internet as a kegerator construction expert.

Geeking Out, Pubbing Out, and Install Hell
In the afternoon, we broke into discussion groups. The discussions and groups were participant lead and moderated, and are one of the hallmarks of the unconference. Unfortunately, at this point, I became caught in one of those installation/configuration hell's of a very deep level as the IBM team labored ceaselessly to determine what was wrong with my QEDWiki setup. About 36 hours and the tremendous help of Will, Flavio, and Dan (thanks again!) I now have QEDWiki running on my local machine -- just a little too late to enter the Mashup challenge.

The discussion I did get to participate in a bit was primarily around widget and service discoverability. The general consensus was that standards should adopted for metadata around widgets and services. While you can't really argue that this is a good idea, I do wonder what can really be implemented that will be practical, work for a wide enough range of situations, and that people will really use.

I am also not so sure that there is that much of a problem with discoverability to begin with. Or at least, a problem that wont be fixed with defacto standards coming out of particular business needs. Also, discovery is where the real value is in mashups, and where some of the key differentiation happens. If discovery were easy, we would have to come up with something else just as hard in order to compete. In other words, easily discovered data has little or no value. So, its nice to play around with, but nobody's going to build a business off of it.

The evening last night ended up in a couple of different pubs, in what was probably a relatively calm Sunday night out for many Dubliners. This morning, things got started a little late...

The Competition
The Competition was definitely one of the most exciting parts of Mashup Camp. After people had put together the mashups, speed geeking began, each mashup creator had 5 minutes to present their mashup to each observer. There was some interesting work done with QEDWiki (there was a QEDWiki mashup challenge as well as the Mashupcamp best mashup award being given - I helped judge for the QEDWiki challenge), and there were a number of mashups being done using mobile phones. The amount of mashup in the mashup varied from one (3rd prize) that very entertainingly made a game out of yahoo news headlines by substituting the terms in the headline with a result from a Yahoo image keyword search, (For example, "Bush Fires Staff" might show a picture of a shrub, a fire, and a big wooden stick, you then try to case what the actual headline is then click through to see the real headline and article.) to many that I would consider to be primarily standard web applications calling free services on the web rather then ones built from scratch. All in all, it was really impressive to see what people had done with very little time and (in some cases) very much beer.

Leaving Camp...
There was definitely a "leaving camp" feeling to leaving mashup camp that distinguishes it from other conferences. It's a nice feeling, and one that makes me reflect on how we can build a stronger development community where I work. I hope I can take some of these lessons back to Reuters.

Finally, as the mashup ecosystem develops I think its important that mashup doesn't become just another check mark to put on software and that it doesn't the term become become diluted to mean just calling a 3rd party service on the web. In my opinion, mashup implies some novelty of use where two (or more) incongruous objects are brought together in a way that creates a third. Or, it could be said this way: Mashups should be easy to build, but hard to conceive.