EWD's Dojo Widgets: the bigger picture

On the face of it, the recent Dojo extensions to EWD may just appear as a way of grafting on a cool set of industry-standard UI components, but if that was all the story, it would imply that if you need fancy widgets when you use EWD, then you have to use the Dojo ones. In fact that's not the case at all. There's a lot more to the exercise than immediately meets the eye.

One of the primary motivations behind the exercise was to create a working demonstration of how to build a set of EWD Custom Tags that would automate and simplify the use of a third-party set of widgets. So let's look a bit more behind the scenes, because, with some outside participation by you, the readers, EWD could become exceptionally powerful: even more so than it is already.

Javascript frameworks and widgets in particular are a very interesting area because you can tap into huge amounts of work that has been already done for you in the design, usability and cross-browser support of these often highly complex components. There are many of them, all of them free to use, including:

- Dojo (http://dojotoolkit.org/), of course. The "grand-daddy" of them all, but notoriously complex to use, often buggy or highly sensitive to being used "just right", and historically at least, extremely badly documented.

- script.aculo.us (http://script.aculo.us/). A very popular widget set, if you can sift through the (IMHO) apallingly laid out web site and find something you'd like.

- Yahoo YUI (http://developer.yahoo.com/yui/). Some very nice widgets here from a major force in the Javascript world, with gurus such as Douglas Crockford behind what they do. Their widgets require a lot of Javascript coding in your pages.

- Interface Elements for JQuery (http://interface.eyecon.ro). Not so well known, but wow! Their widgets are beautiful!

- Backbase (http://www.backbase.com). An up-and-coming force in the Ajax world, again with some extremely sexy widgets, and a very interesting XML-based underlying methodology.

Of course you could spend the time to figure out how to use any one of these libraries and manually hand-craft the widgets into standard EWD pages. That, in itself, is an important feature of EWD that should be understood: there's nothing in EWD that prevents you using any or all of these Javascript libraries. Having said that, most of the libraries have, at their core, a set of solutions for handling the "round-trip" Ajax plumbing (based around the use of the browser's XMLHTTPRequest object) and data binding. These are superfluous if you use EWD: it already contains its own proven built-in solutions.

There's several problems with manually hand-crafting third-party widgets into your EWD applications:

- first, you need to understand the mechanics of the Javascript library you're using. That means a lot of effort, ploughing through their documentation, trying out examples, tweaking them till they do more the kind of stuff you'd like to do. Without exception, all the libraries have a steep learning curve and despite their goal of making it simple to do complex things, they are complex environments to create and use. Dojo is a particularly complex beast.

- secondly, how do you decide which library to use? To do a thorough comparison would take way too long for it to be commercially viable. Can you cherry pick and use one widget from Dojo and another from Backbase? No, it's usually an all or nothing commitment to one, you invest time and effort understanding it and then you're tied to it for better or worse.

- thirdly, the code you typically have to embed in your pages to make the widgets work is, quite honestly, horrendous. One of the most important principles of EWD is that your pages should clearly describe WHAT your pages are trying to do, leaving EWD to generate HOW they do it. Whilst Javascript libraries certainly make it a lot easier to implement complex UI widgets, nevertheless they are still necessarily pretty low-level beasts. Hand-craft them into your EWD pages and a year from now I can guarantee that you'll be scratching your head trying to figure out what they're all about if you need to maintain a page containing them. Worse still it probably won't be you maintaining the page. The result will typically be what I've seen so many times and despaired: it becomes quicker and easier to rewrite the page from scratch than figure out how the old version works. That this is tolerated at all is unbelievable, but my observation based on 12 years experience in this world is that, sadly, it is the commonplace state of the art!

So where's this leading? Well EWD has a secret weapon up its sleeve: EWD pages are actually an abstraction of the pages you'll really run within the environment of your choice. They have to be compiled down into the run-time version that will run on PHP, CSP, JSP or whatever. This is how EWD allows you to have the best of all worlds and describe your pages in terms of what they do: the compiled pages that EWD's compiler generates are all about HOW they actually work, chock full of all kinds of nasty run-time-specific stuff that you'd otherwise have to hand-craft yourself. EWD's XML DOM-based compiler performs the transformations from your high-level tag-based description in your EWD pages into the nasty run-time stuff in, say, your PHP pages. You never really need to look at all that PHP (or CSP, JSP etc) code. If you need to modify or maintain a page, you just modify the high-level EWD page and recompile.

The other secret weapon within EWD is its Custom Tag architecture. All of EWD's own built-in custom tags were implemented using this, and, from the start, the architecture has been exposed and available to the end user to craft his/her own Custom Tags. At the heart of an EWD custom tag is its "tag processor": a method you design to transform an instance of your XML-based custom tag into whatever it needs to be expanded into in the run-time page. You describe that transformation as a set of XML DOM manipulations on your EWD page. This is exceptionally powerful stuff. There really is no limit to the complexity of the transformations you can carry out. And to make matters even better, your transformation can make use of EWD's own custom tag primitives, leaving them to look after the technology-specific aspects, so your Custom Tag will automatically work in PHP, JSP, CSP or whatever!

And so back to those Javascript libraries. My implementation of the Dojo tags was a case study in how to design and express the use of a set of complex third-party widgets as a set of very easy to use EWD Custom tags. I hope you'll agree when you look at the source code in the Dojo examples I've made available on our web site, the EWD/Dojo Custom Tags are all about describing WHAT you want to do: eg I want to have a slider at this point in my page. EWD's compiler and my Dojo tag processors look after converting this to the nasty mass of Dojo tags and Javascript logic that needs to be embedded in your page. Apart from idle curiousity, you'll probably never need to know or care how it's actually done at the Dojo level.

As a result, anyone can now use the Dojo tags without really having to spend any time ploughing through the reams of Dojo's documentation and examples. That steep learning curve has been removed and you can quickly give Dojo a go. And of course a year from now, anyone can pick up that page and see that you used a Dojo slider at that point in the page and tweak its behaviour as required.

And so to where I'm trying to lead this discussion. What I've done is provide a template of how a third party widget set can be expressed as a very simple-to-use set of high-level EWD Custom Tags. It's not intended to be interpreted as a lock-in to Dojo in any way. Quite the opposite. What I'd like to now see is the equivalent implementation as EWD Custom Tags of as many of the Javascript libraries out there as possible. My problem is time. It takes a significant effort to do this. The main effort is actually getting to grips with and understanding the way you use the Javascript library and all its subtle nuances (eg the most complex part of the Dojo implementation was figuring out how it could work with the Ajax fragment replacement mechanism in EWD). The actual custom tag processor development, once you're familiar with DOM processing, is pretty trivial. For example the Dojo FishEye menu custom tags were written on a Sunday afternoon whilst also cooking the dinner!

So there's my request to you, the reader. I'm seeking volunteers to help implement all the major Javascript widget libraries as EWD custom tags. I'll provide as much assistance as you need to understand the mechanics of how to go about such a task. For example, I'm thinking of running an in-depth seminar on Custom Tag creation during DevCon if anyone's interested.

The rewards to the community will be immense. What we'll then have, within the Cache/GT.M community is, without any argument at all, THE most powerful and flexible Ajax/Web 2.0 development environment on the planet. Anyone will be able to use whatever widgets they like without any of the steep learning curve and maintenance nightmares. Users of EWD will be able to focus their efforts where they really should be focusing their efforts: on their application's functionality, concentrating their efforts on WHAT their applications do, rather than fiddling around wasting time and effort on HOW their applications need to do it! EWD + Cache or GT.M will be demonstrably THE most productive web/Ajax development environment, bar none.

Volunteers please? Contact me at rtweed@mgateway.com if you want to help in what I believe will be one of *THE* most influential projects for our community.

I was just reminded by

I was just reminded by someone of another very cool Javascript widget library - Ext JS (http://extjs.com). An amazing example of what it's capable of is demonstrated by http://qwikioffice.com/desktop-demo/login.html

Now wouldn't THAT kind of sophistication be nice to have at your fingertips!

Nice to see my desktop

Nice to see my desktop mentioned :)

I couldn't not mention it -

I couldn't not mention it - it's an incredible example of what's possible these days with browsers. A great piece of work.