Dan Webb has some important thoughts about the Flash v.s. Ajax debate and the comments on his site and on the Ajaxian site are also thought provoking.

Indeed it is anyone’s guess how WPF/E, Flash/Flex/OpenLaszlo, AJAX (Backbase, YUI, Dojo, Prototype, Qooxdoo,…), GWT, Eclipse/RAP and Morfik will mature but this year will be definitely very interesting for web application development!

ACAP (Automated Content Access Protocol) is a new initiative from the international publishing community to turn the challenges facing the industry from web technologies (especially search) into opportunities in a win-win way, and as a side effect can help Web Applications out, too.

First a summary of the publishing angle.

Right now major search engines (especially Google) are crawling (retrieving), indexing and storing web content without being aware of the special needs of publishers. This leads to well documented cases when Google faces charges of massive copyright infringement for its activities. In some special cases Google stores a copy of content publicly available on the net, and later when the publisher changes its policy about a particular piece of content (going from public to a fee or membership based structure) the content is still present in the search results of Google and can be retrieved from it.

It must be underlined that Google (or other search engines) cannot be faulted (at least morally) for doing this since they are indexing millions of web sites automatically and it is impractical to implement special cases for certain specific sites at this scale. On the other hand, publishers do have their moral and legal rights to protect their intellectual property.

A special twist of the situation is that for publishers it is indeed important to be indexed: this helps them to generate major exposure for their content.

This creates a challenging case that seems to be a loose-loose situation: either search engines are constantly sued for copyright infringement (where they can fight back with fair use, etc.), or publishers instruct the search engines not to index their content at all (e.g. with robots.txt) that means lost exposure and thus lost revenue.

An important element of the case is that both parties want to cooperate so if there were a technical means of communicating the intentions/requirements of publishers, it could be converted into a win-win opportunity.

That’s where ACAP comes into the picture. The publishing community has decided to establish a standard way of communicating permissions information, that can be automatically adhered to by the search engine crawler.

Technically it is a challenge to implement such a solution. Although there are a handful of major search engines, there are literally millions of publications on the web that are created and maintained with a lot of different tools in varying structures. Thus such a solution has to be established that can integrate well with all these various platforms and technologies.

In a well designed solution search engines would be able to index even copyrighted material and during a web user search session return only contextual excerpts with proper attribution and in case of non-free content even pointers to how to access the full content, thus creating an invaluable means of dissemination of such content. This could be augmented with publisher-provided taxonomy and auxiliary information that would help the crawler to set the Page rank (or similar) of the particular material.

Now how can this be utilized for Web Applications?

Of special interest is the upcoming Web 2.0 and web application technologies that make it very difficult for crawlers to index content properly. ACAP could be extended in a way to solve this issue so that search engines would benefit from an easier to crawl content with better signal-to-noise ratio and publishers would be able to have fine-tuned search results.

A trivial way of making AJAX apps search-friendly is to fundamentally prohibit indexing the AJAX application itself and direct the crawler to an (otherwise “inaccessible”) area of the site where all the information that the content owner wants to be indexed is available in a search friendly form. An interesting twist is to use such an URL scheme, that when the web server detects that the incoming request is not a crawler, then it redirects the request (based on the URL) into the AJAX application.

This way crawlers would only receive content that is really valuable, they would be more effective with less effort and search results would improve significantly. Content owners would be able to make their AJAX sites searchable the way they like it.

Combined with ACAP (for IP protection) and with the probable extension of the protocol to include “nice permalinks” in the searchable content into the AJAX site this may be a good solution for search in the Web 2.0 era.

Would be a way cool project to work on the specification/implementation…

When I develop with Morfik, it uses Internet Explorer (6.0 XP/SP2) by default to run the application. I discovered the performance problems this way while running the app from the IDE.

Today I started the app from the command line (I wanted to take a look at the generated code) and it came up in Firefox (1.5) since that is the default browser on the same machine. Was I surprised seeing that it was noticeably faster to switch modes!

My first reaction was to think of a few ‘nice’ words about my ignorance in forgetting how an IDE can affect performance by doing some extra hops for debugging in the background (in retrospect: is this so in Morfik’s case? I’ll figure that out…).

Then I remembered that indeed I had done some profiling the other day and narrowed down the possible number of lines for the performance issue quite a lot and none of those lines should be affected by IDE magick.

So decided to do some investigations and did profile the same task (switching from Basic into Expert) in both browsers on the same machine with the same conditions (only the two browsers loaded, average of 5 runs for each). Here are the results:

• Internet Explorer finished on average in 18.34 secs
• Firefox finished on average in 3.38 secs

Now, that IS a difference. In this test Firefox was faster by cca. 5.42 times.

One could say that this is irrelevant and why do I care in the first place? The answer is simple. New web applications (based on AJAX) and many Web 2.0 frameworks (Morfik, Backbase, the new OpenLaszlo DHTML renderer, Ruby on Rails, etc.) rely on the browser as a UI rendering engine. Such great differences in speed mean that either the browser manufacturers will have to tune their solutions to be more in line or we application developers will have again to do some magic to mitigate the differences (of course, this is not optimized code, but that is not the point, OK?).

If you want to take a look at the results first hand, you can do it by either running the Mine Sweeper app from the Morfik Dev2Dev site or downloading the source code and running it locally (it is pre-compiled in the archive).

It is a considerable burden on the user: she/he has to wait for the code to download and depending on her/his subscription this also decreases the quota she/he has for the month or costs money (e.g. for mobile users). Bad for the application hosting company too: needs huge resources to transfer material that virtually does not change for months and can be comparable in size to the actual data that has to be transferred. This is also bad for the whole Internet/intranet networking infrastructure as well because of the traffic. I see this one of the biggest issues in the proliferation of AJAX/smart client applications.

What can we do to enhance this situation? Let’s build a fundamental component of the WebOS: the standard of application caching in the browser (WOSTOC). Browser companies and AJAX framework vendors should work out a common standard for partitioning the code (and auxiliary items like images, sound files, etc.) for applications so that code should be easy for the browser to identify and cache as long as it is not changed.

(As a side note: for many Web 2.0/WebOS application frameworks, the browser will become the virtual machine running JavaScript as the common code base; Morfik’s novelty is recognizing this and going one logical step forward: using JavaScript as the intermediate language/machine code to which it synthesizes form a “traditional” high level language.)

We should avoid two common problems from the past: DLL hell and security issues.

We get to DLL hell if two applications try to share a common library but of different versions. One of the applications will fail due to incompatibilities of the newly installed library code. To avoid this, we should make sure that code is not shared among applications.

There may be several security issues. One is IP protection: the code should be transferred and stored in an obfuscated manner so that reverse engineering and tampering with is made more difficult.

Another related security issue is making sure that tampering with is detected and rectified: the browser should ask for a hash/fingerprint for each file it needs to download and check with its local copy whether the fingerprint matches, and if not, should re-download the complete code. This way we could also get automatic updates for free.

Additional security issues should be uncovered and addressed too.

Using tokenization as part of the obfuscation could also help in speeding up the execution of the code (backward browser compatibility is an issue, though).

We should define this new standard in a compatible way: browsers that do not support the WOSTOC protocol would see the applications as they see them now, probably a bit better structured (and thus even their limited caching abilities would work better).

What is the difference between WOSTOC and current caching technologies in the browsers? Technically virtually nothing, and that is good. Browse developers and application framework developers should agree on a common protocol, examine and iron out security issues and application framework developers should make sure that the applications are well partitioned so that RTL code can be easily cached, application code regularly updated as needed, and dynamic application data is excluded from the overhead of caching checks.

In summary, we would get to a much more responsive web experience with much less load on the infrastructure and better performance on the client, too. Since all contemporary browsers already support caching and most of the protocol is more an organizational/standardization issue than a technical one, WOSTOC 1.0 could be defined and implemented very quickly.

While Morfik is a complete integrated development environment for web applications (xApps in their parlance) both Backbase and OpenLaszlo are “only” client-side UI toolkits, but very capable ones. Morfik will have to add some eye-candy, too. Hmmm… stay tuned ;-)

I have been developing interactive software for twenty years now and it is encouraging for me to see and experience how all this new Web 2.0 stuff is advancing usability. Of course all these technologies are only tools that can be used for good or wrong, but there are a lot of new applications developed with these tools that are really pleasant and easy to use (KISS, anyone?).