Lars Trieloff's Collaboration Weblog

I had a short e-mail exchange with Peter Svenson yesterday, who has some really interesting ideas on web development, Javascript and frameworks and wanted to introduce Sling to him. Unfortunately the Sling website is a bit terse on "big picture" introductions, so I though I share my introduction with my blog readers.

What is Sling? This is a fairly complex issue, because Sling combines many aspects. The main idea is that Sling is a web framework that uses a JCR container such as Apache Jackrabbit as repository. The interesting part here is JCR, which is a storage API that allows you to store nodes which can have child nodes and properties, so the best way is to imagine a JCR repository as a giant storage tree or as a file system on stereoids, as it supports typing, transactions, versioning, full-text-search, oberservation and so on. The main role of Sling is being a translator between the storage repository and the client application and this means representing the Nodes in the JCR tree (that have a path) as Resources (as in REST) that have a URI. ;:A resource can have different representations, for instance a blog can have the HTML representation (blog.html) and the Atom feed representation (blog.atom). Representations are not limited to the resource type, there could be another HTML representation, for example the archive page (blog.archive.html). Representations of a resource are generated on GET requests and there are two high-level ways to do it: Servlets (the classical, heavy-weight Java way) or Scripts and Templates. Among the script engines currently supported are Rhino (Javascript), Ruby, Velocity and Freemarker. Basically, there is a script file or servlet for each representation (in our example html.js, html.archive.js and atom.js could be imaginable). If you want to create a new resource type, e.g. blog entry, you just have to create a new folder and put the appropriately named scripts into this folder. Secondly, a resource can have different behaviors. Again, behaviours are implemented either via Servlets or Scripts. The default behavior is to create a node when posting content to a collection, replacing a node when sending a PUT to a resource and deleting a node when sending a DELETE request. You are, however able to override this default behavior by placing scripts into the right folder. These scripts could for instance validate comment contents before creating the node. This is basically where server side scripting comes into play.

Microjax is based on a clever default handling of certain requests. This means representing resources as JSON objects on GET request and processing JSON request entities for PUT and POST. Additionally, Sling and Microjax introduce two Javascript-based templating mechanisms. Sling comes with the built-in ESP, server-side templating mechanism that will allow you to mash up HTML with embedded Javascript that will be executed server-side, e.g.

<ul>
<% foreach (child in node.children) {
 %><a href="${child.path}"><%= child.properties.name %></a><%
} %>
</ul>

Microsling extends this pattern by introducing EST, which allows you to write the same script, which will then be transformed by the server into a proper Javascript with lots of document.write(...) expressions that can be executed on the client-side.

To sum it up: Take a look at Sling, it is an interesting way to build content-centric applications.

I am happy to tell you that Sandro Groganz, the marketing hero that pushed Mindquarry out of nothing to thousands of downloads, articles in the best publications and instant awareness is now available for hiring: Sandro Groganz officially started his Open Source Marketing Consultancy and will provide marketing consultancy to companies creating (but not officially making money from) open source (but Sandro can tell you how to change this).

Via DZone I found pdf-renderer, a 100% pure Java library for rendering PDF files. Unlike existing libraries like iText or FOP, which concentrate on creating and writing PDF files, this library allows to display PDF files on screen and conversion into images. This - together with the upcoming Java Runtime enhancements would make it possible to create an applet-equivalent of Adobe Flashpaper easily. pdf-renderer is LGPL-licensed, which makes it possible to embed it into proprietary applications, but not Apache projects.

Congratulations to the Apache Roller team for releasing Roller 4.0. Now, all I have to do is updating weblogs.goshaky.com....

Dave Johnson's wrap-up of the ApacheCon contains some interesting pieces: Of course he mentions the Shindig proposal, which I hold as one of the most interesting developments in the social networking space and he has written a longer paragraph on combining Roller (the weblog software) and Jackrabbit (the JCR repository that is the core of Day's CRX).

The idea of using a content management system to store Roller content keeps on coming up. At ApacheCon EU earlier this year, I spent some time talking to Lars Trieloff (who now works for CMS vendor Day Software) about implementing the Roller back-end interfaces using the Java Content Repository (JCR) APIs instead of the Java Persistence API (JPA) that we use now.

My rationale then was, to allow true free form collaboration in Mindquarry, we needed a weblog system. Mindquarry is based on Jackrabbit and I did not want to open another repository backend then, so I thought about creating a JCR-based backend for Roller that would easily integrate with Mindquarry.

At this ApacheCon, Noel Bergman brought up the topic a couple of times and pointed out that Day Software, has blog and wiki modules that are both backed by JCR. We could do the same thing: create version of Apache Roller and Apache JSPWiki (incubating) that share the same content repository.

The main advantage is that Roller and JSPWiki are content-centric applications. Every well-designed content-centric application moves sooner or later into the direction of having a separate repository layer. In JSPWiki the repository-layer allows you to have different backends, from flat files, to RCS to Apache Roller. In Roller there is a domain-specific repository implementation that is called "model", but if you have read my recent posts on microsling, you will note that using Model-View-Controller (MVC) for content-centric applications is disgusing the content-centric nature of the application, which would need a Content-Behavior-Appearance (CBA) model.

Later, Jukka Zitting (who also works for Day Software), suggested the idea of implementing JPA itself with JCR, thus allowing Roller to store its content in a CMS in a totally transparent fashion. This topic is interesting to me, but I don't fully understand the benefits of backing blogs and wikis with JCR. What new use cases would this support? How do the interesting features of JCR, like versioning for example, bubble up through Roller -- especially if Roller is to support both RDBMS and CMS back-ends?

I had a chat with Jukka yesterday in which he pointed out that implementing JPA based on JCR could be a very-cost effective solution and it might be the ideal way to go to migrate applications stuck with relational or object-relational backends to a content-based backend. Of couse, you would lose many of the advanced features that JCR offers you like full-text-search, observations, versioning because you have to mainatain backwards-compatibility to relational databases.

As always, this is a questions of frameworks and the right time to start a project. When the Roller project was started, there was no JCR, no Sling and no practical and standardized way of implementing content-centric-applications. With this in mind, it is easy to map some of the features JCR is offering to the needs of a blog application:

versioning

keeping all versions of a blog post, allowing incremental writing and backup

observations

notifications for new comments

workspaces

having a draft and a publish area for posts

hierarchy

posts belong to weblogs, comments to posts, etc.

export

backup

queries

tagging, categories, full-text-search

Disclaimer: I am Day's product manager for collaboration products, namely Blog and Wiki and I am a long-time user of Roller and JSPWiki. This blog post was written on Roller, using the JSPWiki plugin.

It is good news to see that the The Eclipse Integration for Apache Maven has been approved and that there will be official Maven integration into Eclipse, based on the work done by Carlos and other for Q4E, really soon. (via Carlos).

In other news - if you are using Maven for your Java builds, which you should, have a look at Brett Porter's slides on Maven Best Practices from ApacheCon. (via Steve)

To my suprise I found out that someone posted my presentation slides on slideshare.net before I actually held the presentation. I usually post my slides only after the presentation to update them to reflect questions that came up in the Q&A part of the session, but at the Web 2.0 Expo in Berlin things seem to work a little bit differently.

The presentation went very well and we had a very interesting discussion at the end, so I would guess the awareness of the importance of internationalization in Web 2.0 is rising. One of the most interesting aspects brought up was: Is providing textual translations of a web application enough?. My take on this question is that providing a multi-language user-interface is only the first step in true internationalization, afterwards you have to go for localized content, design and business processes that respects the culture of your audience.

If you are at my presentation, please make sure to give feedback at feedback.berlin.web20expo.com. Thank you.

I could have twittered it, but this is too important to go down the lifestream that fast: Dojo 1.0 is released. Dojo is the most powerful, most solidly engineered open source Javascript and AJAX toolkit and everyone who is still developing his own widget system has no more excuse of not using Dojo, now that it has reached 1.0 and no more API breaks are to be expected. Congratulations to the Dojo team!

In the first part of my micro-series I have come to the conclusion that a content-centric web application allows the user to interact with the content by the means of renderers and behaviors. These renderers and behaviors can be expressed by means of pipelines and processes.

A pipeline takes the existing content, transforms it several times and finally sends a new representation of the content to the client. This concept is anything but new, one of the most successful web application frameworks at apache is actually Apache Cocoon where pipelines are based on the idea of transforming XML events by means of XSLT, STX and other transformation languages.

Content behaviors can be seen as processes in the sense of business processes that trigger events, process input data and eventually manipulate content. A process can be expressed formally by a program or a script, or a graphical process programming language like BPMN.

If you take a look at a typical processing pipeline used by Cocoon, you will see something like this:

<map:match pattern="myThirdPipeline">
  <map:generate src="myXMLFile.xml" type="file" />
  <map:select type="resource-exists">
    <map:when test="myIncludeFile.xml">
     <map:transform src="myXML2include.xslt" type="xslt" />
     <map:transform type="include" />
    </map:when>
  </map:select>
  <map:transform src="myXml2PdfFile.xslt" type="xslt"/>
  <map:serialize type="fo2pdf"/>
</map:match>

The constructs involved here splits (<map:select>) conditions (<map:when>) and joins (</map:select>), activities ({<map:transform>}) can be found with different names in any workflow language, thus another way to interpret rendering pipelines is to see them as processes as well.

A content centric web framework's most important task is mapping content to processes and providing an execution environment for these processes.

(The FMC-construct used here is called structure variance. It means that the framework dynamically creates the renderer and behavior actors, depending on the content requested.) The framework maps a HTTP request to a content resource, identifies the process description for the kind of request ('real' processes that change resources for PUT, processes with side-effects for 'POST' and rendering processes for GET requests), e.g. a script or template file, dynamically instantiates the actor by interpreting the script or running the template engine and connects the request-handling actor to the content and the request and response.

In the next section of the series I will show how a framework like Sling handles request-to-content-mapping, content-to-process-mapping and how processes are enacted.

Yay! Blogbridge 6.0 is out and adds all the features I had to wait for because I did not dare to install the weekly builds (mainly due to the flaky Java Webstart support in Mac OS X). These new features include:

• built-in Meme tracker
• Statistics
• Plugin API

Now: Installing and reading feeds.

I've uploaded my second presentation "Mindquarry for Cocoon Users" to slideshare.net. And as far as I see, Ugo has added his presentation to the Cocoon group at slideshare.net as well.

I just had my first presentation at this year's Cocoon GetTogether in the Rome Bioparco about DAX - Declarative API for XML. I've uploaded the slides to slideshare.net and would invite all other presenters to do the same and add their presentations to the newly created Cocoon group at slideshare.net

You probably have read about the end of Mindquarry PRO and Mindquarry GO, which I don't have to tell you, is very sad for all of us, but I would like to assure you that this is not the end of Mindquarry as an open source project. As long as there is a community that cares for Mindquarry I will continue to invest my time in Mindquarry.

This is the second session I moderated in the Office 2.0 Unconference 2007. I started with a prepared presentation on open source tools for Office 2.0.

The tools covered are

• Roller - Blogging
• Mindquarry - Collaboration
• Jabber/Tigase - Instant Messaging
• Chandler - Calendaring

The presentation incited an interesting discussion, some of the questions I can remember are:

Isn't Chandler a dead project? It has been in works for seven years, but no release has been made and you can see zero adoption.

From a technical point of view, Chandler is by no means an abandoned project. Eight active developers, frequent releases and a product that is technically up to date tell a different story. However, from the viewpoint of adoption, Chandler is not yet a big success. The main reasons I see are:

• No 1.0 release
• Community: most chandler developers are still employees of OSAF
• No commercial backgrouding providing support for in-production scenarios.

What are the main obstacles for integrating these four projects?

The main topics for an integration effort would be single-sign on and user-management, single user interface and data integration.

What is the open source business model?

This has been answered before, for example here.

How does the MPL (Mozilla Public License) work

Mindquarry is using the MPL, this caused the question. Under the MPL, interested developers can take the source code, modify and extend it and sell the modified and extended product to their clients provided all modified source code (but not added code) is made available under the terms of the MPL again. This model benefits the original developer, the extender and the customer.

Via Dave Johnson I learn that there is a proposal to move JSPWiki to the Apache Incubator, which I regard as a great idea as the Apache Community and the JSPWiki community could benefit from each other and JSPWiki one of the two Wiki systems I use daily.

Jitendra Gupta wrote an interesting article at Read/Write Web: Open Source Economics Driving Web 2.0 Innovation which explains some Open Source Business models and how they play nicely with Web 2.0. Much of this article is based on the research of Dirk Riehle, a scientist at SAP Labs whom I met some time ago. Jitendra sums up the reasons why contributing to open source makes sense to developers as follows:

It used to be the case that contributors did not get paid for their contributions and had to work on their own time, but things are changing. With the popularity of open source projects, more companies are paying contributors to support the community, or are even contributing proprietary modules to the community. This provides direct incentives to contributors to work with the community.

Open source participation is a great way to establish credibility if you are a programmer…Have you seen ads like these: "With your resume, please include some php and javascript code snippets or refer us to an open source project you've worked on."

Most good open source developers have an opportunity to become a public voice for the project. This extended role for engineers not only means an ego boost, but also translates to higher salaries etc.

As a developer of proprietary software, most of your know how is bound to this proprietary product and when you leave the company, this know how is in most cases of no use to your new employer. With every piece of knowledge you gather in proprietary development, you become more valuable to your current employer. As a developer that contributes to an open source project, e.g. because your employer uses this open source software you become an expert in this particular open source software and gather knowledge that is valuable to your current employer, and to potential future employers. This allows you to bargain stronger for your next raise.

You can increase your visibility to even further by using services like SourceKibitzer: This service donwloads source code from public open source subversion repositories, analyzes it for various metrics that are all related to your actual contribution and allows you to host a developer biography to tell even more about your skills. A similar service is ohloh.net, which in contrast to Java-centric Sourcekibitzer covers all programming languages popular in OSS development, but offers less visibility to developer profiles and has no Java-specific metrics.

There has been another release of Mindquarry, the Open Source Collaboration Software. Intended to be only a bug fix release there have been visible usability improvements additionally to the 43 bug fixes.

My slides for my talk Supporting Technical Documentation Processes with Open Source Tools held at Linux Tag 2007 in Berlin are available at SlideShare: Supporting Technical Documentation Processes with Open Source Tools Slides.

We just released Mindquarry 1.1 the latest version of our open source collaboration platform. Since 1.1-beta we have fixed lots of bugs, updated documentation and changed the design of some dialogs slightly. Since 1.0 we have added lots of new features including timelines for all content, an AJAX-powered interface, a better WYSIWYG Wiki editor and some more things I already blogged about when Mindquarry 1.1-beta was out. In other news, I will go to Boston this weekend and for the whole next week, because Mindquarry presents at [Enterprise 2.0. I am very excited, not only because I am visiting the united states for the first time, but also because I am going to present some new features in that will be part of Mindquarry 1.2.

If you are visiting Enterprise 2.0, please send me a mail, I would be delighted to meet you, present our new features and talk about collaboration, open source and web technology.

I've been invited to moderate the Web 2.0 track of the Linux Tag conference. The first presentation this day was Semantic Web, Wiki and Mashup: how they can all work together by Nicolas Vandenberghe. In his presentation Nicolas introduced ITerating, a software directory for open source software, proprietary software and software as a service that

• is editable like a wiki
• pulls data from RSS feeds published by Sourceforge, Freshmeat, etc.
• Stores everything in a Triple Store powered by JENA
• Outputs data as RDF+FOAF
• Outputs data as RDF+DOAP
• Outputs data as RDF+DublinCore
• Supports Reviews

To me it looks like the the open source software portal Ugo Cei was looking for and it is one of the first portals I know build upon semantic web technology.

Howard Anderson rips open source and shows some common misconceptions about open source:

Let me give you an example. If you are No. 1 or No. 2 in your industry, you hate open source. You make your money by selling proprietary solutions: Microsoft and Cisco. If you are No. 3 to No. 10, you look at open source as a way to get back to those serious RSEUs, because they are where you make money.

As a proprietary #1, you hate open source, because it drives your prices down. Even if it is not feature-equiavlent to your offering, it still can take away a significant part of your user base (those who are happy with a 20%-solution) and, as it has the potential to develop faster can finally disrupt your market.

As Dirk Riehle put it in a recent talk, if you are a number 1 and fear your competitors going open source, go open source yourself. Accordingly if you are not the number one in your market, you can go open source to cut this deficit, reduce costs and reduce prices to attack your competitors. It is very clear economics, with no need for an religion.

Some bold users actually will have open source as part of their master plan; some will find that their smartest, young computer-application designers will migrate here naturally because it is “cool” and they are true believers. They may actually have some open source projects just to keep this next generation happy.

In most cases, these young application designers will not go for open source components because of coolness or because they are true believers, but because they have no other choice. Try to build a modern web application without using open source software and you will fail.

A cynic might suggest that the people writing open source software are the ones who are making their daytime living working for a proprietary-solutions vendor and spend their nights tearing down the very house they live in. And that if open source replaced proprietary solutions, these people would not be able to make a daytime living that supports their night time hobby.

Another common misconception, is that open source committers are not getting paid for their work on open source projects. At least for high quality community open source projects, e.g. those hosted by the Apache Software Foundation, this is clearly not the case. Most Apache committers are getting paid for their work, needless to say, developers in commercial open source companies like Redhat, MySQL, SugarCRM or Mindquarry are getting paid as well and a study by Roy Fielding has shown that open source committers in fact recieve higher salaries than equally qualified non-committers. Thus it actually pays off to be a committer.

A cynic would be right.

Unfortunately being cynic is not enough, as being objective is a requirement for finding truth. (via Vineet Sinha)

Having heard Ugo Cei's presentation on Web 2.0 open source project directories just some days ago, I had another look at Ohloh, the open source project directory that offers some interesting metrics on listed open source projects - and to my surprise, Ohloh was improved with many new features:

• adding a project to your stack has become much easier
• you get suggestions for software based on similar stacks
• you can find out what other projects people are using that have stacked a project
• there is a user and contributor map of users who have entered their location to their profile
• you get an experience report based on the commits you have done on open source projects, so you can prove you have 25 years of HTML experience
• you can give kudos to other developers and there is a ranking of contributors to open source projects based on activity and kudos' count

It is an interesting set of new features, even if Ohloh does not yet provide DOAP exports or activity feeds, which would help creating mashups based on this data, and even if the metrics can be inaccurate and popularity can be faked, I like the concept and the way Ohloh presents the gathered data.

So far it has been a great conference. I hda the opportunity to meet many interesting people, have many interesting discussions, among those the BOF of the text processing crowd that discussed details of the incubated Tika project, a content-extraction engine that can be used in search engines like Nutch, in ECM and document management servers and other tools. Once again I took the opportunity to get some code done with Jeremias by updating Batik's POM to the latest release which is among other things able to process the Tango Icons, so it will be possible soon to process icons using Cocoon and Batik on the fly to generate PNG or GIF buttons.

Friday, 2pm, I will give a 15-minutes-presentation in the fast feathers track about lessons learned integrating Cocoon, Jackrabbit and Solr, the Apache projects we have been involved most closely when implementing Mindquarry and I hope to see you there and share experiences.

Meet me in Amsterdam next week at ApacheCon. Alexander and I will be there, if you want to talk to me in person, drop me a line (lars@trieloff.net) or leave a comment.

In the last days I have been blogging a bit too much about Mindquarry, so here are three open source software releases you should not miss:

• BlogBridge 5.0 - BlogBridge is still the best open source desktop feed reader and with the addition of the new BlogBridge service which allows posting to your weblog directly from blogbridge. This makes it ideal for short-circuted information workers (remind me to blog about this concept)
• Eclipse 3.3M6 - another Eclipse milestone. This time with improvements for Mac OS X and Vista.
• Hudson 1.91 - with support for version control repository browsers. Currently there is FishEye, ViewCVS, and ViewSVN support. When I have time some day, I will write a plugin for Hudson that provides better integration with Mindquarry, I promise. I have already checked out the source code and built it once, but there is also a lot of other stuff to do, so this might take some time.

I am currently in Cannes, France at the Red Herring Venture Market Europe. Yesterday Stephan and I took the award for Mindquarry being one of the top 100 private companies in europe and later in the evening we used the opportunity to speak to founders and CEOs of other european top 100 companies like Manel Sarasa of openbravo, the open source ERP and Stéfane Fermigier of nuexo, an modular open source ECM.

The last two days Jan, intern at Mindquarry spent fixing a bug in Cocoon's AJAX form handling. In the end Alexander and I joined him because we had more experience with Cocoon's internals and finally we found and fixed the problem. Having identified a bug, but not the bug's cause or resolution is a terrible feeling. You try this approach, you try that approach, you discuss, consult search engines, and feel more and more incapable of understanding the problem. But once you have found the solution, you feel great, you feel like being able to grasp the most complex technical structures.

I keep saying "Software development is 90% baning a head agains the wall and 10% breaking through the wall". Bug fixing multiples this principle. 99% of the time, the actual bug fixing, searching for the cause and solution makes you feel bad, but the 1% of time when the bug is finally fixed is simply great. The joy of bugfixing outweights the pains of fixing the bug, this is way bug-days when a whole development community spends a day fixing bugs are so popular.

From my posting history (only del.icio.us links in the last five days) you may have seen that I have been very busy in the last days. The reason for this is now visible for everyone: The new Mindquarry Website is online and, even more important Mindquarry 1.0-M1 is available for download.

Mindquarry is a teamwork support software that supports the teamwork of software developers, technical writers, engineers, scientists, designers and analysts by enabling file sharing with version control, easy-to-edit WYSIWYG-Wiki and a flexible task management system that features customizable fields that can be added on-demand. The basic idea of Mindquarry is to use tools that have proven their ability to solve teamwork problems in the open source community like version control systems, wikis or bug trackers and to make them available to other information workers (those without a degree in CS) by integrating the tools and providing a simple, streamlined user interface.

Mindquarry is open source (Mozilla Public License) and uses many of the great open source products of the Apache Software community, most notably Apache Cocoon, Apache Jackrabbit, Apache Lucene and Apache Maven. Additionally Subversion, DocBook and Dojo are used intensively in Mindquarry. So my thanks goes to the open source community, especially to Jeremy and Ellen who helped us getting up to speed with open source development.

Some days ago I wrote about Hudson, a continuous integration server:

It is not as feature rich as some other continuous integration systems listed on the Continuous Integration Server Feature Matrix, it's Maven integration is not as deep as Continuum's but it does its job, is easy to setup and offers a clean and friendly user interface.

At least for the Maven 2 support I have to revise my statement. With Hudson 1.78 released yesterday there is experimental Maven 2 support. Kohsuke Kawaguchi, creator of Hudson writes:

Since I rely more on Maven2, I wanted more Maven2 support in Hudson. It can already build a Maven project, but it didn't natively understand Maven — so you still have to specify what jar files to archive, what javadoc to copy, what builds to run once your build is finished, etc. But since much of the information is in POM already, I thought it would be really nice if Hudson can understand POM. So I started working on this some time ago, and it's finally getting into a presentable shape. The newly posted Hudson 1.78 is the first version with this feature enabled.

One thing I like most is the relatively loose integration with Maven. In Continuum for example you have to upload a pom.xml and if this pom does not include properly configured version control system links, you cannot start a build. With Hudson you still have the opportunity to start with an underspecified pom and a Subversion URL and your sceduled build is prepared in minutes.

Mark Phippard is developer of Subclipse, an Eclipse-integrated GUI for the version control system Subversion. In two recent blog entries he describes how Subversion clients can improve the usability of Subversion by leveraging the properties feature of Subversion.

• Enhanced Support for Branches and Tags in Subclipse is a Subclipse-only-feature that uses properties to track creation and location of branch and tag directories. The relevant property is called subclipse:tags and has a simple format that designates location, revision of creation (important for merging) and type.
• Integrating Subversion with your Issue Tracking System describes how subversion properties can be used to include issue tracking ids in a standard manner in Subversion's commit logs.

From my point of view a must-read for everyone that develops tools with Subversion-support.