Saxon diaries

A stylesheet conversion

Michael Kay — Thu, 20 Aug 2009 22:13:55 +0100

I'm doing what looks like a fairly simple project to upgrade an XSLT 1.0 stylesheet. It's XSLT 1.0 because it has to run in the browser, but fortunately it doesn't really need any 2.0 features. The old version of the stylesheet worked with XML files in format A, described by a DTD. The new version of the stylesheet has to produce the same HTML output, but this time from XML files in format B, described by an XSD schema...

Beyond Saxon 9.2

Michael Kay — Fri, 14 Aug 2009 20:16:10 +0100

So, Saxon 9.2 is finally out. I haven't had much chance to sit back and think yet - it's been a busy and stimulating week at the Balisage conference. Regulars will know that I don't do planning, either of dates or facilities: I prefer to keep the programme completely flexible, so that I can always find room to put new things in if the opportunity presents itself. But it's worth thinking a little bit about what's on the to-do list.

Analyzing dependencies in a class library: a use case for XSLT streaming

Michael Kay — Fri, 26 Jun 2009 18:01:10 +0100

IKVM (in version 0.40) now has much more complete coverage of the Java class library, but an unfortunate consequence of this is that the libraries have become rather large. To cut down the size of the library, we need to understand the dependencies between its parts. Read on to find how I'm doing this using the streaming facilities of the new Saxon 9.2 release...

Saxon repackaging

Michael Kay — Fri, 19 Jun 2009 22:44:31 +0100

Saxon9.2-EE is out on beta release, so the release process has finally started. It was beginning to feel as if the light at the end of the tunnel was an optical illusion. I decided not to change the licensing. GPL is just too distasteful, and other licenses that prevent use with commercial products just seem too burdensome. So here's what I've decided to do...

The GNU Public License

Michael Kay — Sun, 07 Jun 2009 17:47:55 +0100

I've been thinking carefully about whether there's a way to reduce the number of people "freeloading" on the back of Saxon - that is, selling commercial software that relies heavily on Saxon functionality without contributing anything in return. One potential way of doing that is switch in some way to a dual licensing approach: make application vendors choose between paying for a commercial license, and using the product under the GPL (GNU public license), which prevents them from creating "derivative works" unless those works are also licensed under the GPL (that is, made available as open source products). This is essentially the way MySQL works, and it seems to work well for them...

All nodes untyped

Michael Kay — Thu, 14 May 2009 09:34:08 +0100

Users sometimes imagine that just by running an application under Saxon-SA instead of Saxon-B, they will automatically get a performance boost. Sadly, this isn't the case. Sometimes Saxon-SA's more powerful optimizer will give a dramatic benefit, sometimes it will give none at all. In fact, sometimes if you move a workload to Saxon-SA without change, you see a performance regression. This is caused by the fact that Saxon-B can assume all nodes are untyped, whereas Saxon-SA can't make this assumption....

Repackaging Saxon

Michael Kay — Wed, 06 May 2009 13:32:31 +0100

I've been thinking for a while about some repackaging for Saxon, and I've now got to the point where I'm working out the fine detail and creating appropriate build files. The main idea is that instead of two products, Saxon-B and Saxon-SA, there will in future be three: Home Edition (HE), Professional Edition (PE), and Enterprise Edition (EE).

Streaming templates

Michael Kay — Mon, 23 Feb 2009 23:48:41 +0000

The streaming facilities in Saxon-SA have been proving very popular with those users who have very large documents to process. However, at present the only thing you can do in streaming mode is to split the document into a flat sequence of subtrees, and then process each subtree independently. That meets many needs, but not all. There are many simple tasks that can intrinsically be streamed despite the fact that they don't fit this model: for example, renaming all the elements in a document, or deleting all the NOTE elements. So I've started implementing "streaming templates", where the document is processed hierarchically in classic XSLT style by applying template rules to each node at every level....

Some threading tests

Michael Kay — Wed, 11 Feb 2009 18:20:43 +0000

Continuing the investigation of Tatu Saloranta's XSLTMark measurements, I was puzzled by the fact that on certiain tests, he was seeing a very different ratio between Saxon and XSLTC performance from the figures I was seeing.... I thought one theory worth testing was that XSLTC might be making better use of multiple processors than Saxon (which is essentially single-threaded).....

Another five-finger performance exercise

Michael Kay — Sat, 07 Feb 2009 13:16:52 +0000

Tatu Saloranta has started doing some work looking at the performance of XML parsers in an XSLT environment, and in that context he started running the old Datapower XSLTMark tests (no longer available, sadly) which are all 1.0 stylesheets, and the figures comparing Saxon and Xalan seemed to be worth investigating....

Bugs that don't crawl out of the woodwork

Michael Kay — Mon, 22 Dec 2008 15:30:05 +0000

I've said this before, but sometimes I'm truly amazed by some of the bugs I find: how on earth can they remain undetected for so long?

Ten Reasons why Saxon XQuery is Fast

Michael Kay — Sat, 13 Dec 2008 15:42:16 +0000

A paper under this title is published this month in a special issue of the IEEE Data Engineering Bulletin. Available online at:

http://sites.computer.org/debull/A08dec/saxonica.pdf

Of course, nearly all of what it says applies equally to XSLT. But for some reason, the academic community continues to be much more interested in XQuery.

Schema-Awareness and XMark performance

Michael Kay — Sun, 30 Nov 2008 21:27:41 +0000

When people ask what performance benefits they can expect from using schema-aware transformations and queries, I've often replied in a way that avoids setting expectations too high. Some queries can benefit significantly, others actually slow down because the extra cost of validating the input is not recovered by improvements in query execution speed. I've often stressed that the main benefit of schema-awareness is in the speed and ease of debugging and testing the query, not primarily in performance. But I've been taking another look at it, and I think I can probably start to be a bit more up-beat...

TEI Conference

Michael Kay — Tue, 11 Nov 2008 15:14:25 +0000

I spent a couple of days last week at the annual members meeting of the TEI (Text Encoding Initiative). It was good to meet so many Saxon users: a few familiar faces, rather more familiar names, and quite a few who introduced themselves as avid fans of the product, the book, or both...

XML Schema: allowing new lexical forms

Michael Kay — Thu, 23 Oct 2008 12:39:49 +0100

In a suggestion to the XML Schema Working Group, Axel Dahmen suggested defining a facet that allows the introduction of new lexical forms for existing data types. This would allow you, for example, to write a decimal value as "1,23", or a date as "12DEC2008", or a boolean as "yes". The Schema WG has already put out two "last call" drafts of the XSD 1.1 spec, so it's not really welcoming ideas for new features at the moment, but I thought I would take a look at this one independently...

Software Patents

Michael Kay — Thu, 02 Oct 2008 12:20:25 +0100

I have absolutely no doubt that software patents are unquestionably a bad thing. It simply isn't in the public interest to grant them. The only possible reason for doing so is to reward and encourage innovation, and there is no evidence that they have that effect, and plenty of evidence that they do exactly the opposite...

Just-in-Time Optimization

Michael Kay — Sat, 20 Sep 2008 21:44:53 +0100

Rereading my last post on compile-time performance, the thought leaps out at me: if I don't want to rely on users setting a switch to control whether optimization happens or not, then I should be doing lazy (or just-in-time) optimization...

More thoughts on compile-time performance

Michael Kay — Thu, 18 Sep 2008 19:11:59 +0100

Another user has sent me some samples of generated code (XSLT this time) with a request to take a look at compile-time performance issues. It takes about 3900ms to compile (5000ms under Saxon-SA), and the new switch to suppress optimization reduces this to about 1180ms. So optimization is the culprit here. So the switch will be useful here. But I hate performance improvements that depend on the user setting a switch, because 99% of users will never discover the switch is there. So, are there opportunities to improve the compile-time performance of the optimizer? You bet there are....

Compile-time Performance

Michael Kay — Tue, 16 Sep 2008 12:48:43 +0100

There was a support request a couple of days ago asking for a switch to turn optimization off. The user had a large query that was taking a long time to compile, and naturally felt that the cost could be reduced by omitting the optimization phase. But as always with performance, assumptions are dangerous until they have been verified by measurement...

Tweaking the TinyTree

Michael Kay — Wed, 10 Sep 2008 08:26:44 +0100

It's unusual to find anything these days that gives Saxon a 5% performance boost across the board, but I seem to have achieved that with some tweaks to the TinyTree data structure...

There's an R in the month

Michael Kay — Mon, 01 Sep 2008 10:57:15 +0100

September is upon us; summer is over; so it's time to start thinking about a new season. This blog has become rather dormant. Well, I'm going to try and reform, and post more regularly, and if I haven't got much to say then I'll just witter on about what I've been up to recently...

XQuery Update and Node Identity

Michael Kay — Thu, 14 Feb 2008 10:36:45 +0000

An interesting little spat on the XQuery internal mailing list over the last couple of days on the question of whether or not XQuery Update guarantees to preserve node identity. The spec says that it does. Someone remarked during a telcon that there were difficulties meeting this requirement in a SQL/XML environment. After the meeting I sent an email claiming that the assertion in the spec that updates preserve node identity is untestable and therefore ought to be removed.

Further progress on XQuery Update

Michael Kay — Wed, 02 Jan 2008 14:15:13 +0000

I've no got to the point where my XQuery Update implementation is passing almost all the published tests. That doesn't mean its complete, because the test suite doesn't cover certain areas, such as revalidation of updated documents. But it's looking good.

Progress on XQuery Update

Michael Kay — Wed, 19 Dec 2007 19:03:48 +0000

A progress report on the state of the XQuery Update implementation in Saxon, in particular, a list of things still to be done and some thoughts on how they might be tackled.

Implementing XQuery Update

Michael Kay — Tue, 11 Dec 2007 18:00:30 +0000

I've been asked a number of times whether I planned to implement XQuery Updates, and my answer has always been that I would do it when the time was right (i.e. when the spec seemed to be stable enough and when I had a suitable gap in my hectic schedule)...

Final Draft of XQJ

Michael Kay — Wed, 21 Nov 2007 16:29:45 +0000

A "Final Draft" of XQJ has been published (http://jcp.org/en/jsr/detail?id=225). There aren't many changes, but this time the specification is accompanied by actual interface definitions, and by a test suite. Swapping in the published interface definitions for my previous handmade versions proved no trouble - one or two discrepancies as regards which methods throw exceptions, and one or two methods that have had an extra argument added. Running the test suite is likely to be a bit more effort....

Transforming 20 Gigabytes

Michael Kay — Tue, 25 Sep 2007 13:20:28 +0100

My heart sank when I received an email yesterday from a Saxonica customer reporting a failure when running a transformation on a 20Gb source file (using the "streaming" capabilities described at http://www.saxonica.com/documentation/sourcedocs/serial.html.) It was apparently failing half way through with no error message of any kind. How do you go about debugging such a problem, especially when you're in a different continent from the client?

Document Projection

Michael Kay — Sat, 25 Aug 2007 10:34:13 +0100

Document Projection [1] is a technique invented by Amélie Marian and Jérôme Siméon designed to reduce the amount of memory occupied by a source document used by a query or transformation. The idea is to analyze the query and determine what nodes it is capable of accessing, and then while parsing and building the document, to filter out all those nodes that the query can never access. I decided to have a go at implementing this in Saxon...

A new XQJ draft

Michael Kay — Wed, 13 Jun 2007 22:54:32 +0100

XQJ is the proposed XQuery API for Java. A new draft of the XQJ specifications came out yesterday, with the psychologically significant version number of 0.9, and I spent today updating Saxon's implementation to conform. The spec can be downloaded from http://jcp.org/en/jsr/detail?id=225 ....

Path Expressions Revisited

Michael Kay — Fri, 27 Apr 2007 15:01:39 +0100

The core of XSLT and XQuery performance is the ability to evaluate simple path expressions efficiently. This area of Saxon has been untouched for some time, and I tend to assume it has been tuned to death. But I've always had a nagging feeling that the machinery created for evaluating a multi-step path expression might have room for improvement....