In the upcoming Jetty 7.6.3 and 8.1.3 (due in the next days), the Jetty-SPDY module has been enhanced with support for prioritized streams and for SPDY push (although the latter only available via the pure SPDY API), and we have fixed a few bugs that we spotted and were reported by early adopters.
Also, we are working on making really easy for Jetty users to enable SPDY, so that the configuration changes needed to enable SPDY in Jetty will be minimal.

After these releases we will be working on full support for SPDY/3 (currently Jetty-SPDY supports SPDY/2, with some feature of SPDY/3).
Browsers such as Chromium and Firefox are already updating their implementations to support also SPDY/3, so we will soon have support for the new version of the SPDY protocol also in the browsers.

Stay tuned !

SPDY is a fundamental rethink of how HTTP is transported over the internet, based on careful analysis of the interaction between TCP/IP, Browsers and web page design .  It does not entirely replace HTTP (it still uses HTTP GET’s and POST’s), but makes HTTP semantics available over a much more efficient wire protocol. It also opens up the possibility of new semantics that can be used on the web (eg server push/hint).  Improved latency, throughput and efficiency will improve user experience and facilitate better and cheaper services in environments like the mobile web.

When is the revolution?

So when is SPDY going to be available?  It already is!!! The SPDY protocol is deployed in the current Chrome browsers and on the Amazon Kindle, and it is optionally supported by firefox 11.  Thus it is already on 25% of clients and will soon be over 50%. On the server side, Google supports SPDY on all their primary services and Twitter switched on SPDY support this month.  As the webs most popular browsers and servers are talking SPDY, this is a significant shift in the way data is moved on the web.   Since Jetty 7.6.2/8.1.2, SPDY is supported in  Jetty and you can start using it without any changes to your web application!

Is it a revolution or a coup?

By deploying SPDY on it’s popular browser and web services, Google has used it’s market share to make a fundamental shift in the web (but not as we know it)!  and there are some rumblings that this may be an abuse of Google’s market power.  I’ve not been shy in the past of pointing out google’s failings to engage with the community in good faith, but in this case I think they have done an excellent job.  The SPDY protocol has been an open project for over two years and they have published specs and actively solicited feedback and participation.  More over, they are intending to take the protocol to the IETF for standardisation and have already submitted a draft to the httpbis working group.   Openly developing the protocol to the point of wide deployment is a good fit with the IETF’s approach of “rough consensus and working code“.

Note also that Google are not tying any functionality to SPDY, so it is not as if they are saying that we must use their new protocol or else we can’t access their services.  We are free to disable or block SPDY on our own networks and the browsers will happily fallback to normal HTTP.  Currently SPDY is a totally transparent upgrade to the user.

Is there a problem?

So why would anybody be upset about Google making the web run faster?  One of the most significant changes in the SPDY protocol, is that all traffic is encrypted with TLS. For most users, this can be considered a significant security enhancement, as they will no longer need to consider if a page/form is secure enough for the transaction they are conducting.

However, if you are the administrator of a firewall that is enforcing some kind of content filtering policy, then having all traffic be opaque to your filters will make it impossible to check content (which may be great if you are a dissident in a rogue state, but not so great if you are responsible for a primary school network).  Similarly, caching proxies will no longer be able to cache shareable content as it will also be opaque to them, which may reduce some of the latency/throughput benefits of SPDY.

Mike Belshe, who has lead the development of SPDY, points out that SPDY does not prevent proxies, it just prevents implicit (aka transparent) proxies.  Since SPDY traffic is encrypted, the browser and any intermediaries must negotiate a session to pass TLS traffic, so the browser will need to give it’s consent before a proxy can see or modify any content.  This is probably workable for the primary school use-case, but no so much for the rouge state.

Policy or Necessity?

There is nothing intrinsic about the SPDY protocol that requires TLS, and there are versions of it that operate in the clear.  I believe it was a policy rather than a technical decision to required TLS only. There are some technical justification by the argument that it reduces round trips needed to negotiate a SPDY and/or HTTP connection,  but I don’t see that encryption is the only answer to those problems.  Thus I suspect that there is also a little bit of an agenda in the decision and it will probably be the most contentious aspect of SPDY going forward.  It will be interesting to see if the TLS-only policy survives the IETF process, but then I might be hard to argue for a policy change that benefits rogue states and less personal privacy.

Other than rouge states, another victim of the TLS-only policy is eas of debugging, as highlighted by Mike’s blog, where he is having trouble working out how the kindle uses SPDY because all the traffic is encrypted.  As a developer/debugger of a HTTP server, I cannot over stress how important it is to be able to see a TCP dump of a problematic session.  This argument is one of the reasons why the IETF has historically favoured clear text protocols.  It remains to be seen if this argument will continue to prevail or if we will have to rely on better tools and browser/servers coughing up TLS sessions keys in order to debug?

In Summary

Google and the other contributors to the SPDY project have done great work to develop a protocol that promises to take the web a significant step forward and to open up the prospects for many new semantics and developments.  While they have done this some what unilaterally, it has been done openly and with out any evidence of any intent other than to improve user experience/privacy and to reduce server costs.

SPDY is a great development for the web and the Jetty team is please to be a part of it.

]]> http://webtide.intalio.com/2012/03/jetty-spdy-is-joining-the-revolution/feed/ 9 http://webtide.intalio.com/2012/03/spdy-support-in-jetty/ http://webtide.intalio.com/2012/03/spdy-support-in-jetty/#comments Thu, 15 Mar 2012 10:50:52 +0000 simon http://webtide.intalio.com/?p=914 Continue reading →]]> SPDY is Google’s protocol that is intended to improve user experience on the web, by reducing the latency of web pages, sometimes up to a factor of 3. Yes, three times faster.

How does SPDY accomplish that ?

SPDY reduces roundtrips with the server, reduces the HTTP verboseness by compressing HTTP headers, improves the utilization of the TCP connection, multiplexes requests into a single TCP connection (instead of using a limited number of connections, each serving only one request), and allows for server to push secondary resources (like CSS, images, scripts, etc.) associated with a primary resource (typically a web page) without incurring in additional round-trips.

Now, the really cool thing is that Jetty has an implementation of SPDY (see the documentation) in the newly released 7.6.2 and 8.1.2 releases.
Your web applications can immediately and transparently benefit of many of the SPDY improvements without changes, because Jetty does the heavy lifting for you under the covers.

With Chromium/Chrome already supporting SPDY, and Firefox 11 supporting it also (although it needs to be enabled, see how here), more than 50% of the web browsers will be supporting it, so servers needs to catch up, and where Jetty shines.

The Jetty project continues to foster innovation by supporting emerging web protocols: first WebSocket and now SPDY.

A corollary project that came out from the SPDY implementation is a pure Java implementation of the Next Protocol Negotiation (NPN) TLS Extension, also available in Jetty 7.6.2 and 8.1.2.

To prove that this is no fluke, we have updated Webtide’s website with Jetty’s SPDY implementation, and now the website can be served via SPDY, if the browser supports it.

We encourage early adopters to test out Jetty’s SPDY and feedback us on jetty-dev@eclipse.org.

Enjoy !

This release updates the embedded Jetty to jetty-7.6.0.RC4, although the majority of the changes have been to the Console, which is a webapp that allows you to interact with your Android device from a remote browser.

Higlights include:

• pagination of large data sets such as Contacts and Media thumbnails (images, videos)
• re-implementation of generated content as json & ajax REST
• ability to cause the device to ring (helpful for finding it around the house!)
• ability to show current location of the Android device on Google maps , or track its location on the map as the device moves.

Here’s a screenshot showing tracking my phone as it moves from the Sydney Opera House to Fort Denison on Sydney Harbour:

Tracking phone via i-jetty console webapp

Enjoy.

The CometD project leverages the Jetty WebSocket implementation to its maximum, to achieve great scalability and minimal latencies.

Until now, however, support for WebSocket over SSL was lacking in Jetty.

In Jetty 7.6.x a redesign of the connection layer allows for more pluggability of SSL encryption/decryption and of connection upgrade (from HTTP to WebSocket), and these changes combined allowed to implement very easily WebSocket over SSL.

These changes are now merged into Jetty’s master branch, and will be shipped with the next version of Jetty.

Developers will now be able to use the wss:// protocol in web pages in conjunction with Jetty on the server side, or just rely on the CometD framework to forget about transport details and always have the fastest, most reliable and now also confidential transport available, and concentrate in writing application logic rather than transport logic.

WebSocket over SSL is of course also available in the Java WebSocket client provided by Jetty.

Enjoy !

Web applications often need to access resources residing on different servers, making the request to access those resources a cross origin request and therefore subject to the same origin policy.

Fortunately, all modern browsers implement the Cross Origin Resource Sharing (CORS) specification, and with the support of Jetty‘s Cross Origin Filter, it’s a breeze to write applications that allow cross origin resource sharing.

That is, all modern browsers apart Internet Explorer 8 and 9.

Without CORS support, CometD fallbacks to another Comet technique known as JSONP.
While JSONP is much less efficient than a CORS request, it guarantees the CometD functionality, but it’s 2011 and JSONP should be a relic of the past.

Microsoft’s browsers have another JavaScript object that allows to make cross origin request: XDomainRequest.

Unfortunately this object is non-standard, and it is not, in general, supported by the JavaScript toolkits on which CometD relies for the actual communication with the server.
I cannot really blame toolkits authors for this lack of support.

However, I recently found a way to make XDomain request work with CometD 2.4.0 and the Dojo toolkit library.

The solution (see this blog post for reference) is the following:

Add this code to your JavaScript application:

dojo.require("dojox.io.xhrPlugins");
...
dojox.io.xhrPlugins.addCrossSiteXhr("http://<crossOriginHost>:<crossOriginPort>");

What remains is to configure CometD with the crossOriginHost:

dojox.cometd.configure({
    url: "http://<crossOriginHost>:<crossOriginPort>"
});

The last glitch is that XDomainRequest does not seem to allow to send the Content-Type HTTP header, so all of the above will only work in CometD 2.4.0.RC1 or greater where this improvement has been made.

I do not particularly recommend this hack, but sometimes it’s the only way to support cross origin requests for the obsolete Internet Explorers.

How to Run


mvn jetty:run-forked

That will kick off a Jetty instance in a brand new jvm and deploy your unassemabled webapp to it. The forked Jetty will keep on running until either:

• you execute a mvn jetty:stop (in another terminal window)
• you <cntrl-c> the plugin

The plugin will keep on executing until either:

• you stop it with a <cntrl-c>
• the forked jvm terminates

NOTE: I’m interested in obtaining feedback about the lifecycles of the plugin and the forked Jetty. Is the lifecycle linkage that I’ve implemented the way you want to use it? Do you want the forked jvm to continue on, even if the plugin exits? Please post your input to the Jetty list at jetty-users@eclipse.org.

How to Configure

You need a few different configuration parameters from the usual jetty:run ones. Let’s look at an example:

     <plugin>
        <groupId>org.mortbay.jetty</groupId>
        <artifactId>jetty-maven-plugin</artifactId>
        <version>7.5.2.v20111006</version>
        <configuration>
          <stopPort>8087</stopPort>
          <stopKey>foo</stopKey>
          <jettyXml>src/main/config/jetty.xml</jetty.xml>
          <contextXml>src/main/config/context.xml</jetty.xml>
          <contextPath>/foo</contextPath>
          <tmpDirectory>${project.build.directory}/tmp</tmpDirectory>
          <jvmArgs>-verbose:gc -Xmx80m</jvmArgs>
        </configuration>
      </plugin>

You need to specify the stopKey and stopPort so that you can control the forked Jetty using the handy maven goal mvn jetty:stop.

You can use the jettyXml parameter to specify a comma separated list of jetty xml configuration files that you can use to configure the container. There’s nothing special about these config files, they’re just normal jetty configuration files. You can also use this parameter with the jetty:run goal too.

The contextXml parameter specifies the location of a webapp context xml configuration file. Again, this is a normal jetty context xml configuration file. You can also use this with the jetty:run goal too, either in conjunction with, or instead of, the <webAppConfig> parameter (which configures the webapp right there in the pom). As the jetty:run-forked goal does NOT support the <webAppConfig> element, you MUST use contextXml if you need to configure the webapp.

The contextPath parameter specifies the context path at which to deploy the webapp. You can use this as a simple shortcut instead of the contextXml parameter if you have no other configuration that you need to do for the webapp. Or, you can specify both this AND the contextXml parameter, in which case the contextPath takes precedence over the context path inside the context xml file.

tmpDirectory is the location of a temporary working directory for the webapp. You can configure it either here, or in a contextXml file. If specified in both places, the tmpDirectory takes precedence.

With the jvmArgs parameter, you can specify an arbitrary list of args that will be passed as-is to the newly forked jvm.

There’s also the same parameters as the mvn jetty:run goal:

• skip – if true the execution of the plugin is skipped
• useTestScope – if true, jars of <scope>test</scope> and the test classes are placed on the webapp’s classpath inside the forked jvm
• useProvidedScope – if true, jars of <scope>provided</scope> are placed on the container’s classpath inside the forked jvm
• classesDirectory – the location of the classes for the webapp
• testClassesDirectory – the location of the test classes
• webAppSourceDirectory – the location of the static resources for the webapp

Also, just like the mvn jetty:run case, if you have dependencies that are <type>war</type> , then their resources will be overlaid onto the webapp when it is deployed in the new jvm.

However, recently a problem was reported by the BlastChat guys: with Opera, long-polling requests were strangely disconnecting and immediately reconnecting. This problem was only happening if the long poll request was held by the CometD server for the whole duration of the long-polling timeout.

Reducing the long-polling timeout from the default 30 seconds to 20 seconds made the problem disappear.

This made me think that some other entity had a 30 seconds timeout, and it was killing the request just before it had the chance to be responded by the CometD server.

Such entities may be front-end web servers (such as when Apache Httpd is deployed in front of the CometD server), as well as firewalls or other network components.
But in this case, all other major browsers were working fine, only Opera was failing.

So I typed about:config in Opera’s address bar to access Opera’s configuration options, and filtered with the keyword timeout in the “Quick find” text field.
The second entry is “HTTP Loading Delayed Timeout” and it is set at 30 seconds.

Increasing that value to 45 seconds made the problem disappear.

In my opinion, that value is a bit too aggressive, especially these days where Comet techniques are commonly used and where WebSocket is not yet widely deployed.

The simple workaround is to set the CometD long poll timeout to 20-25 seconds as explained here, but it would be great if Opera’s default was set to a bigger value.

With the upcoming CometD 2.4.0 release, one of the biggest changes is the implementation of a WebSocket transport for both the Java client and the Java server.
The WebSocket protocol is finalizing at the IETF, major browsers all support various draft versions of the protocol (and Jetty supports all draft versions), so while WebSocket is slowly picking up, it is interesting to compare how WebSocket behaves with respect to HTTP for the typical scenarios that use CometD.

We conducted several benchmarks using the CometD load tools on Amazon EC2 instances.

HTTP Benchmark Results

Below you can find the benchmark result graph when using the CometD long-polling transport, based on plain HTTP.

Differently from the previous benchmark, where we reported the average latency, this time we report the median latency, which is a better indicator of the latencies seen by the clients.
Comparison with the previous benchmark would be unfair, since the hosts were different (both in number and computing power), and the JVM also was different.

As you can see from the graph above, the median latency is pretty much the same no matter the number of clients, with the exception of 50k clients at 50k messages/s.
The median latency stays well under 200 ms even at more than 50k messages/s, and it is in the range of 2-4 ms until 10k messages/s, and around 50 ms for 20k messages/s, even for 50k clients.

The result for 50k clients and 50k messages/s is a bit strange, since the hosts (both server and clients) had plenty of CPU available and plenty of threads available (which rules out locking contention issues in the code that would have bumped up threads use).

Could it be possible that at that message rate we hit some limit of the EC2 platform ? It might be possible and this blog post confirms that indeed there are limits in the virtualization of the network interfaces between host and guest. I have words from other people who have performed benchmarks on EC2 that they also hit limits very close to what the blog post above describes.

In any case, one server with 20k clients serving 50k messages/s with 150 ms median latency is a very good result.

For completeness, the 99th percentile latency is around 350 ms for 20k and 50k clients at 20k messages/s and around 1500 ms for 20k clients at 50k messages/s, and much less–quite close to the median latency–for the other results.

WebSocket Benchmark Results

The results for the same benchmarks using the WebSocket transport were quite impressive, and you can see them below.

Note that this graph uses a totally different scale for latencies and number of clients.
Whereas for HTTP we had a 800 ms as maximum latency (on the Y axis), for WebSocket we have 6 ms (yes you read that right); and whereas for HTTP we somehow topped at 50k clients per server, here we could go up to 200k.
We did not merge the two graphs into a single one to avoid that the WebSocket resulting trend lines were collapsed onto the X axis.

With HTTP, having more than 50k clients on the server was troublesome at any message rate, but with WebSocket 200k clients were stable up to 20k messages/s. Beyond that, we probably hit EC2 limits again, and the results were unstable–some runs could complete successfully, others could not.

• The median latencies, for almost any number of clients and any message rate, are below 10 ms, which is quite impressive.
• The 99th percentile latency is around 300 ms for 200k clients at 20k messages/s, and around 200 ms for 50k clients at 50k messages/s.

We have also conducted some benchmarks by varying the payload size from the default of 50 bytes to 500 bytes to 2000 bytes, but the results we obtained with different payload size were very similar, so we can say that payload size has a very little impact (if any) on latencies in this benchmark configuration.

We have also monitored memory consumption in “idle” state (that is, with clients connected and sending meta connect requests every 30 seconds, but not sending messages):

• HTTP: 50k clients occupy around 2.1 GiB
• WebSocket: 50k clients occupy around 1.2 GiB, and 200k clients occupy 3.2 GiB.

The benefits of WebSocket being a lighter weight protocol with respect to HTTP are clear in all cases.

Conclusions

The conclusions are:

• The work the CometD project has done to improve performances and scalability were worth the effort, and CometD offers a truly scalable solution for server-side event driven web applications, for both HTTP and WebSocket.
• As the WebSocket protocol gains adoption, CometD can leverage the new protocol without any change required to applications; they will just perform faster.
• Server-to-server CometD communication can now be extremely fast by using WebSocket. We have already updated the CometD scalability cluster Oort to take advantage of these enhancements.

Appendix–Benchmark Details

The server was one EC2 instance of type “m2.4xlarge” (67 GiB RAM, 8 cores Intel(R) Xeon(R) X5550 @2.67GHz) running Ubuntu Linux 11.04 (2.6.38-11-virtual #48-Ubuntu SMP 64-bit).

The clients were 10 EC2 instances of type “c1.xlarge” (7 GiB RAM, 8 cores Intel Xeon E5410 @2.33GHz) running Ubuntu Linux 11.04 (2.6.38-11-virtual #48-Ubuntu SMP 64-bit).

The JVM used was Oracle’s Java HotSpot(TM) 64-Bit Server VM (build 21.0-b17, mixed mode) version 1.7.0 for both clients and server.

The server was started with the following options:

-Xmx32g -Xms32g -Xmn16g -XX:-UseSplitVerifier -XX:+UseParallelOldGC -XX:-UseAdaptiveSizePolicy -XX:+UseNUMA

while the clients were started with the following options:

-Xmx6g -Xms6g -Xmn3g -XX:-UseSplitVerifier -XX:+UseParallelOldGC -XX:-UseAdaptiveSizePolicy -XX:+UseNUMA

The OS was tuned for allowing a larger number of file descriptors, as described here.

This is a major release that brings in a few new Java API (see this issue) – client-side channels can now be released to save memory, along with an API deprecation (see this issue) – client-side publish() should not specify the message id.

On the WebSocket front, the WebSocket transports have been overhauled and made up-to-date with the latest WebSocket drafts (currently Jetty implements up to draft 13, while browsers are still a bit back on draft 7/8 or so), and made scalable as well in both threading and memory usage.

Following these changes, BayeuxClient has been updated to negotiate transports with the server, and Oort has also been updated to use WebSocket by default for server-to-server communication, making server-to-server communication more efficient and with less latency.

WebSocket is now supported on Firefox 6 through the use of the Firefox-specific MozWebSocket object in the javascript library.

We have performed some preliminary benchmarks with WebSocket; they look really promising, although have been done before the latest changes to the CometD WebSocket transports.
We plan to do a more accurate benchmarking in the next days/weeks.

The other major change is the pluggability of the JSON library to handle JSON generation and parsing (see this issue).
CometD has been long time based on Jetty’s JSON library, but now also Jackson can be used (the default will still be Jetty’s however, to avoid breaking deployed applications that were using the Jetty JSON classes).
Jackson proved to be faster than Jetty in both parsing and generation, and will likely to become the default in few releases, to allow gradual migration of application that made use of Jetty JSON classes directly.
The applications should be written independently of the JSON library used.
Of course Jackson also brings in its powerful configurability and annotation processing so that your custom classes can be de/serialized from/to JSON.

Here you can find the release notes.

Download it, use it, and report back, any feedback is important before the final 2.4.0 release.