Xtext 2.11 Beta 1 is here!

The Xtext 2.11 release has been rescheduled for January 24th 2017, as we underestimated the amount of work in front of us and overestimated the amount of time we could spend. Today’s beta release is merely a sanity check to ensure that we can still build a complete SDK update site. Don’t use it in production, also we will modify and enhance the code base further in the coming weeks until the release. That said, if you could spend a couple of minutes to install it and check how it behaves in your environment, don’t hesitate to submit a GitHub issue if you find a potential problem! Update site: https://download.eclipse.org/modeling/tmf/xtext/updates/milestones Also please note that we haven’t done any beta-specific release for the maven/gradle artifacts. Please use the SNAPSHOT releases. This post describes the most important changes that we’ve been working on so far.

Xtext Core

The Xtext project is evolving! In order to make Xtext more manageable and maintainable in the long run, the code base at eclipse/xtext has been split up into several parts, e.g. xtext-core, xtext-eclipse, and xtext-web. The ideas behind this new structure have been described recently. Of course, these structural changes won’t be noticeable by Xtext users, i.e. you will find the released bundles at the same locations as before. There will also be a talk on this matter at next week’s EclipseCon.

Language Server Protocol

Microsoft’s Language Server Protocol that was developed in the context of their new editor VS Code has been a perfect match for Xtext. The LSP defines how a generic editor client can use language specific services from a reusable language server. If you’ll be at EclipseCon Europe next week, you might want to check out our session about the language server protocol as well as the experiments of the Orion team. In a previous post I described the basic steps to combine your DSL with a Language Server Protocol client. The Xtext Language Server implementation shipped with today’s beta version already includes most of the features defined by the protocol. You can expect a complete implementation with the final 2.11 version.

The Grammar Language

There is a new feature in the Xtext Grammar Language: Annotations. They may be placed in front of any kind of rule. While this provides a very generic mechanism to add additional meaning to a rule, our primary use case for now is @Override. It serves the same purpose as in Java, but for rules instead of for methods. Example:

grammar pkg.EnglishGreeting with org.eclipse.xtext.xbase.Xbase

generate english "http://www.xtext.org/english-1"

Greeting:
  'Hello' name=ID '!';

grammar pkg.NordicGreeting with pkg.EnglishGreeting

generate nordic "http://www.xtext.org/nordic-1"

Model:
  greetings+=Greeting*;

@Override
Greeting:
  'Moin' name=ID '!';

The language pkg.NordicGreeting inherits from the language pkg.EnglishGreeting. Also, the parser rule Greeting from NordicGreeting overrides the parser rule Greeting from EnglishGreeting. In previous Xtext versions, overriding happened implicitly based on rule name equality. As of now, it should be made explicit by placing an @Override annotation in front of the rule declaration. The advantage is that if somebody removes or renames the rule in the super language, there will be an error marker in the sub language. For reasons of backwards compatibility, a missing @Override annotation by default only creates a warning. We are thankful to NumberFour AG, whose sponsorship made this feature possible. You may have heard of one of their Xtext-based languages, N4JS, a statically typed version of JavaScript, which is in the process of becoming an Eclipse Project.

Serializer performance

We are also happy with the achieved massive performance improvements in the serializer. The serializer is Xtext’s component for turning models (aka ASTs) back into text based on the grammar. The resulting text is fully compliant with the syntax that you specified in your grammar. So the serializer is the antagonist of the parser, and thus sometimes also referred to as un-parser. Common use cases range from non-textual editors, such as diagram- or form-based editors to semantic modifications of the AST, for example during a QuickFix or refactoring. In real-world projects we observed speed improvements between a factor of 2.5 and 6. These achievements are the result of more than a week of profiling, optimization, and simplification. Thanks to ETAS who supported these improvements with their sponsorship.

For contributors

New build system

To better address multiple platforms, we re-implemented our build system using Gradle. Maven Tycho is only used for the projects that need Eclipse-P2 for dependency resolution. The Gradle build produces both Maven artifacts as well as OSGi bundles. To be more specific, the repositories xtext-core, xtext-lib, xtext-extras and xtext-idea are built with Gradle, while xtext-eclipse is built with Maven Tycho.

Build server

TypeFox is happy to provide a new Jenkins build server for the Xtext projects. The server is operated by the committers to allow them to optimally adapt it to their development workflows. The major improvements have been achieved via:

• The awesome GitHub Branch Source Plug-in. The plug-in automatically creates a new Jenkins Job for each new git branch.
• Job configuration via Jenkinsfile. This turned configuration into git-managed code and eliminated pages-long job-configuration-forms.
• Elimination of central artifact repositories for snapshot builds. There is no central repository the server publishes to. Instead, each job archives the repository it produced as its artifacts. Since there is a job for each git-branch, this gives us a repository for each git-branch. Of course, milestones and releases will be published to the usual central repositories.