Xtext Core – less is more

In this post I want to give a short update of what we’ve been doing in Xtext and what the future plans are. As you probably know, Xtext has been around for a couple of years growing into a very mature framework for implementing full blown programming languages like Xtend as well as simpler more focussed domain specific languages. One of the big features is the additional tool support that the framework provides. We started out with Eclipse support and now added support for IntelliJ IDEA and the web editors Ace, CodeMirror and Orion.

Divide & conquer

As you can imagine supporting a full language development framework with tool support for different editors and IDEs means we need to maintain a lot of code and dependencies. Also, the different IDEs use different build systems which doesn’t make it easier. So far everything was developed in one gigantic repository. For collaborators and users it was very hard to get into it and identify the different parts. To ensure a more sustainable future for Xtext and especially the core component, which is platform (editor/IDE) independent, we have split up the repository into individual ones and reimplemented the build system. The goal was to end up with coherent, easy to understand sub projects. Xtext Core now contains only the basic framework for both runtime and tool support, but without any tool specific dependencies. It uses Gradle as the build system, which we tried to use where possible (only the Eclipse bundles are built with Tycho). So you simply checkout and call

./gradlew build

Language Server Protocol

If you follow this blog or other public activities from us, you might have noticed that we are actively pushing the Language Server Protocol initiative. So in addition to the restructuring this is the second ingredient to a smaller more coherent Xtext. The LSP is an editor and language agnostic protocol that can be used between any editor and any language for advanced language support like content assist, find references and so on. The LSP is the perfect match for Xtext, and the support of it will be the main new feature in the upcoming version (2.11). For the future this means that we are not going to add more sub projects like “xtext-coolneweditor” but focus on the core and let the tools focus on the proper implementation of LSP. I see this as a very important scope adjustment for a sustainable life of the project. Today, VSCode is the only editor that fully supports the protocol, but other teams are working on support for further platforms, too! Especially in the Eclipse community Che, Orion and the classic Eclipse IDE are adopting it.

By supporting the LSP your Xtext language will run in these editors without further ado. It will take some time until the LSP support in Eclipse catches up with the native Eclipse integration of Xtext. Until then (and probably beyond) we will maintain the specific platform support. So if you are nearly as excited about this as I am, you could try it out today already (there is still work ahead of us). Miro published a tutorial about it a couple of days ago.

Release in January

Xtext 2.11 is going to be released in January 2017. It will be fully API compatible to earlier versions, it is just built with Gradle and developed in smaller more cleanly separated modules. And of course you only need the Core to run in VSCode and other tools through the Language Server Protocol! P.S: I will give two sessions at EclipseCon Europe related to this. One is about the LSP and the other is about Xtext Core and the vision behind it. Looking forward to see many of you in Ludwigsburg!