Xtext Core – Less is More

In this post I want to give a short update of what we’ve been doing in Xtext and what the future plans are. As you probably know, Xtext has been around for a couple of years growing into a very mature framework for implementing full blown programming languages like Xtend as well as simpler more focussed domain specific languages. One of the big features is the additional tool support that the framework provides. We started out with Eclipse support and now added support for IntelliJ IDEA and the web editors Ace, CodeMirror and Orion.

Divide & Conquer

As you can imagine supporting a full language development framework with tool support for different editors and IDEs means we need to maintain a lot of code and dependencies. Also, the different IDEs use different build systems which doesn’t make it easier. So far everything was developed in one gigantic repository. For collaborators and users it was very hard to get into it and identify the different parts.

To ensure a more sustainable future for Xtext and especially the core component, which is platform (editor/IDE) independent, we have split up the repository into individual ones and reimplemented the build system. The goal was to end up with coherent, easy to understand sub projects. Xtext Core now contains only the basic framework for both runtime and tool support, but without any tool specific dependencies. It uses Gradle as the build system, which we tried to use where possible (only the Eclipse bundles are built with Tycho). So you simply checkout and call

xtext_2_11

A simplified overview of Xtext’s components

Language Server Protocol

If you follow this blog or other public activities from us, you might have noticed that we are actively pushing the Language Server Protocol initiative. So in addition to the restructuring this is the second ingredient to a smaller more coherent Xtext. The LSP is an editor and language agnostic protocol that can be used between any editor and any language for advanced language support like content assist, find references and so on.

The LSP is the perfect match for Xtext, and the support of it will be the main new feature in the upcoming version (2.11). For the future this means that we are not going to add more sub projects like “xtext-coolneweditor” but focus on the core and let the tools focus on the proper implementation of LSP. I see this as a very important scope adjustment for a sustainable life of the project.

Today, VSCode is the only editor that fully supports the protocol, but other teams are working on support for further platforms, too! Especially in the Eclipse community Che, Orion and the classic Eclipse IDE are adopting it. By supporting the LSP your Xtext language will run in these editors without further ado. It will take some time until the LSP support in Eclipse catches up with the native Eclipse integration of Xtext. Until then (and probably beyond) we will maintain the specific platform support.

So if you are nearly as excited about this as I am, you could try it out today already (there is still work ahead of us). Miro published a tutorial about it a couple of days ago.

Release in January

Xtext 2.11 is going to be released in January 2017. It will be fully API compatible to earlier versions, it is just built with Gradle and developed in smaller more cleanly separated modules. And of course you only need the Core to run in VSCode and other tools through the Language Server Protocol!

P.S: I will give two sessions at EclipseCon Europe related to this. One is about the LSP and the other is about Xtext Core and the vision behind it. Looking forward to see many of you in Ludwigsburg!

By | 2016-11-30T08:34:08+00:00 August 2nd, 2016|DSLs, Eclipse, Language Server, Xtext|4 Comments

About the Author:

Sven is a passionate software developer. He has been doing language engineering and tool development for over 10 years now. He is the founder of Xtext.

4 Comments

  1. […] Die neue Architektur von Xtext / Quelle: TypeFox […]

  2. […] parts, e.g. xtext-core, xtext-eclipse, and xtext-web. The ideas behind this new structure have been described recently by Sven. Of course, these structural changes won’t be noticeable by Xtext users, i.e. you will find […]

  3. Dan Hemming January 22, 2017 at 14:38 - Reply

    I notice on YouTrack that you have asked JetBrains about their intentions towards the LSP. From your article it appears that over time you would want to deprecate support for eclipse and idea specific plugins in favour of a single LSP plugin. This makes a lot of sense from the perspective of Xtext maintainability and reach but I am left wondering if this would be limiting in some ways. As you know products like Idea distinguish themselves with fancy refactoring capabilities based on a structural understanding of the code. So far as I can tell LSP assumes a dumb client and to date all I have seen in VS Code by way of refactoring is rename refactoring. So, do you envisage LSP supporting the full richness of refactoring support available in high end IDEs or are you thinking that the reach of LSP is worth sacrificing the richness of specific IDEs?

    • Miro Spönemann January 26, 2017 at 08:48 - Reply

      The LSP is well extensible, and extending the protocol is also supported by the LSP4J implementation. This should allow to realize custom refactorings as you like. For sure the Java support in Idea will never be extracted to a separate server component, but given the amount of effort that is needed to properly integrate support for a language in Idea, adding generic LSP support would boost that platform quite a lot. Eclipse is already going that way.

Leave A Comment