Langium – the new language engineering tool

We are thrilled to announce a new open source project: Langium.

Langium is a language engineering tool with built-in support for the Language Server Protocol. Here are some key facts:

• Simple and direct integration with the VS Code extension API
• Implemented in TypeScript, runs in NodeJS
• Derives a parser and abstract syntax tree from a grammar declaration
• The parser is based on Chevrotain
• Grammar declaration language similar to Xtext
• High out-of-the-box functionality, high extensibility

Why?

A domain-specific language (DSL) enables domain experts to contribute automatically processable content without any programming knowledge. This is often referred to as low-code. The contributed content can describe anything like structural data, behavior, or a mixture of both. By plugging in code generators that operate on the DSL, you create a powerful tool chain that creates technical artifacts from high-level specifications, improves communication between engineers as well as non-technical stakeholders and boosts the overall efficiency.

The main goal of Langium is to lower the barrier of creating a DSL / low-code platform. We achieve this by providing a DSL that describes the syntax and structure of your language: the grammar language. Langium’s out-of-the-box functionality is based on the information extracted from a grammar declaration. The Xtext framework has proved this approach to be suitable both for rapid prototyping and large-scale applications.

How does it relate to Xtext?

Eclipse Xtext was started by TypeFox co-founder Sven Efftinge in 2008. It is written in Java and initially targeted the Eclipse Platform via its plug-in mechanism. Support for the Language Server Protocol (LSP) was added in 2016, a few months after Microsoft published the LSP specification. Despite its age, Xtext is still an excellent basis for building languages and related tools with a Java technology stack. However, we have seen the VS Code extension API become increasingly relevant in the past years, not only for VS Code itself, but also for other tools supporting that format such as Eclipse Theia.

Langium enables language engineering in TypeScript, the same technology that is used for VS Code extensions. In contrast, building a tool that uses an Xtext-based language server with VS Code or Theia means creating a hybrid technology stack with some parts implemented in Java and others in TypeScript. Development and maintenance of such a mixed code base is more demanding for the involved engineers, and long-term maintenance is likely more difficult compared to Langium’s coherent technology stack.

Xtext is heavily based on the Eclipse Modeling Framework (EMF). This can be an advantage if you want to integrate with other Eclipse modeling tools (e.g. Sirius), but it can also be a burden due to its complexity. Langium uses the simplest possible solution to describe the parsed contents of a text document, called the abstract syntax tree (AST): TypeScript interfaces. By relying on the built-in language constructs of TypeScript, we avoid the additional abstraction layers and steep learning curve of a modeling framework.

In short: we want to keep the concepts that have made Xtext successful, but lift them onto another platform.

Let me try it!

Langium offers a Yeoman generator to create a new language extension. The only prerequisite for the following terminal commands is NodeJS version 14 or higher.

1. Install Yeoman and the Langium extension generator.

   npm install -g yo generator-langium
   

2. Run the generator and answer a few questions.

   yo langium
   

3. Open the new folder in VS Code (replace hello-world with the extension name you chose).

   code hello-world
   

4. Press F5 to launch the extension in a new Extension Development Host window.

5. Open a folder, create a file with your chosen file name extension (.hello is the default), and see that validation and completion (ctrl+space) works:

Follow the instructions in langium-quickstart.md (in your extension folder) to go further.

Features and roadmap

Langium is available as npm package. For its first release 0.1.0 we focused on laying the groundwork: grammar language, parser, AST construction, dependency injection, scope computation and linking. The most essential language services are included as well:

• Completion (content assist)
• Validation (diagnostics / problems)
• Go to definition
• Find references
• Document highlights (mark occurrences)
• Document symbols (list all symbols in a file)

We will work on further functionality in the upcoming months:

• Indexing and cross-file linking
• Testing utility for writing tests of your language
• Default implementations for more LSP services
• Multi-language packages
• API stability and documentation

TypeFox ❤️ Langium

We are determined to bring language engineering to the next level with Langium. This is fully in line with our general strategy of innovating in our key areas of expertise and at the same time providing support for Xtext and other tools we have created in the past.

Please give Langium a try and send us feedback on GitHub:

• Discussions for questions and discussion
• Issues for bug reports and feature requests