GIMTEC By Llorenç Muntaner - 🎒 Intro to Linters, Parsers, and AST

Introduction

I would like to explain two sentences from the Eslint Readme:

ESLint uses Espree for JavaScript parsing.
ESLint uses an AST to evaluate patterns in code.

Linters were always a mystery to me. How can a tool read our code and tell us what’s wrong and right? What kind of dark magic is that?

That’s until I discovered the Abstract Syntax Tree.

Abstract Syntax Tree

The Abstract Syntax Tree (or AST for short) represents the code in a tree structure. This tree is built based on the syntax rules of the language and the tokenization of the code. The tokens are the small pieces that form the code. For example; a variable name is a token, a string, a keyword like def or function, even a symbol can be a token—like { or ;.

Let’s see a simple example:

const num = 10;

This code is represented by the following AST (following exactly the same structure as the library below for consistency):

All programming languages (and DSLs) have an AST representation used by many different tools to compile, interpret, transpile, or perform static analysis.

I don’t want this to become an article on AST; therefore, I recommend this article if you want to learn more. And also my previous article on programming language syntax.

Parsing

The act of creating the AST from the source code is called parsing. Therefore, we can understand now the first sentence we had in the Readme:

ESLint uses Espree for JavaScript parsing.

It means that ESLint uses Espree to get the AST of the code. So let’s play with the library to find out what it is:

Given the JS code in "test.js":

// test.js
const num = 10;

Let's use Espree to parse the code:

const espree = require('espree');
const fs = require('fs');
// File above
const fileContent = fs.readFileSync('./test.js');
const ast = espree.parse(fileContent.toString(), { ecmaVersion: 6, loc: true });
console.log(ast);

The tree printed by console.log(ast); is the same as the schema we saw earlier.

This is what we get:

I separated it into different screenshots to make it more readable. But if you want to check the whole object, here is the JSON in a gist.

These objects are how Espree decides to represent the tree structure. And this is what the sentence: “ESLint uses Espree for JavaScript parsing.” means.

Rule Evaluation

How about the second sentence?

ESLint uses an AST to evaluate patterns in code.

This sentence means that given the AST that Espree created, ESLint checks whether the rules are being followed. Let’s give an example: ”no-var”: “error”.

Can ESLint determine whether we use var and signal an error?

The answer is yes; the AST has this information. So, let’s change our code to var num = 10; and take a look at the AST.

As you can see, the object with VariableDeclaration has a property called kind, which before was const and now it’s var. Therefore, to find out whether the code follows the rule, we need to visit all the nodes with the type VariableDeclaration and check the value of the kind attribute.

Some rules might be more complex than others, but they all follow the same pattern: traversing the AST and checking whether the rule is followed.

Recap

So, after this explanation, I hope that you understand the sentences that we started at the beginning:

ESLint uses Espree for JavaScript parsing.
ESLint uses an AST to evaluate patterns in code.

I explained the concepts with Javascript, ESLint, and Espree. Yet, the pattern is the same for any linter in any programming language.

Further Readings

• Introduction to AST by the Twilio Team.
• ASTs are developed based on the syntax of programming languages. To learn more about syntax and programming languages, check my article on EBNF.
• To learn more about parsing. Check this article on Parsing in Java and download the free PDF.
• Espree is built on top of Acorn. A JS parser developed in JS 🤯.

If you like this post, consider sharing it with your friends on twitter or forwarding this email to them 🙈

Don't hesitate to reach out to me if you have any questions or see an error. I highly appreciate it.