"Tree-sitter - a new parsing system for programming tools" by Max Brunsfeld

Channel:Strange Loop Conference · Length:38:37 · Published:2018-10-14 · Views:63511

Watch on YouTube

Summary

Max Brunsfeld introduces Tree-sitter, a parsing system for developer tools that offers incremental parsing and robust error recovery through a uniform C API. The talk explains the design, based on LR extended to GLR parsing.

Why HN recommends it

HN points to it as the best explanation of tree-sitter's design choices and as an accessible introduction to parser generators.

Highly recommend Max Brunsfield's strange loop talk if you want to learn more about the design choices behind tree-sitter — HN, 2024

Compilers, interpreters & program analysis · Testing, debugging & developer tools

Subtitles (excerpt)

so hi everyone my name is Max I work at GitHub on the Adam text editor team and um I want to tell you today about a project called treesitter that I've been working on for about four years now to try to improve on some of the core features of text editors uh so tree sitter started out as a as a hobby project for me uh before I even started working at GitHub but now it's it's part of a couple products that we we're making a GitHub and U so I'm going to tell you today uh explain what treesitter is and talk about how it's different from some other things you might have used and then I'll show you some of the things that we're building at GitHub with treesitter and then finally I'll talk about how treesitter works and some of the algorithms that it uses so first what is tree sitter um treesitter is a library for parsing source code it's written in C and C++ and it's designed to be used in applications like atom that work with code written in many different languages so you can use treit to parse files written in all different languages and produce syntax trees that all have the same interface but the really unique thing that treesitter does is what's called incremental parsing and that means that once you've parsed the file with treesitter you can then edit the file and

On Hacker News

Absolutely agreed, and copying from a comment I wrote last year: I think the fact that tree-sitter is dependency-free is worth highlighting. For context, some of my teammates maintain the OCaml tree-sitter bindings and often contribute to grammars as part of our work on Semgrep (Semgrep uses tree-sitter for searching code and parsing queries that are code snippets themselves into AST matchers). O… — ievans, 2024-03
I also enjoyed the Treesitter talk from 5 years ago by Max Brunsfeld [VIDEO] I'm currently building a query language whose grammar is very much inspired by Github's search syntax. I'm using Lezer, which is a GLR like Treesitter, so this talk learned me some parser generators (I've no formal CS education). Here's my grammar, a playground, and an example search query if anyone wants to play with it… — AlexErrant, 2023-12
It's great to see more tools adopting tree-sitter [1]. Having a (fast) single tool that can accurately parse most commonly used programming languages is incredibly useful, but it requires the maintenance of dozens of grammars, which is difficult without a large community effort. Hopefully increased adoption means more accurate parsers and support for even more languages. Tree-sitter powers syntax… — rtsao, 2020-10
For a really nice solution to the error message problem, see this recent strangeloop talk: [VIDEO] Basically it uses the parse tree disambiguation from the GLR parser to look for the most likely mistake the user made - it's very clever. — lemming, 2018-11
Parsing (use of rather than theory) matters as it affects my work. So I followed up. See [VIDEO] Tree sitter is based on LR parsing (see 23:30 in above video) extended to GLR parsing (see 38:30). I've had enough of fools on HN posting unverified crap to make themselves feel cool and knowledgeable (and don't kid yourself that you helped me find the right answer by posting the wrong one). Time I wi… — throwaway_pdp09, 2020-12

Related videos